What Level Programmer Are You? - <AUTHOR> http://www.whattofix.com/blog/archives/2012/02/what-level-prog.php

<comment><USER3> I disagree with the premise of this article. By trying to differentiate the different "programmers" by nothing other than their _skill_ , he is ignoring the fact that you simply shouldn't group together low-level chipset programmers with PHP programmers with LABview drag-and-drop-individual- components programmers with computer scientists who also happen to be programmers. These people are fundamentally different, and trying to say that they are all the same and only differ in "programming experience" is fundamentally missing the boat on these distinctions.

<comment><USER4> I agree completely. I work on high performance C API's. Most of the code I write is as close to the silicon as possible because of that. So where am I? Not level 7, because our OO design doesn't have the performance we need.

But I've never done anything in the web space, which means I don't actually qualify for level 1.

</comment></comment><comment><USER5> I think his levels are really stupid. I can write functional & OO code somewhat well ( _in dozens of languages (Java, ObjC, C++, Scheme, ...) - <USER2> 3-4 of them I'm more experienced but with the help of StackOverflow can do almost anything I want, given a few hours time_ ), read/write bash scripts, can (and do) write iOS/Mac GUI applications, can use PHP and Ajaxish things, have good experience with MongoDB/MySQL and a lot of other things. I use OS X Terminal.app regularly and have no problems doing tricks with bash (from piping to xargs'ing); _But_ , I spent almost a whole day a few months ago and couldn't install an OpenVPN server on a linux machine. I just couldn't. I knew about IPSec, and that I must create exceptions in the ipfw for this thing to work, etc. but it wouldn't work, and I didn't have a single clue how to fix that mess. So by his _test_ , I wouldn't even pass the first level...

You could be good at some aspects of programming and weak at other aspects. Programming is a multi-faceted art. You don't start at level 0 and move up; You learn different parts of it simultaneously.

</comment><comment><USER6> While this article doesn't help at all to classify programmers, it's interesting in that it gives a good insight into the author's approach to computer programming.

It's a very book-oriented, "teach yourself how to program computers, then program them" way of looking at the sport. It's what I'd expect to see from a CS major who'd never programmed until he hit university, or possibly a history-major-turned -programmer who learned his trade through those big red Wrox Press books with 24 authors on the cover. My personal path never intersected this curve at all.

I (and I suspect a lot of other folks) came up the Tinker route, being given an Apple II or C64 as a kid with no games on it and needing to find the shortest route to get games written so that you could play them. Every exposure to a new technology (with a new _thing_ that needs accomplishing) gets met with a new batch of playing around with it until it all makes sense, then on to the next thing. Windows programming, web development, database stuff, OOP, functional programming, and everything in between, but never with a plan or anything resembling any of the "stages" the author describes.

Even now, 35 years in, I can't find a place on that scale where I fit. I bet there are a lot of programmers out there in the same position.

<comment><AUTHOR> Author here.

The funny thing is not that you're wrong about me, it's how completely wrong you are. I'm self-taught, and started programming Commodore Pets under PetBasic with, woohoo, 4K of RAM and a cassette tape. Wrote my first contract program when I was 16, on an Apple II. I had written my first game a few months before that. I think I had four college-level programming classes in my early 20s. That's about it for formal training. I'm very much a tinkerer.

But you're right about the books. Being self-taught, I had to voraciously read, especially starting out. Since I ended up doing this for a living, and since everybody else had the advantage of a college degree, I felt very much under the gun: learn or starve. I read like a maniac.

I don't think people "fit" in the scale in any one spot. Different jobs require different skills. Much more interesting (to me) is if folks understand the levels and are able to work as needed in them. So far the comments look very much hit and miss, which was about what I expected.

Interesting analysis, though! I appreciate the feedback on how I sound.

</comment></comment><comment><USER7> I don't really get this stuff. Building stuff that's considered to be hard are actually the things I learned when starting out with programming. Things like a driver, a boot loader, a compiler aren't things that are harder than any other piece of software. Of course everything gets hard when you go beyond the basic stuff, but I never got why people that program in C++ or Java (usually) tend to be so arrogant when it comes to these things. Assembly isn't hard and building an interpreter or compiler also is pretty simple. The thing that's really hard when programming is the "maintenance" stuff. Be it a web application, an operating system kernel, a compiler, video game or an IRC client. Implementing the feature itself usually isn't that hard. Fixing bug is, finding the right balance between performance and simplicity is too. Oh and progressing without breaking API or ABI compatibility.

It usually not important what paradigm you follow. I myself like languages like Perl, where I can choose what way I want to go. Also functional programming languages like Haskell are awesome for building interpreters, but really I consider Assembly to probably be the easiest language there is and everything should start with it, especially C or C++ programmers, because it will prevent you from making assumptions that are wrong and prevent stupid code.

Also looking at different languages is fun and everything that is considered hard is usually lots of fun. Haskell and Assembly are the best examples.

About research. If you don't do the same thing over and over it's always a way of researching things. Most people building awesome stuff are people connecting things that haven't been connected by now.

What I want to say in short is that I don't see how these "levels" can ever fit to anyone. Also I don't get what paradigms and well, everything thing listed fits the term "level". It's about what I thought when I had even less of an idea of what programming is about. I am still exploring stuff and am a complete beginner. I never had a serious project. I just hack (as in writing bad code) some scripts together.

Disclaimer: I know I am a bad programmer, if you can even call me one at all. I have a very idealistic view on things.

<comment><USER8> "Building stuff that's considered to be hard are actually the things I learned when starting out with programming."

Maybe it's not harder than other things on the list, but the author is saying that this order is the most common progression for programmers in his experience.

</comment></comment><comment><USER9> I think Donald Knuth, who must be placed at the Mystery Level, never went through the functional phases.

<http://homepages.cs.ncl.ac.uk/m.j.bell1/blog/?p=49>

DK on the subject of functional programming: "Out of the ~250 programs I wrote last year, 2-3 would have benefited from being written in a functional style."

DK on the subject of functional programming: "With 1 or even 2 hands tied behind your back it's hard to do anything dangerous."

<comment><USER10> IMHO, when you reach this kind of math level programming, you do functional programming with indexed n-dimensional arrays.

</comment></comment><comment><USER11> I don't accept that there's one track.

I know scientific programmers who still use FORTRAN and write incredibly fast programs in a domain that's much harder to get things correct than you think.

On one level these people are "geniuses" but many of them would be completely lost if it came to object-oriented programming or relational databases.

As for the computer scientist, he's an odd animal.

I remember a time when I got a program that did something almost magical from a guy who's a household name in machine learning and CS instruction. It turned out this program crashed before getting into main()! On close inspection the problem was that he was allocating a huge array statically that he never used! This worked OK on the 64-bit monster he was using, but would kill any 32 bit machine.

My experience is that code written by computer scientists is very rarely suitable for commercial use because computer scientists usually aren't professional programmers. They get rewarded for writing papers instead of writing programs, so a succesful endpoint for them is producing some artifacts that would demonstrate how the program would have worked if it really worked.

</comment><comment><USER12> I think there should be a level "I believe I've reached a level where I can speak wisely about programming ability". Followed by the "Oh, oops. Now I see I STILL know nothing" level. ;)

<comment><AUTHOR> Of course, the problem here is that by making that statement, you are both trying to wisely speak about programming ability while still claiming it's mostly unknown to you. Both bases effectively covered. Score! :)

It's just a blog entry with observations. I make no claim on being an expert or creating the next standard. Just throwing some ideas out. I knew this would generate a bit of a stir on HN, so I thought it would be fun. What can I say? I like putting out ideas and having people throw rocks at them. If the criticism is poor, then I know I'm on to something.

Having said all that, I think there is a level of symbols, symbol assembly, and abstraction involved, even if I did a poor job of grouping them together for this entry. Writing your own language is dealing with a problem from about 3 or 4 meta levels away. It's nothing like simple scripting. There has to be some kind of natural level involved there.

EDIT: I restated the programming levels in a really short list posted to HN here: <http://news.ycombinator.com/item?id=3562422>

I think this might be a case where more words are worse. That is, by providing concrete examples it just gave people a bunch of stuff to complain about. Hopefully the follow-up put it in generic enough terms to be more tractable.

<comment><USER13> I'm sure those who have wrote complex scripts would probably disagree with you.

Do you know what writing complex scripts in equally complex environments entails? I sure don't, therefore it makes me sufficiently unqualified to determine "meta levels" (whatever that means) for that skill.

Unless you are deeply familiar with both, i'm not sure how you can say one is higher then the other.

<comment><USER14> Complex scripts?

I hit the "reply" button to tell you that no, no amount of "complex scripting" is as mentally challenging as developing a Turing complete language.

But maybe you mean scripts as any program? In that case, I agree that developing things like, I don't know, database engines and file systems and such are equal in complexity to developing a language.

I suppose the only caveat is that those things are usually teams -- sometimes huge teams. A language is often a 1-man or small team effort.

</comment></comment></comment></comment><comment><USER13> I personally don't like the idea of "levels", especially with programming. IMO it adds to the "I'm better then you cause I do xyz code" mindset which seems to get adopted more and more each day. It actually almost turns me off and makes me not want to participate in the community.

Perhaps that mindset is more common among those in the valley or other tech "hot" areas. But around here, Washington DC, if you know People Soft and SharePoint you can easily command a salary ~130k salary. Knowing how to write your own Ruby implementation or js library gets you pretty much nowhere.

/end My first HN comment :)

<comment><USER15> If you increase your skill/knowledge that doesn't hinder your ability to use PeopleSoft/SharePoint. It certainly expands your options. While you can get a good salary using those technologies, eventually they will be obsolete. When that happens, we'll start to see news stories posted on some future-HN about how older IT workers are being passed up for younger, fresher IT workers and how it is a biased industry. When in fact, its just a group of people who decided that sitting in their comfort zone doing their day-to-day was good enough and when it was time to move on, they couldn't find any other jobs that fit their particular niche they'd had for the last decade(s).

<comment><USER13> Or it could just be that the younger crowd is using the tech that is popular today, therefore they are in higher demand.

10 years from now some next best thing will replace Ruby and the cycle continues..

As in my original post, the issue I have is the concept of knowing how to write your own language being considered a HIGHER skill then someone knowing Sharepoint or visual basic in depth.

Who's right is it to say that my particular skill is better or higher then yours..

<comment><USER15> Knowing how hardware works, how languages are compiled/designed, how libraries are architected. It can only increase your knowledge of [insert technology here].

I think you're confusing "Levels" with A > B (A is better than B) when in reality the point is that: A comes after B. You must first understand B, then you can move on to the next level in your path of understanding programming.

</comment></comment></comment></comment><comment><USER16> I am always a bit perturbed by lists such as this.

I have worked with people that would be categorized as a "script kiddie" and the work they put out is far more useful than what has been done by someone who has a doctorate in computer science.

I'd much rather employ someone who can get stuff done rather than someone who knows everything yet does next to nothing. The man driving the delivery truck is a lot more useful than the worlds best nascar driver.

My level: Some days I wonder if I am even a level 1 by this list.

<comment><USER17> My advice to you: stop taking it so seriously. It is just meant to be a metaphor for one's skills evolving from the concrete as a beginner to acquiring the ability to think more abstractly as your skills improve.

</comment></comment><comment><USER18> Personally I see this as a permanent cycle where I'm on different levels for different things all the time. With some languages I'm on a high level, with others on a low one. Even within languages I know very well, there are parts I didn't use that much. Same for many other things like commandline tools, frameworks, api, databases. Often enough I go down on a lower level and for example work through a beginners tutorial just to have a reason to use a few things in Ruby on Rails which I don't use often enough in everyday work. Or simply to go through the cycle of setting up a developer machine with rvm, RoR, some gems, git, github. All this more as a kind of programming kata than because there would be any need.

</comment><comment><USER19> This same idea seems to crop up every month or so on HN, and it just seems like ego stoking. My friend is learning to program with a functional language. Does that automatically shoot him up to level 9?

<comment><USER20> Well it's part ego stroking and part just a normal wondering of how one stacks up to others who do the same thing. For me it's also part learning experience. For example, I know Carmack is a much better programmer than I am, but having never seen him program or his process it's hard for me to put it into concrete terms other than he's better. But what does better mean? Faster, more creative, more X? Knowing the details of better is how I can build a plan to improve.

I also liken the fascination with ranking to when I thought I was pretty fast running a 5 second 40yd dash. Of course I had read about or seen sub 5s of other people, but until I finally ran next to a D1 track athlete it was a foreign concept just how much faster he was. Once again, putting something into concrete terms helps people understand where they rank.

</comment></comment><comment><USER21> The levels in this article kept moving towards higher and higher levels of complexity (writing your own DSL), while in my experience, getting better at my job has resulted in writing code that's less and less complex.

I used to refactor things into inheriting models with overridable behavior, etc if it could save me from repeating myself. Now I just copy/paste code when it's appropriate, and it's way easier to maintain. Key word is appropriate.

<comment><USER22> I totally agree.

In most programming projects, the most important thing is maintainability, and that is relative to the skill of the maintainer.

If the maintainer is going to be junior developer, then writing complex OO design is not good, even if it makes the program more concise; if the maintainer is a business developer who is not good in math, then crafting non- trivial algorithms will throw him off, even if it means shorter program; if the maintainer is smart and willing to learn, then you can do whatever.

</comment></comment><comment><USER23> I think it's difficult to group up knowledge like this. I read through the levels and found that I had bits and pieces from different levels, meaning the whole system sort of failed me.

Levels work in a structured environment. Programming and the learning of programming is far from structured.

</comment><comment><USER24> Reminds me of the classic Programmer Competency Matrix: [http://www.indiangeek.net/wp- content/uploads/Programmer%20co...](http://www.indiangeek.net/wp- content/uploads/Programmer%20competency%20matrix.htm)

</comment><comment><USER25> I was curious about something a former boss told me. He had been a programmer before he had been a manager, and he said, "oh, you haven't written a compiler yet? When you write a compiler you graduate to a whole new level of programming."

I have learned Lisp and in that sense I have graduated to a new level, although I've still not used it enough to make macros an everyday joy. I still prefer JavaScript and I am going to learn some Ruby and some CoffeeScript.

But I have no idea what writing a compiler gets you, so if anyone wants to tell me about it, I'd love to hear.

</comment><comment><USER26> I think the most interesting element was the idea that as you become more abstract you get a target audience of those who are less abstract. Sometimes it seems like programming (with the start-up space in particular) is a kind of pyramid scheme where you advance to create nicer tools for the people who are less advanced. Of course there are plenty of other ways that start-ups and programming are _not_ like that, but it's a really interesting lens through which to view a lot of the news that is popular on HN.

</comment><comment><USER27> "Programming ability" absolutely does not fall onto a linear scale, it's an n-dimensional vector.

<comment><USER28> Programming ability == quality of mental model, IMHO. A mental model is essentially a map, and there are many types of maps.

More on this: <http://esr.ibiblio.org/?p=316>

</comment></comment><comment><USER29> I think in an effort to categorize himself, the OP has created layers of knowledge that only apply to him.

Personally, my expertise slides up and down all of these _types_ , proving you can't really nail this one down.

There are going to be those who write to automate, write for design sake, write for scale, etc. Even saying that is a bit of a misnomer because there are so many facits that incorporate one or many of those functions.

</comment><comment><USER30> Reading the comments, theres a lot of disagreement (for some valid reasons) but this list is getting more hate than it deserves. If you consider that the MAJORITY of programmers are essentially corporate code monkeys this is actually a very accurate list.

If I read this list and then apply its rules to the average programmer at fortune 500 companies it is almost perfectly accurate at measuring the least to most talented programmers I've worked alongside.

In the corporate world, the bulk of developers fall into one of the following groups: guys who can barely modify a custom report, guys who can write a report, guys who can create a basic CRUD class for reuse across reports, guys who can contribute to a complex application, guys who can design a complex application, and then the guys who are hired to create the tools/languages that are leveraged when creating complex applications.

There are obviously many areas of programming that do not fall under these definitions but the AVERAGE programming (i.e. average corporate developer) probably fits very neatly into one of these definitions.

</comment><comment><USER28> I would say I'm close to 7, with a lot of peeking ahead toward levels 8 and 9.

It's an interesting way to look at prowess. Instead of looking at specific languages/technologies, we look at the progression of understanding. Ie how rich, efficient, and effective the programmer's mental model is.

What's a good salary aspiration for someone at this level who's situated in Sweden? I am currently making nowhere NEAR the ludicrous fantasy sums (ie $100k+) that you Americans consider to be a human right but I'm thinking I could aim higher.

Maybe I could use this level-based thinking when discussing salaries with employers?

<comment><USER31> > _I am currently making nowhere NEAR the ludicrous fantasy sums (ie $100k+) > that you Americans consider to be a human right but I'm thinking I could aim > higher._

When comparing USA and €U salaries, you always have to factor in all the things you basically get for more-or-less free like complete medical care which you are paying ridiculously little for, way less than the amount of money you actually need in case of a serious illness - plus social insurance, more than twice as many vacation days than the average USA employee who just started out, plus way more restrictive labor laws, plus some sort of retirement money the government pays you (at least still for now) and paid sick leave. Then factor in the money USA employees have to spend to "keep up" with the on-average higher working hours - ever wondered why there are so many convenient services for practically everything in the USA that we Euros can only dream of? And ultimately you have to factor in costs of living in the valley or surroundings which are way higher than over here and way higher than in most other places in the USA.

Ultimately, I think this level of proficiency might be hard to sell in an interview because employers care about what you can actually do and if they want a Java or C genius, then they probably don't care that much that you could "write a library" but have no working experience in the languages they are requesting. But if you can plausibly sell them on your broader view and understanding of all things IT, ideally backed by some sort of study and degree, that might work but I am sure it would also depend on the position you are applying for.

You want to make close to or more than €100k as a programmer/it-expert in Euroland? Shouldn't be too very hard, either find a fantastic niche and work it and/or combine that niche with consulting work... I think the unfortunate truth is that unless you are some sort of coding rock star and/or become technical lead or something like that, just being good at writing lines of code in an arbitrary language isn't such a highly paid job anymore. Make that some obscure language for a very specific system, ideally serious-business system, somehow directly business/profit-related and all of a sudden you have leverage. A lot of the old host neckbeards are retiring or, excuse me, dieing out and I have to yet see a bank or stock exchange that runs completely without those old "legacy" systems and has successfully migrated 100% away from them. So, time to polish your COBOL and what-not skills, get some working experience there... I am sure you could find a LOT of happily paying customers with all sorts of software running on some host system and they no longer have anyone who knows what is going on. Or look into the myriads of software systems surrounding banks and stock exchanges in general. It's all just some software and a lot of it is pretty damn terrible but the prices customers have to pay for using it and for support is still ridiculous plus you have a niche with lots or a couple of customers pre-made for you.

If you like working as a consultant and carrying all the risks and benefits, then 5-figures a MONTH aren't a fantasy or wishful thinking but of course you are giving more than just your work and you are "paying" in other ways.

Just don't expect you will be making more than €100k just because you are so much better at LAMP than Johnny at age 16.

<comment><USER28> Thanks, great feedback.

What are some highly specialized niches that I could look into for consulting?

I actually don't care that much about money, in absolute terms. I care about maximizing my free time and maintaining a somewhat decent standard of life. Ideal scenario for me: a high hourly wage + the freedom to take on work in whatever chunks I prefer.

<comment><USER31> I listed some quick-shot ideas there, did you see them? Ultimately you will have to see and try for yourself how well they work and what other opportunities are there - scout the terrain yourself.

> _I care about maximizing my free time and maintaining a somewhat decent > standard of life. Ideal scenario for me: a high hourly wage + the freedom to > take on work in whatever chunks I prefer._

Most of this SCREAMS consulting, you could be making more a week than you are making a whole month now... but you have to put that into perspective. As a self-employed consultant, you will have to take care of all your taxes and insurances but at the same time you can effectively buy computers and certain other things MUCH cheaper (practically no value-added tax). You will have to find customers, projects or are lucky enough to find longer lasting maintenance gigs but YOU will have to do that. You will be your own boss and depending on your customers, your working hours might be extremely flexible but you will also have new and different responsibilities... taxes and insurances are one huge part of that, the other is that lovely thing known as "liquidity" that can break your neck faster than anything else. And when you are a consultant in some software-product niche, then chances are whoever makes that software will actually sign the deal with the customer and then pass it on to you but of course they will bite a nice chunk off first... but on the other hand this might also be a chance to get started and meet people and customers, use their name, reputation and connections.

Another thing is "taking on in whatever chunks I prefer"... generally, yes, you could say yes and no to any offer but chances are you will NEED them. So, while you are a lot more flexible that also means you are more dependent on actually having something to do that will ultimately pay the bills. You won't be able to simply rely on recurring, reliable amounts of money. Oh and there are customers who will just make you wait for whatever money they owe you because like I said above, everybody is liquidity's bitch and in the worst case you might not get paid at all... that is definitely a risk you will have to carry and be aware of.

So ultimately, those 4 figures a day sound insane at first but you will have to dig deeper and really weigh the ups and downs and whether it is something you like because ultimately it ends up being less; possibly still handsome but chances are your actual work won't just be "doing task x for customer y" but you will also have to do a, b and c just like a real company would have to - because ultimately, you actually ARE your own company at that point. Your best bet is to read up on it as much as possible, talk to self-employed consultants a lot and since you are European, chances are your government offers some sort of "kick-start" help, training or even funding when you start your own little "company".

Oh and, stay self-employed and if you need help, find other self-employed contractors before you take on actual employees working for you. European law doesn't frakk around when it comes to employees... save yourself that trouble for MUCH later.

The bottom line is: it has a lot of positive and a lot of negative sides when compared to a regular 9-5 job and you will have to weigh them yourself and see how it works out... it is definitely an interesting experience and can be such an exhilarating, empowering feeling because YOU are working for YOURSELF and make all the decisions but remember that most "startup" companies like that typically don't make it over the first 1 or 2 years and a LOT of "I can write Java really well" guys are desperate for gigs on the side because it is hard finding a project... so they write books or teach. That's why the idea with the niche. And see what your government, chamber of commerce etcetctec can do for you.

Good luck.

<comment><USER28> Well, I think I could swing it. What kills me right now is not my meager salary, but rather that it takes so much damn time to acquire it. And most of that is unnecessary time. Working hard and playing hard is more my melody, I dislike the middle ground.

<comment><USER31> I should also mention: consulting can also require (much!) more flexibility and traveling depending on partners and customers and staying in hotels and flying around all the time can get very old very quickly... most of the time you don't really have any chance to enjoy the place you are staying at anyway - if you are independent and don't have a wife or especially kids to worry about, if you are young and just starting out, it is definitely a great option to see a lot, meet people, get "front experience" and valuable contacts a lot easier and natural than any other way - if you want to have a family and enjoy lots of time with them and provide for them reliably, it might not be the best option.

And I hear you about working and playing hard... I would love to do my week's work in 2-4 days and have the rest of the time off but the overwhelming majority of employers is stuck in Ford's 40-hours-at-the-belt kind of ideology, especially in older, settled industries.

</comment></comment></comment></comment></comment></comment><comment><USER32> Seems naive. I came up with an engineering scale that I think is pretty solid, at least over the 0.5 to 2.2 range. [http://michaelochurch.wordpress.com/2012/01/26/the- trajector...](http://michaelochurch.wordpress.com/2012/01/26/the-trajectory- of-a-software-engineer-and-where-it-all-goes-wrong/) . I'm only about 1.7 so beyond about 2.2 I had to start using "bigger and badder" type definitions because I could only delineate those levels based on visible accomplishments, not experience or observation. Also, it's more of an "liberal arts" _engineering_ (adding value to organizations) than one focused on domain- specific _programming_ capabilities, which are still important but much harder to measure. You can become a 2.0 _engineer_ in artificial intelligence, or in front-end web development, or in embedded device development and the processes of getting there are rather similar. Being a "good programmer" in each discipline is different.

What OP's system describes is the progress from a novice (0.5) to a competent adder (1.0) to a general-purpose multiplier (2.0). Most serious programmers are between 1.0 and 2.0, so the growth process is about becoming a multiplier. At 1.1, you start automating annoying tasks by writing scripts. At 1.4, you're starting to become aware of the need for language-level support for code reuse and complexity management, so functional programming becomes attractive. The 1.5-1.7 range is when Lisp's language-oriented features (macros) start to shine. Most 2.0 programmers are cautious about when they use such things, though. Yes, DSLs can be very powerful; they can also be unmaintainable, miserable cruft-piles. Being a great (2.0+) programmer is about adding the most value though team- or company-wide multiplier effects, _not_ being clever. Often, you add the most value by designing systems so simple and elegant that they almost insult the user's intelligence (in a good way).

Functional programming is a shibboleth of good (1.5+) engineers because it's usually what they first learn as they break through the 1.2 ceiling observed in Javaland (since Java and enterprise programming actively work to prevent people from having multiplier effects). And pretty much every 1.5+ programmer knows what functional programming is and what its rules are (and when to break them). That said, it's not a strict rule that to reach or surpass 1.4-1.5 you have to spend X amount of time using functional languages.

<comment><USER33> You know, one thing I like about your system is that you aren't at the top of it. I have to question peoples' motives when I see an article like this where they're clearly at the top of the hierarchy.

<comment><USER32> I developed the scale because I found myself frustrated by "95th-percentile" syndrome. I noticed that, from when I started programming to now, I always considered myself around 95th percentile. And so did most other programmers I knew, whether they were "objectively" 80th or 99.5th. Which meant that these percentile-based self-assessments were worthless.

Similarly, if I were at the top of the scale, that would show a lack of imagination on my part, and also make the scale worthless in a "I won at a game where I just wrote the rules" sort of way. Personally, the territory that's _most_ interesting to me is 1.7 to 2.2, because that's what I intend to be covering over the next 5-10 years.

People tend to estimate themselves as average within what they consider their peer group when comparisons are difficult or impossible to make (and the fact is that it's really hard to compare two programmers). So I think the 95 comes from the fact that it's "common knowledge" that 90% of "programmers" are incompetent.

Of course, these percentiles depend on a population and competence distribution that are by no means immutable, which makes them meaningless as technologies evolve and more people come into software, so I needed a scale for professional development of an engineer based on general knowledge and approaches. The actual 95th percentile is probably around 1.5 on that scale, but that's just a wild guess.

I'll still use "5-percenter" (loosely; no clue what the actual percentage is) to describe an elite set of programmers, but that has more to do with attitude, bearing, and trajectory than present-time skill. For example, the guy who picks up SICP because he's topped out and bored at his Java job is probably only 1.2-1.3, which is not yet 95th-percentile, but he's a "5-percenter" because he has the attitude and has found the resources that will make 1.5 likely and 2.0 possible-- which wouldn't be the case if he stuck with his same-every-day enterprise Java job.

</comment></comment></comment><comment><USER34> I like how programmers target other programmers _at different levels_ (and also that the more advanced you get, the more able you are to target _less_ skilled programmers). Not sure that's fully correct, and lots of details to disagree with, but I appreciate this point of view.

An overall objection: though it does both, it focuses on coding solutions rather than understanding problems. While there's a case for defining a programmer as "someone who programs", to me, understanding the problem (aka asking the right question) is the difficult bit (see Douglas Adams).

</comment><comment><USER35> If I ignore the rating system in this post, I see this list as an exercise to acknowledge my strengths and recognize where my weak points are. Doing so, I came up with a list of areas to explore in the near future.

</comment><comment><USER36> This seemed oddly familiar, and sure enough:

[http://michaelochurch.wordpress.com/2012/01/26/the- trajector...](http://michaelochurch.wordpress.com/2012/01/26/the-trajectory- of-a-software-engineer-and-where-it-all-goes-wrong/)

What I said when that hit HN still applies. Treating the creation of yet another programming language as somehow being the pinnacle seems like a bad idea.

</comment><comment><USER37> Somehow this seems like saying that Paul Erdös, John von Neumann, and Andrew Wiles are different levels of mathematician. Not a productive endeavour.

</comment><comment><USER38> I think this might be more accurate: <http://www.infiltec.com/j-h-wrld.htm>

</comment><comment><USER10> Where does Chuck Moore would fit there ?

</comment><comment><USER39> The first few levels make sense, but beyond that unless the author is the highest level described here i have to ask if this is a prediction or a description of their experiences?

</comment><comment><USER40> So I'm a 4.5, that's nice to know.

</comment><comment><USER31> I think the only "level" you should really care about and isn't on that list: being fluent enough on all systems, APIs, libraries and infrastructure you are connecting to or using so you ultimately do not really care all that much which language you are programming in, on which system and OS and even if you didn't know the system or language all that well to begin with, you would still manage to become fluent and productive in it in reasonably short time.

It takes experience and some understandings of the fundamentals of computers, software and networks and some curiosity and will to find things out and better yourself but if you want to be a programmer and even work as one, then this is what you should aim for... be "fluent" enough on (almost) all OSI layers so you can figure things out and understand what is really going on and whatever very-specialized knowledge will be required at a certain point in time, you will be able to figure it out and pick it up along the way because you will have the broader overview and understanding to connect things.

Level 5 sounds close but still too limited still.</comment>


Kenyans chase down and catch goat-killing cheetahs - <AUTHOR> https://www.bbc.com/news/world-africa-24953910 <comment><USER2> In his book Born to Run, Christoper McDougall talks about the history of human persistence hunting....this is a fantastic modern example of same!</comment>


Is EF the New IQ? - <AUTHOR> http://www.newsweek.com/id/139885/output/print

<comment><USER2> Like IQ, EF is creating a self-fulfilling prophecy with regard to success and failure. IQ was used to justify the failure of some students relative to others--so the students with a reason why they were allowed to fail did so, and the teachers paid more attention to the students expected to succeed, so that they did so.

EF, being a measure of "mental discipline" (read: obedience + creativity) separates the students that teachers typically like to teach from those that teachers typically do not. But the students that teachers like to teach are the ones that they focus on, so those students do well academically. Presto-- instant self-fulfilling prediction.

Of course, this implies no correlation at all with what it takes to do well outside of these sorts of environments.

<comment><USER3> I don't think that's what the article is saying at all. EF is not some supposedly innate ability, such as IQ. It's rather a skill that can be taught, and the actuall measured EF is just a way of examining how well the teaching of the skill is going. This gives teachers another tool in the toolbox, to work with those students that are difficult to teach - specifically, give them extra training to improve their EF.

Of course, my fad detectors were going off big time reading the article. I think I'd like to see some data on whether this produces longterm benefits before jumping on the bandwagon. Otherwise it might just be nothing more than showing that kids that get extra attention - such as those in a research study - do better in the short term, something that we know already.

I think perhaps the most interesting aspect of the article is that this potentially gives preschool teachers something actually useful that they can teach to their students, something that could aid the students when "big school" begins :-)

</comment></comment><comment><USER4> "Most people can recall a kid from grade school who couldn't stay seated, who talked out of turn and fidgeted constantly, whose backpack overflowed with crumpled handouts and who always had to ask other kids what the homework assignment was. Those kids weren't bad kids, but they seemed to have absolutely no self-control, no internal disciplinarian to put a brake on their impulses, to keep their attention focused. Not surprisingly, they were almost always lousy students as well."

Heh, I think I was that kid - but a pretty good student as well. My grades were relatively poor (because I forgot or didn't want to do the homework), but it wasn't uncommon for me to have the class high on exams.

Fortunately, I had one of the highest set of test scores (SAT/ACT/AP) of my class and that more or less made up for the poor GPA and class rank.

<comment><USER5> I used to be exactly that kid, until school got challenging (university)... Then I had less time to fuck around in class...

<comment><USER6> I was that kid. When school got more challenging, I spaced out and did worse.

It's a source of frustration for me. I know, and always knew that I had the intellectual acumen to to better, but I never had the self discipline to do so. I tried every day to just buckle down and finish my work, but it is easier said than done.

On the other hand, it was always those mandatory literature classes that hurt me the worst.I always did better in the courses that required more thinking than reading.

<comment><USER7> I'm much the same way. For me it was the classes I didn't like that I did poorly in, like marketing. The less interesting it is going in the worse I do.

<comment><USER8> So, it's settled, then? Entrepreneurs are smart people with ADD who learn to channel their energies toward interesting enough ends? Seems like I've been hearing pretty much that message for most of my life.

After all, if you excel at working within established institutions throughout your life, you'll probably continue to follow whatever career path is expected of you as an adult.

</comment></comment></comment></comment></comment><comment><USER9> Funny. The research seems to show that an ability to quiet one's mind, listen to others even if you're not interested, and follow arbitrary orders is helpful in school. To me, this is obvious, and once again reinforces the fact that school is a harmful institution based on conformity and obedience. To the school researchers, this shows the fundamental importance of this "new cognitive skill".

</comment><comment><USER10> I think AS is the new IQ: Academic Success has been shown to be the best predictor of Academic Success so far...

<comment><USER11> Tell that to college admissions...

</comment></comment><comment><USER12> While executive functions are crucial in controlling primal urges, some of which arguably trigger uncooperative behavior in group settings and blind them from competing perspectives, I really dislike this buzzwordification of something that isn't surprising nor new.

The whole principle of Diamond's education method seems to be teaching kids how to inhibit their urges, or "think before your act" (I'm sure there's an elegant quote/prophrase for this in English. Anyone know?). Big deal. I bet you can achieve similar results or even better via dialectics a la Plato. Once they have the patience, teach them and all their best friends Go/chess.

<comment><USER13> > or "think before your act" (I'm sure there's an elegant quote/prophrase for > this in English. Anyone know?)

"Look before you leap" comes to mind.

</comment></comment><comment><USER14> "New research shows that EF, more than IQ, leads to success in basic academics like arithmetic and grammar."

This research is from the 60s. It isn't new, and plus we've already had better stories about this same concept on news.yc before.

<comment><USER15> We've already had better comments about that same complaint in this _thread_ before.

</comment></comment><comment><USER16> Looked like an interesting article, but the phone rang so I never read it.

</comment><comment><USER17> The corollary is also true. From what I saw in school, academic success didn't really require or didn't often encourage high IQ.

</comment><comment><USER18> Focusing is good, but if you don't have any good ideas, it doesn't matter whether or not you can focus on them.

I think ADD tendencies can be good. Having a lot of open ideas to switch between is a good way to avoid getting burned out.

<comment><USER19> Q. How many ADD kids does it take to write good code?

A. Let's go ride bikes!

<comment><USER20> Wrong.

The correct answer was "Code is hard! Lets go shopping!"

Bleh, I couldn't resist that, but now I feel like I have to go wash the troll off me. Sorry.

</comment></comment></comment><comment><USER21> Here's a study on the heritability of EF:

[http://scienceblogs.com/developingintelligence/2008/05/99_ge...](http://scienceblogs.com/developingintelligence/2008/05/99_genetic_individual_differen.php)

</comment><comment><USER22> The printer-friendly version is much easier to read without all those annoying ads: <http://www.newsweek.com/id/139885/output/print>

<comment><AUTHOR> Yes. I tried to change the link to OP, but it wouldn't let me. Sorry about that.

</comment></comment><comment><USER23> The state of mind required to execute on thoughts of course is different than the state of mind required to abstract thoughts.

Maybe all the article is saying is that we still have not figured out how to correlate creativity with academic success.

Me, bored HS student, good college student. Success as an entrepreneur has been the ability to control how I switch between the two mindsets.

</comment><comment><USER24> It would be better to develop the personality of the child rather than judge them by their IQ (or EF). Moulding children to become "practical" thinkers would benefit them for life.

</comment><comment><USER25> I think they're just talking about ADD (which I suspect is highly prevalent in HN users. It doesn't mean you're stupid, it just means you could get a lot better at everything if you got it treated (ie got ritalin)...just a thought).</comment>


The tech industry needs a moral compass - <AUTHOR> https://medium.com/doteveryone/the-tech-industry-needs-a-moral-compass-3ce1665a287f <comment><USER2> If you want to find a moral compass, look to the history, rituals, philosophy, and timeless truths of the past. It will be there waiting.</comment>


The Last Ride of Cowboy Bob - <AUTHOR> http://www.texasmonthly.com/content/last-ride-cowboy-bob

<comment><USER2> Notorious Denver playhouse Buntport Theatre wrote and produced a play, "Peggy Jo and the Desolate Nothing" ([http://www.buntport.com/reviews/peggyjo.htm](http://www.buntport.com/reviews/peggyjo.htm)) based on this article. Buntport's production was last summer.</comment>


Ask HN: Opportunities for pro-bono work democratizing strong encryption - <AUTHOR> As I'm sure many of you are also feeling right now, I'm worried about the direction the US is taking toward total surveillance and loss of freedoms both on and offline. I'm already making donations to organizations like the ACLU, but I would really like to put my engineering skills to use on a project that could help fight back.<p>Is there any existing organization whose goal is building and marketing a highly distributed, open source messaging platform with end-to-end encryption, with good enough UI/UX that it's understandable and attractive to the average American? <comment><USER2> Yes, Open Whisper Systems. Signal is open source and meant to be accessible. There's also ToxCore - [https://github.com/irungentoo/toxcore](https://github.com/irungentoo/toxcore) \- which I have no experience with and I've heard sometimes the community is a little trollish, but it has similar goals.

Remember, _any_ privacy project could use this help. _All_ of them want to be accessible to the end user, including old-school tools like GnuPG. Often times these security engineers aren't very UX oriented and we like our command lines. Build good interfaces on top of them. Although a shitty name, Felony looked good for this: [https://github.com/henryboldi/felony](https://github.com/henryboldi/felony)

Someone else asked a similar question in another Ask HN earlier, and I replied with this:

> Welcome to the fray.

> First: decide what your skills are, and where you can contribute.

> Try the Tor Project first:

> > [https://www.torproject.org/getinvolved/volunteer.html.en](https://www.torproject.org/getinvolved/volunteer.html.en)

> The community is great there and very accepting of newbies that have the > skills.

> Signal / open whisper systems also are open source and accept pull requests:

> [https://github.com/whispersystems/](https://github.com/whispersystems/)

> Read the anonbib:

> [http://freehaven.net/anonbib/](http://freehaven.net/anonbib/)

> Also, there are other mixnet/darknet projects that could use help, such as > i2p:

> [https://geti2p.net/en/get-involved](https://geti2p.net/en/get-involved)

> And if you want decentralized, mesh communications, cjdns and the Hyperboria > experiment:

> [https://hyperboria.net/](https://hyperboria.net/)

Get involved with any of it. Push for the user, fight for the user. Make it accessible and message the shit out of it. All of them have their merits and any work toward any of them is progress.</comment>


Piracy and new content creation: a Bollywood story (2014) - <AUTHOR> http://repository.cmu.edu/cgi/viewcontent.cgi?article=1395&context=heinzworks <comment><USER2> Instead of simply measuring volume of _all_ content created, I would like to see a measure of _high quality_ content created. I think that sharing of high quality content leads to a higher demand for it, while sharing of low quality content leads to lower demand. So things like the practice of putting a single good song on an album would overall decrease demand, showing an overall drop in revenue for those companies.

</comment><comment><USER3> Now I'd really like to know what Hollywood/US distributors did differently to win in large numbers despite the piracy, which affected Bollywood.

They obviously made the megaplex "going to the movies" experience more attractive. This helped Bollywood. Was this all?

</comment><comment><USER4> Fascinating.

>> Thus, our study provides affirmative evidence on a central

>> tenet of copyright policy, that stronger effective

>> copyright protection effects more creation.</comment>


Show HN: I launched my first solo side project - <AUTHOR>

Sysadmin is my bread and butter, but I taught myself PHP and have been coding on and off for the past 6 years. I've worked on numerous freelance gigs, but mainly internal apps that I use for my real job. I worked on this project (on and off) for the past year and I'm happy to finally release it to the public.
Even if it doesn't succeed, I'm happy that I was able to "scratch an itch" I've had for the past couple of years. I wanted to make something and prove to myself that I can program. I'm not the greatest, but I'm proud of what I've achieved.
http://www.ikeepm.com <comment><USER2> The problem I see is this: How can anyone prove anything? Will they be entering exact factory numbers etc? I ask because one could easily include a bunch of very expensive laptops with stock photos / photos taken from a friend and then get "robbed" and provide this evidence. This might be the only trouble with attacking the insurance providers (although I could be wrong if there is a way to insure they actually own it).

In any case I wish you the best of luck!

<comment><AUTHOR> I got feedback from a couple of folks who had the unfortunate task of going through an insurance claim process. As long as they have receipts, they're fine. Obviously this could pose a problem for some of your items that you purchased years ago but he told me lots of photos that show your item inside the house goes a long way. e.g. This won't work ... Photo of your 47" LCD in a room that doesn't look like anything inside your home.

Generic photos won't work, I know many folks keep a video recording of their home.

</comment></comment><comment><USER3> Congratulations. I think this is a neat little service. I could certainly see a market outside the domestic one, too. I've worked for small businesses and public sector organisations where maintaining an inventory of office equipment was necessary not just for insurance, but also for funding and auditing purposes.

Best of luck.

<comment><AUTHOR> I'm debating whether I should focus more on businesses. At my current job, we're in the middle of inventorying everything from IT to furniture. We're using spreadsheets of all things but I can't push my product since it would be a conflict of interest.

<comment><USER3> Updating a spreadsheet with quantities of desk lamps and keyboards sounds painfully familiar.

I would target businesses. Here's something to consider. With a home user, you first have to convince the potential customer of the benefits of keeping an inventory. Then you have to persuade them to use your service over a competing one, or a spreadsheet, or pen and paper. Then you have to convince them to pay you for your service, presumably, at some point in the future.

Keeping an inventory is already an established practice for businesses, so that first home user hurdle is cleared. You just have the remaining two: explaining why iKeepm is better than struggling with a spreadsheet (photos and reports are two features that spring to mind for me straight away) and getting people to pay. My hunch is that businesses, already persuaded of the need to keep an inventory, would be more ready to pay than home users.

<comment><AUTHOR> Thanks! I need to talk to some folks in finance that handle this stuff. What exactly do they look for, I assume the finance department are the ones who request such a list.

</comment></comment></comment></comment><comment><USER4> This is fine. As long as its possible to attach scans of warranties and receipts for insurance claims. Great layout, tho one nitpic: 'Getting started is hard' repelled me. Draw the user in with 'Getting started is easy' or some other attraction.

"A+++ hacker. would buy from again" :)

<comment><AUTHOR> Thanks! I've modified it.

</comment></comment><comment><AUTHOR> Link: <http://www.ikeepm.com>

</comment><comment><USER5> Great idea, kind of like home user asset management i guess. I'll definitely be signing up. It's looking pretty polished too.

</comment><comment><USER6> Genuine question: what are the benefits for an individual to track everything they own?

<comment> dan-g Insurance purposes, most likely.

</comment></comment><comment><USER7> Congratulations. It looks very polished.

</comment><comment><USER8> While I realize it's not a "flashy" project in terms of "look how much fun you can have with this app" or "look how it can streamline your workflow," I like its utility. Every time I look around my condo, I can't help but think "I really need to do a better job of recording my assets in case something happens and I need to file a report with my insurance company. So in that sense, anything that makes it easier for me to do that would come in handy.

But I'm not sure you give that need enough emphasis in your CTA's. I don't really see anything on the homepage (which is where it's going to be most important) about needing to document this stuff for my insurance company. As far as I can tell, that's pretty much the number one reason, right? Perhaps you could even play off the same basic cautionary desires that make us buy insurance in the first place. For example, I know the chances of my condo burning down are incredibly slim, but I still spends hundreds of dollars insuring all my stuff as a "just in case." Why not play on that? Why not have messaging that reminds people that they're doing it for the "just in case"?

You mention you've got a full time job and this is a sort of side project, but you might also consider reaching out to some insurance companies to see if you could get some sort of "Seal of Approval." Right now, I don't know if your product would be an approved means of categorizing assets for my insurance company, but if I saw State Farm's logo (and All-State, or whoever) right there on the homepage, I, as your consumer, would get that immediate warm- fuzzy feeling of being extra secure (even if having those logos come with all sorts of caveats about not guaranteeing claims, necessary comprehensive documentation, etc, etc.)

<comment><AUTHOR> Totally understand what you're saying and it's on my to-do list. I'm going to spend the next two months on getting the message across. Thanks for your input!</comment></comment>


Wedwell.co - <AUTHOR> Does anyone know whatever happened to WedWell.co and it's founder Vishal Gupta on here? I'd like to find out if they ever got any engagement. <comment><USER2> Is/was it the same as this?

[https://www.bewedwell.com/](https://www.bewedwell.com/)</comment>


Naughty Dog still uses Lisp in its videogames - <AUTHOR> http://game.watch.impress.co.jp/docs/news/20100315_354802.html <comment><USER2> <http://en.wikipedia.org/wiki/Game_Oriented_Assembly_Lisp> seems to contradict that - they seem to be moving to C++ for future projects, which is kind of a shame.

<comment><USER3> The post appears to be highlighting that despite this announcement lisp is used to tweak the gameplay in recent games. It could be that the engine related functions like disk streaming are now C++.

Naughtydog has now been part of Sony for some time.

</comment></comment><comment><AUTHOR> My colleague pointed me to this:

<http://www.gameenginebook.com/coursemat.html>

Check 'GDC 2009 Lecture'. Now ND uses PLT Scheme.

</comment><comment><USER4> See Lisp here: [http://game.watch.impress.co.jp/img/gmw/docs/354/802/html/u2...](http://game.watch.impress.co.jp/img/gmw/docs/354/802/html/u219.jpg.html)

<comment><AUTHOR> Oh, thanks, I thought I linked to that picture, but not actually.</comment></comment>


Ask HN: Which VC is allowing founders to pitch their idea with just a prototype? - <AUTHOR>

A few months ago, a VC firm announced that they were going to allow a certain amount of pitches per month which only required an MVP. Does anyone know which one? Thanks! <comment><USER2> Most VC firms that make seed investments will hear pitches from founders who just have an idea. However this would require for you to be in the VC's sweet spot in terms of vertical or that you get a referral from someone who knows the VC personally.

Not sure about the specific one you're asking about. I'd recommend looking at what seed-stage funds like to invest in your vertical and then seeing if anyone in your network has a connection there.

</comment><comment><USER3> As long as you get introduced/meet in the right place, pretty much every non- famous VC will. A lot of people don't even have prototypes. And honestly, they'll only care about seeing the prototype if it's the sort of thing that you can use on spot or they really get your vertical.</comment>


OS X Lion Terminal Colours - <AUTHOR> http://noiseandheat.com/blog/2011/12/os-x-lion-terminal-colours/

<comment><USER2> I wish I could down vote apple news stories. Does that make me a hater?</comment>


Does AR Need Better PR? - <AUTHOR> https://arinsider.co/2019/01/01/does-ar-need-more-pr/ <comment><USER2> It's ironic that "AR Insider" is saying this because they are a source of the duckspeak that causes people to grow earflaps.

Their favorite word now seems to be "XR", which I've also seen people using in connection with 5G and Telecoms.

It seems that most people roll their eyes when they hear about "AR" and "VR", are baffled if you ask them what "MR" is, but there just might be some corporate executive who sees blather about "XR" and wonders if they need to get an "XR Strategy".

</comment><comment><USER3> No, it needs to stop producing things like Magic Leap. Leave the PR for when you have a product that at least resembles the hype, otherwise you're just building your own winter to die in. The tech is the problem, not the hype.</comment>


Ask HN: How to decide if our team should use Kanban or Scrum? - <AUTHOR> Some context. For now, there are just two of us: a junior developer (myself) and a senior one on this new team. The company has around other 30 developers, all of them using Scrum on their teams.
Our team works closely with the marketing team. As it is a startup, all the time new, usually small, independent items are entering our backlog and changing priorities. For this reason, we thought Kanban would be a better fit. But we are struggling with it a little bit with it.
I was told Scrum is a little bit easier to follow through for unexperienced devs and teams. But I do believe Kanban is a better fit if we can master it. I would like to have some advice on what factor do you count in to decide if Scrum or Kanban is a better fit for a team. Thanks! <comment><USER2> Scrum is better for feature development; Kanban is better suited for maintenance and the sort of small independent tasks you describe. Scrum involves more meetings and overhead but provides more structure, which juniors often need; Kanban requires more discipline from individual developers, and tends to work better with more senior teams.</comment>


Modern real-time aviation map of Finland - <AUTHOR> https://ilmailukartta.fi/ <comment><AUTHOR> The map updates automatically every 5 minutes showing in real-time all permanent and temporary danger-, restricted- and prohibited areas, military reservations, restricted areas for aerial photography, obstacles, airfields, aerodromes, VFR waypoints, drone rules, heliports, ...

I'm happy to answer any questions about the map!</comment>


Ask HN: Are all Facebook accts in america phone-verified? - awinter-py Swallowed bile and created an FB account so I could pay them for their cheap ad platform. The account locked after 30 minutes and is asking me for a mobile number to continue.<p>Do I have a suspicious behavior pattern or is this standard operating procedure? <comment><AUTHOR> I've never given them my phone number - they've guessed it, likely through scanning the address book on some friend's phone, and they persistently ask me to confirm their guess, but I've never agreed. Perhaps they'd be more insistent if it were a newer account.

<comment><USER2> Same here. I keep getting the popup every now and then to confirm my number.

At one time, the mobile app used to pre-fill the username field with my actual phone number, and I had to delete it manually and enter my email address to log-in... those sneaky sons of...

</comment></comment><comment><USER3> All accounts or just those trying to place ads? Mine isn't phone verified but I also don't purchase ads via FB.

<comment> awinter-py Asking about all accounts They didn't say 'you need to provide a phone number to pay us', my account was just suddenly locked.</comment></comment>


NLP's generalization problem, and how researchers are tackling it - <AUTHOR> https://thegradient.pub/frontiers-of-generalization-in-natural-language-processing/ <comment><USER2> To be fair, poor generalisation is not a problem of NLP, but of NLP using deep neural networks, which is a very recent phenomenon, following from the success of deep learning for image and speech processing.

If you asked me (you shouldn't, I'm biased), I'd tell you that we're never going to get deep neural nets to behave well enough to learn meaning. Those damn things are way too smart for us. They're so smart that they can always find the easiest, dumbest way to map inputs to outputs- by simply overfitting to their training set (or their entire data set, if cross-validation is thorough enough). You can see lots of examples of that in the article above.

The big promise of deep learning was (and is) that it would free us from the drudgery of feature engineering- but, the amount of work you need to do to convince a deep net to learn what you want it to, rather than what it wants to learn, is starting to approach that of hand-crafting features.

And we still don't have language models that make any sense at all.

<comment><USER3> What are the methods you're thinking of that have better generalization properties?

<comment><USER2> Actually, I wasn't thinking of any specific methods, but now that you mentioned it, Inductive Logic Progamming (the subject of my PhD - see my comment about being biased) is a fine example.

For a slightly more impartial opinion, here's a DeepMind paper that performs neural ILP: [https://deepmind.com/blog/learning-explanatory-rules- noisy-d...](https://deepmind.com/blog/learning-explanatory-rules-noisy-data/)

The authors begin by extolling the virtues of ILP, including its generalisation abilities, as follows:

 _Second, ILP systems tend to be impressively data-efficient, able to generalise well from a small handful of examples._ [1]

You can find more references to the generalisation power of ILP algorithms sprinkled throughout that text and in any case the entire paper is about getting the "best of both worlds" between ILP's generalisation, interpretability, ability for transfer learning and data efficiency and deep learning's robustness to noise and handling of non-symbolic data (I disagree about these last two bits with the authors, but, OK).

From my part, below is an example of learning a general form of the (context- free) a^nb^n grammar from 4 positive and 0 negative examples, using the Meta- Interpretive Learning system Metagol (a state-of-the-art ILP learner, referenced in the DeepMind paper; my PhD research is based on Metagol). You can clone metagol from its github page:

[https://github.com/metagol/metagol](https://github.com/metagol/metagol)

Metagol is written in Prolog. To run the example, you'll need a Prolog interpreter, either Yap [2] or Swi-Prolog [3]. And Metagol.

Copy the code below into a text file, call it something like "anbn.pl" and place it, e.g. in the "examples" directory in metagol's root directory.

      % Load metagol
      :-['../metagol']. % e.g. place in metagol/examples.

      % Second-order metarules providing inductive bias
      metarule([P,Q,R], ([P,A,B]:- [[Q,A,C],[R,C,B]])).
      metarule([P,Q,R], ([P,A,D]:- [[Q,A,B],[P,B,C],[R,C,D]])).

      % Grammar terminals, provided as background knowledge
      'A'([a|A], A).
      'B'([b|A], A).

      % Terminals actually declared as background knowledge primitives
      prim('A'/2).
      prim('B'/2).

      % Code to start training
      learn_an_bn:-
      	% Example sentences in the a^nb^n language
      	Pos = ['S'([a,a,b,b],[])
      	      ,'S'([a,b],[])
      	       % ^^ Place second to learn clauses in terminating order
      	      ,'S'([a,a,a,b,b,b],[])
      	      ,'S'([a,a,a,a,b,b,b,b],[])
      	      ]
      	% You can actually learn _without_ any negative examples.
      	,Neg = []
      	,learn(Pos, Neg).

Load the file into Prolog with the following query:

      [anbn].

Finally, start training by calling learn_an_bn:

      ?- learn_an_bn.
      % learning S/2
      % clauses: 1
      % clauses: 2
      'S'(A,B):-'A'(A,C),'B'(C,B).
      'S'(A,B):-'A'(A,C),'S'(C,D),'B'(D,B).
      true .

That should take a millisecond or two, on an ordinary laptop.

You can test the results by copy/pasting the two clauses of the predicate 'S'/2 into a prolog file (anbn.pl will do fine), (re)loading it and running a few queries like the following:

      ?- 'S'(A,[]). % Run as generator
      A = [a, b] ;
      A = [a, a, b, b] ;
      A = [a, a, a, b, b, b] ;
      A = [a, a, a, a, b, b, b, b] ;
      A = [a, a, a, a, a, b, b, b, b|...] ;

      ?- 'S'([a,a,b,b],[]). % Run as acceptor
      true .

      ?- 'S'([a,a,b,b,c],[]). % Run as acceptor with invalid string
      false.

      ?- 'S'([a,a,b,b,c],Rest). % Split the string to valid + suffix (Rest)
      Rest = [c] .

Note that the leraned grammar is a general form of a^nb^n, for example it accepts strings it's never even seen in testing (let alone training):

      ?- 'S'([a,a,a,a,a,a,a,a,a,a,b,b,b,b,b,b,b,b,b,b],[]).
      true .

In any case, it's just a couple of first-order rules so it can be readily inspected to judge whether it's as general an a^nb^n grammar as can be, or not.

I guess you might not be much impressed by mere learning of a puny little grammar of a's and b's. You might be slightly more impressed if you know that learning a Context-Free language from only positive examples is actually impossible [4]. Metagol learns it thanks to the strong inductive bias provided by the two second-order metarules, at the start of the example. But, that's another huge can of worms. You asked me about generalisation :)

btw, no, you can't learn a^nb^n with deep learning- or anything else I'm aware of. The NLP people here should be able to confirm this.

_________________________

[1] [https://arxiv.org/pdf/1711.04574.pdf](https://arxiv.org/pdf/1711.04574.pdf)

[2] [http://www.dcc.fc.up.pt/~vsc/Yap/](http://www.dcc.fc.up.pt/~vsc/Yap/) (Yap is fastest)

[3] [http://www.swi-prolog.org/](http://www.swi-prolog.org/) (Swi has more features)

[4] [https://scholar.google.gr/scholar?hl=en&as_sdt=0%2C5&q=langu...](https://scholar.google.gr/scholar?hl=en&as_sdt=0%2C5&q=language+identification+in+the+limit+mark+e+gold+&btnG=)

Well, actually, it _is_ possible - but you need infinite examples or an Oracle already knowing the language.

<comment><USER3> LSTMs can learn a count mechanism that lets them recognize a^n b^n and a^n b^n c^n: [https://arxiv.org/pdf/1805.04908.pdf](https://arxiv.org/pdf/1805.04908.pdf)

<comment><USER2> LSTMS can't learn a^nb^n or a^nb^nc^n, neither can they learn to count, and that paper shows why (because they generalise poorly).

From the paper (section 5, Experimental Results):

>> 2\. These LSTMs generalize to much higher n than seen in the training set (though not infinitely so).

The next page, under heading Results, further explains that "on a^nb^n, the LSTM generalises "well" up to n = 256, after which it accumulates a deviation making it reject a^nb^n but recognise a^nb^n+1 for a while until the deviation grows".

In other words- the LSTM in the paper fails to learn a _general_ representation of the a^nb^n, i.e. one for unbounded n.

This is typical of attempts to learn to count with deep neural nets- they learn to count up to a few numbers above their largest training example. Then they lose the thread.

You can test the grammar learned by Metagol on arbitrarily large numbers using the following query:

      ?- _N = 100_000, findall(a, between(1,_N,_), _As), findall(b, between(1,_N,_),_Bs), append(_As,_Bs,_AsBs), 'S'(_AsBs,[]).
      true .

You can set _N to the desired size. Obviously, expect a bit of a slowdown for larger numbers (or a mighty crash for lack of stack space).

Again, note that Metagol has learned the entire language from 4 examples. The LSTM in the paper learned a limited form from 100 samples.

Results for the LSTM are similar for a^nb^nc^n. The GRU in the paper does much worse.

Btw, note that we basically have to take the authors' word for what their networks are actually learning. They say they're learning to count - OK. No reason not to believe them. Then again, you have to take them at their word. The first-order theory learned by Metagol is easy to inspect and verify. The DeepMind paper I quoted above makes that point about interpretability also (that you don't have to speculate about what your model is actually reprsenting, because you can just, well, read it).

I have an a^nb^nc^n Metagol example somewhere. I'll dig it up if required.

<comment><USER4> Thanks for great insight.

Am I right in thinking Metagol requires all training examples to be flawless? LSTM presumably can handle some degree of erroneous training examples.

The ideal learning system would combine these properties: sample efficiency more like Metagol but also some degree of tolerance to errors in training data like deep learning.

<comment><USER2> Yes, classification noise is an issue, but there are ways around it and they're not particularly complicated. For instance, the simplest thing you can do is repeated random subsampling, which is not a big deal given the high sample efficiency and the low training times (seconds, rather than hours, let alone days or weeks).

See for instance this work, where Metagol is trained on noisy image data by random subsampling:

[https://www.doc.ic.ac.uk/~shm/Papers/logvismlj.pdf](https://www.doc.ic.ac.uk/~shm/Papers/logvismlj.pdf)

The DeepMind paper flags up ILP's issues with noisy data as a show stopper, but like I say in my comment above, I disagree. The ILP community has found various ways to deal with noise over the years since the '90s.

If you are wondering what the downsides are of Meta-Interpretive Learning, the real PITA with Metagol for me is the need to hand-craft inductive bias. This is not different to choosing and fine-tuning a neural net architecture, or choosing Bayesian priors etc, and in fact might be simpler to do in Metagol (because inductive bias is clearly and cleanly encoded in metarules) but it's still a pain. A couple of us are working on this currently. It's probably impossible to do any sort of learning without _some_ kind of structural bias- but it may be possible to figure the right kind of structure out automatically, in some cases, under some assumptions etc etc.

I think there's certainly an "ideal system" that is some kind of "best of both worlds" between ILP and deep learning, but I wouldn't put my money on some single algorithm doing both things at once, like the δILP system in the DeepMind paper. I'd put my money (and research time) on combining the two approaches as separate module, perhaps a deep learning module for "low-level" perceptual tasks and a MIL module for "high-level" reasoning. That's what each system does best, and there's no reason to try to add screw-driving functionality to a hammer, or vice-versa.

</comment></comment></comment></comment></comment></comment></comment><comment><USER5> I'm not sure we read something as much as we develop a shared mental model and associated new language with another human. Most of the time these models are just slightly different from before, so the change is so subtle as to be invisible. We have the appearance of a universal language called, say, "English". We don't actually have one. It's close enough.

If you take a look around, printed words don't exist in most languages. Most languages are spoken. Printing is an extremely new thing we've only had for a very short amount of time. The only thing print can do is present a stilted, over-formal version of what listening to a monotone person give a speech in a dark room might be like. That's good enough for most cases, since the brain makes stuff up it needs. But I don't think it's language, at least not in the same way real spoken languages are language. Real languages are a messy and confusing affair, even more than printed words can be. The way it works is through the interaction over time, not over definitions. (Something something language games)

I wish the guys luck. I'm not so sure we understand the problem yet. We may end up creating a machine that makes up answers to our questions after reading such that it sounds like a real person is doing the work. That's cool -- but it'd be a horrible disaster for humanity if something like that started being the primary interaction point with people. Over time it would make us a horribly stupid and unimaginative species. Like everything else in tech, we have good intentions and endless optimism. We're going to solve a problem whether we understand what it is or not, dang it. plus ça change... (And no, I don't think giving up the idea is any good. I'm just encouraging more understanding of the real goal versus the apparent goal)

<comment><USER6> On the one hand I agree, but on the other hand, I'd like to sprinkle a bit more of that endless optimism and assume that embodied cognition is the next logical step. Perhaps once these NLP agents are put into robotic bodies and begin interacting with our world more fully, they will "evolve" to attach "real meaning" to their language processing.

<comment><USER5> I love the optimism, I'd just like to see tech folks clearly delineate "This is what we're doing. This is what it might _look like_ we're doing but we're not." The hype cycle in tech, combined with a tech-illiterate public isn't such a good thing, especially in a democracy. Too often the only way we figure things out is by getting screwed. That's a recurring behavior pattern that has to end. Somehow.

<comment><USER7> > Too often the only way we figure things out is by getting screwed. That's a > recurring behavior pattern that has to end. Somehow.

That's just it - exploration vs exploitation trade-off. Don't believe this can be dusted off.

</comment></comment></comment></comment><comment><USER8> "a mouth without a brain" analogy is good one. Current NLP is impressive but there are limits.

People have spatiotemporal model of the world, different physical models, social and behavioral models of the world, organizational model of the society, economic model, etc. Humans parse the language and transform it into multiple models of the world where many indented meanings and semantics are self-evident and it becomes "a common sense". They have crude understanding of how fabrics, paper, gas, liquid, rubber, iron, rock, etc. behave and they understand written text based on this more complete model zoo.

There is similar limit in computer vision. Humans reason about 2d images using internal 3d model. Even if they see a completely new object shape, they can usually infer what the other side of the object looks like using basic symmetries and physical models.

Image understanding must eventually transform into spatiotemporal + physical model and there are several approaches underway. NLP has much harder problem, because the problem is more abstract and complex.

</comment><comment><USER9> Related article on the same website: [https://thegradient.pub/nlp- imagenet/](https://thegradient.pub/nlp-imagenet/)

NLP (or specifically NLP using deep learning) seems to be having a breakout moment in the last year or so where there have been large advancements back to back.

Generalization is hard - you're often tuning millions of parameters at once, and often the most "sane" thing for the loss function to do is rote memorization. It'll be interesting to see what comes about from this discussion.

<comment><USER10> _> NLP (or specifically NLP using deep learning) seems to be having a breakout moment in the last year or so where there have been large advancements back to back._

I don't know about that. Everything with deep learning seems to attract so much hype that it's hard to measure the actual progress without being a researcher.

On the other hand, "classic" AI projects seem to get no recognition, even when they deliver astounding results.

For example, how many people here heard about MIT's Genesis project?

([http://groups.csail.mit.edu/genesis/](http://groups.csail.mit.edu/genesis/))

If you don't know your past, everything seems like progress.

</comment></comment><comment><USER11> Great article! It is very true that NLP is amongst the most lagging divisions within machine learning. Mainly because text content is very unstructured everywhere, let alone working consistently for different languages.

It is fascinating to see how things got better over the last couple of years though!

<comment><USER12> "Mainly because text content is very unstructured everywhere"

Encoded into this somewhat unstructured data is the very thing NLP is after: the meaning.

If you cannot teach a machine a language by talking to it with "unstructured" language, how are you supposed to make it both understand it, grok it, and speak it?

We have no trouble teaching small kids about history and math using untructured language.

Language is maybe unstructured, but the internets has lots of it, so to me, the state of NLP today is a dissapointment. I'm convinced though that general AI will be achieved through NLP. I mean, most of what I know I have either been told or I have read it somewhere. And my parent's didn't use annotated data much.

<comment><USER13> As a (former) linguist, I'm baffled about talking about language as "unstructured". Sure it isn't _rigidly_ structured, but "un"? There's a lot of structure into it, it's just so complex (and so connected to extralinguistic structures) that our ML models don't grok it!

<comment><USER14> Is a (never) linguist, I think the edge case successes demonstrate how far we really have to go in ML generally. Driving is an entirely engineered phenomenon. Down to the species level of things that might run out in front of it, every atmospheric perturbation, we can define every event that's going to affect a car, and they have been designed over more than 100 years to deal with them.

Medical imaging: another area where highly trained humans have thought rigorously (for about the same amount of time) about what this all means.

What ML is lacking is the data and labels a baby bootstraps from into the "common world". How many ML models have been training on the taste of breast milk, smell of mother, the smell of dirt, the upward view of everything (think about how much time babies, since the stone age, have spent laid on their backs, looking up.

These things have to be segregated in a fairly unsupervised way using little more than reflexes (cry, suck, fencer, grasp, etc) for a while: smell of mom sometimes comes with warm milk, but not always. Associating this warm body with the smell, not just food. Sound of mom doesn't always come with smell of mom.

</comment></comment></comment></comment><comment><USER15> If nothing else, you should open up this article to look through the images just below intro - often surprising how non-intelligent these learned NLP models often actually are, especially for non AI-researchers.

</comment><comment><USER16> This isn't an NLP problem, it's an AI problem. Despite all the hype, we haven't actually made any progress toward solving strong AI, human-like general intelligence. NLP, computer vision, and many other AI problems are probably AI-complete, meaning that solving them entails solving strong AI. And the inability to generalize is exactly what separates the partial solutions we currently have from strong AI.

</comment><comment><USER17> Recently there has been lot of interesting research in field of NLP. Lot of new models, algorithms and techniques have been developed along with lot of research papers published. Personally I feel that NLP is now heading in direction where we had an aha moment with ImageNet.

</comment><comment><USER18> A thought I had the other day was how much of a driver was our need for rhythm and rhyme in the development of synonyms. Which led me to thinking how much is language driven by fun, as much as it is by any real logic. It seems word choice for any given topic is dominated by early mover and then just plain pleasant sound.

To that end, any system that is just looking for the logic of an underlying system is going to be stymied by the fundamental lack of it in language. There is the appearance of logic. And it can actually get you quite far. However, I question if it can encompass all of it. At least until the next best seller comes out.

To that end, I do not find it surprising that the best model will be an ensemble model. More, as long as we are dealing with English (a non- pictographic and a non-phonetic language), then we are ultimately trying to build a model on top of a malleable base where appeals to the logic of yesterday in a language have a non zero chance of failing today. (Edit to make this point clearer, it is amusing to me how many lines in classic Shakespear no longer pop in the same way for current language. "Now is it Rome indeed, and room enough". That a pair of words could cease to be homophones is not something that seems at all obvious. Of course, I have not studied this in decades, so mayhap my grade school lied to me.)

</comment><comment><USER19> We basically expect deep neural networks to acquire language without a body that it can use to interact with an environment. I believe you cannot separate language from the body or the environment. A corpus of text is simply not holistic enough to capture the meaning behind the symbols.

<comment><USER20> How would you design a training scenario with a virtual agent(s)?

</comment></comment><comment><USER21> Many miles down the path where hours (slowly) lays to rest, I can imagine a vivid portrait being hung in a tree outside a dungeon. Of course created with non-existing poisonous paint. A painting of good natured, voluptuous beings huddled down and whispering to each other. Whispers that should then be set free to travel the invisible rails of the web trainset everyone play with almost everywhere. The trainset that the strict parents watches without blinking their tiny hidden trinoculars. Small talk carefully elaborated and twisted to do with the ultimate meaning as you did with the dead cat, in the backyard, but not only buried within but hidden with fragrant leafs that hasn't been seen, smelled or heard before. Later, another image can be conjured of the train with heavy soldiers that hasn't gotten a single flag on the journey, embarking, and upon inspection of their long seen captain, immediately recognized for their true unchanging self.</comment>


Color Vision: How Our Eyes Reflect Primate Evolution - <AUTHOR> http://www.sciam.com/article.cfm?id=evolution-of-primate-color-vision&print=true <comment><USER2> "In the case of primate color vision, trichromacy based on the "new" M and L pigments (along with the S pigment) presumably conferred a selective advantage over dichromats in some environments. The colors of ripe fruit, for example, frequently contrast with the surrounding foliage, but dichromats are less able to see such contrast because they have low sensitivity to color differences in the red, yellow and green regions of the visual spectrum. An improved ability to identify edible fruit would likely aid the survival of individuals harboring the mutations that confer trichromacy and lead to the spread of those mutant genes in the population."

Sean Carroll has a very interesting discussion of the evolution of color vision in primates, and the consequent loss of selection pressure on many genes that develop sensitive sense of smell, in his book The Making of the Fittest.

[http://www.amazon.com/Making-Fittest-Sean- Carroll/dp/0393330...](http://www.amazon.com/Making-Fittest-Sean- Carroll/dp/0393330516/)</comment>


Create React App 2.0: Babel 7, Sass, and More - <AUTHOR> https://reactjs.org/blog/2018/10/01/create-react-app-v2.html <comment> b-nint I'm disappointed setting up typescript is still such a pain with 2.0.

<comment><USER2> Sorry. We needed to prioritize to get this release out because it's already been snowballing for about a year.

We actually did have "TypeScript support" on a nice-to-have list for this two week sprint but unfortunately didn't get to it:

[https://i.imgur.com/l0CpdZy.png](https://i.imgur.com/l0CpdZy.png) (I promise I'm not making this up :-)

I'm confident we'll ship it by the end of this year though.

<comment><USER3> What is the status of Flow in the JS ecosystem?

<comment><USER4> Flow offers a more sophisticated type system that is designed to express types that correspond with the sort of idioms you see in JavaScript in the wild. TypeScript is simpler and less flexible, but that simplicity can translate into increased productivity since there are less foot-guns and also less bugs. Strange/buggy behavior and odd performances issues are not totally uncommon with Flow whereas TypeScript doesn't seem to suffer from this as much. Since TypeScript is much more popular, IDE support tends to be a lot better out of the box, but the flow-langauge-server gives you all the same IDE-like features (intelligent autocomplete, jump-to-def, module resolution etc).

Personally, I like Flow more, and I also think its easier to integrate into an existing codebase (either as comment annotations or gradually via per-file flow comment header), but at the end of the day, _not_ choosing TypeScript amounts to swimming against the current without much justification.

<comment><USER5> >Flow offers a more sophisticated type system that is designed to express types that correspond with the sort of idioms you see in JavaScript in the wild. TypeScript is simpler and less flexible, but that simplicity can translate into increased productivity since there are less foot-guns and also less bugs.

Source? That's the primary design principle behind TypeScript too. TypeScript's type system is powerful enough to practically do arbitrary type- level transformation of data structures, and even do some crazy recursive pure functional type level programming.

The only things I'm missing from TypeScript are variadic generics (which can be emulated quite well with TS 3.0) and higher kinded types - and I don't think Flow supports them either.

<comment><USER4> [https://flow.org/en/docs/types/utilities/](https://flow.org/en/docs/types/utilities/)

Most of that can't be achieved out of the box with TypeScript. TypeScript was designed as a _language_ that would bridge the quality of life gap between es5 and more modern languages, Flow was designed exclusively as a _type system_ for annotating standards compliant JS.

<comment><USER6> Not true, most of that can be done with TypeScript. $Values might be the only one missing.

<comment><USER7> $Values should be possible with Typescript 3.0+ conditional infer types, presumably something very similar to the built-in ReturnType<T> conditional infer type.

Untested and off the top my head but maybe something like:

        type Values<T> = T extends { [K keyof T]: infer R } ? R : any

<comment><USER8> It works, just need to add `in` in-between `K` and `keyof`.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER9> > _More styling options: you can use Sass and CSS Modules out of the box._

This is what I'm most excited about. Emotion is great but I've heard a lot of great things about CSS Modules, just haven't gotten around to trying them out yet because it's been so complicated to do. I'll definitely be giving it a try now.

<comment><USER10> I wrote a bit[1] about CSS Modules with React, but interestingly it looks like CRA isn't using babel-plugin-react-css-modules so I'm trying to figure out what they're doing instead.

[1] [https://dev.to/daveirvine/postcss-with-css-modules- andreact-...](https://dev.to/daveirvine/postcss-with-css-modules- andreact-51ln)

<comment><USER11> It's perfectly fine to just use css-loader to handle CSS modules. No need for babel. Just do:

       import styles from "./style.css";

       <Component className={styles.myClassName} />

</comment></comment></comment><comment> duncan-donuts These are some pretty great improvements. I never felt I had to have cssmodules or sass, but I'll definitely use them. Great work!

<comment><USER12> If you don't feel the need to add another dependency, why would you do so?

</comment></comment><comment><USER13> I see Facebook's React team is in the comments. Please keep in mind that the open source aspect of React is part of Facebook's hiring strategy. While I believe the React team are genuine in their friendliness and passion for their work, they also represent and are payed handsomely by one of the worst offenders in surveillance capitalism.

(I realize some might feel a comment like this is out of place here, but considering the recent news about enormous leaks and privacy violations, I think it would be more strange if it wasn't brought up.)

<comment><USER14> I decided to finally bite the bullet and learn React yesterday. Every time they mentioned "we do it like this at Facebook" in the tutorial/docs, my eagerness died a little more. Even little tongue-in-cheek things, like their shopping list app consisting of companies they've acquired (Instagram, Oculus, WhatsApp) felt like a bummer.

<comment><USER2> Hey, thanks for feedback!

Regarding "we do it like this at Facebook" \-- could you point out examples where this isn't useful, and we could remove them? I wrote some of these, mostly to highlight that if something works for a massive codebase with 50,000+ components, it will probably work for yours, even if it doesn't look familiar. I don't know what's an alternative way to say it without losing the core of the message.

Regarding the tongue-in-cheek one. Well. I don't feel strongly about changing it if many people feel this way.

</comment></comment></comment><comment><USER15> I tried Babel some time ago, but I can remember it was painfully slow. Has this improved?

I also didn't understand why so much functionality for different languages are harbored by a single tool, instead of multiple smaller tools.

<comment><USER16> I'd say 'multiple smaller tools' is actually what Babel has become. It started out as a tool to compile ES6 to ES5 (6to5 it was called), but now it's a kind of framework(?) that you can add various smaller packages to for various 'transpiling' purposes.

</comment></comment><comment><USER17> Stoked to see service workers will use Workbox!

</comment><comment><USER18> Thanks for all the hardwork. I used this library when I worked at Uber and Netflix.

</comment><comment><USER19> Is it possible to use Redux without an eject?

<comment><USER20> It always been...

<comment><USER19> and redux-saga?

<comment><USER21> Of course redux-saga can be used. Unless your goal is to rewire the project compilation step, ejecting is rarely required.

</comment></comment></comment></comment><comment><USER22> > If you previously ejected but now want to upgrade, one common solution is to > find the commits where you ejected (and any subsequent commits changing the > configuration), revert them, upgrade, and later optionally eject again.

That seems insane to be honest, yet it's seems to be the norm to upgrade a framework in JavaScript (same with React Native). Is that the best solution there is? In many other languages, you can upgrade a framework without reverting commits or trying to figure out what to keep or change in huge diffs. Surely there should be a way to keep the base config untouched, and have user changes in a separate config that overrides the base on?

<comment><USER23> Create React App is not a framework, it is more similar to a scaffolding system that sets up a group of tools and processes. You can upgrade React normally without issue.

The equivalent to what you suggest would be to autogenerate some files with a tool, modify the location and structure or those by hand and then expect the tool to be able to modify then again seamlessly.</comment></comment>