JSLinux – Run Linux or other Operating Systems in your browser - <AUTHOR> https://bellard.org/jslinux/ <comment><USER2> It takes less time to boot Linux in Chrome on my laptop than it takes Reddit to load a discussion on my phone. That's pretty amazing. Not sure which is more amazing: How slow Reddit mobile is or how fast Linux boots in JS.

<comment><USER3> The old Reddit .compact page loads very fast : [https://reddit.com/r/.compact](https://reddit.com/r/.compact)

Don't use the mobile web view! It sucks!

<comment><USER4> wow, this is really insanely fast, great share!

I currently use Joey for Reddit and OpenGur for Imgur.

</comment></comment></comment><comment><USER5> I feel like Bellard is a super hero. I wonder if he has a day job and just takes on these interesting (and large?) projects during his off time.

<comment><USER6> He's the co-founder and CTO of Amarisoft, an LTE/4G hardware and software company: [https://www.amarisoft.com/company/](https://www.amarisoft.com/company/)

<comment><USER7> Companies with this type of leadership are the ones I'm interested in giving my money to, long-term, as a consumer. Not because of any sort of principled decision based on his skills, but because I feel these types of workers would stay sharp and be quicker to solve problems correctly should they appear.

</comment></comment></comment><comment><USER8> I just played Pinball in Windows 2000 in Chrome on Windows 10. I lost, but winning was not the point. Playing was. :)

I think that that sentiment encapsulates my CS experience.

<comment><USER9> Just want to say that Windows Pinball may be MS's best software ever. The gameplay is pretty deep and well balanced.

<comment><USER10> This sounds a bit sarcastic, considering it's not by Microsoft: [https://en.wikipedia.org/wiki/Full_Tilt!_Pinball](https://en.wikipedia.org/wiki/Full_Tilt!_Pinball)

</comment></comment></comment><comment><USER11> Oddly, no one here linked to Gary Bernhardt's talk: The Birth & Death of JavaScript (YavaScript)

[1] [https://www.destroyallsoftware.com/talks/the-birth-and- death...](https://www.destroyallsoftware.com/talks/the-birth-and-death-of- javascript)

</comment><comment><USER12> It hung at the boot screen for me, which really took me back.

<comment><USER13> Disable your ad blocker.

<comment><USER14> Works fine for me with uBlock Origin enabled.

<comment><USER15> Took a few minutes on my Chromebook, but it eventually loaded.

</comment></comment></comment></comment><comment><USER16> What a great nostalgic color. I think I'll change my Windows 10 desktop to it.

[http://www.colorhexa.com/3a6ea5](http://www.colorhexa.com/3a6ea5)

<comment><USER17> better get rid of Windows10

<comment><USER18> Please don't reddit on HN.

<comment><USER19> Well put.

</comment></comment></comment></comment><comment><USER20> Fabrice Bellard's jsLinux and Jeff Parsons's [https://www.pcjs.org/](https://www.pcjs.org/) are pretty fascinating.

I think Bellard uses emscripten (transpiled machine and device emulator code from C to JS), while pcjs is hand-coded JS (which is neat: [https://github.com/jeffpar/pcjs/blob/master/modules/pcx86/li...](https://github.com/jeffpar/pcjs/blob/master/modules/pcx86/lib/x86cpu.js)).

Also is the source for jslinux available anywhere? The emscriptified file at [https://bellard.org/jslinux/x86emu.js](https://bellard.org/jslinux/x86emu.js) is quite compex.

<comment><USER21> It doesn't look like emscripten to me

<comment><USER22> Bellard wrote JSLinux in 2011, well before Emscripten was a thing.

</comment></comment></comment><comment><USER23> How does it do wget, etc. without violating CORS? I was able to download from a couple different websites.

<comment> s-macke It uses a proxy. JSLinux emulates a network device and sends ethernet frames to a proxy via websockets. The proxy sends these frames to a Linux TAP device. The rest is a decent firewall configuration.

You might even be able to connect to another emulated JSLinux machine if you know that IP address.

More information here:

[http://www.benjamincburns.com/2013/11/10/jor1k-ethmac- suppor...](http://www.benjamincburns.com/2013/11/10/jor1k-ethmac-support.html)

</comment></comment><comment><USER24> ad blockers here translate nicely to boot errors: Boot failed: could not read the boot disk ntoskrnl.exe missing or corrupt

</comment><comment><USER25> Any relation to [https://github.com/copy/v86](https://github.com/copy/v86)?

<comment> s-macke These are 2 independent developers. However the emulators have some technologies in common like the 9p virtio filesystem under Linux.

</comment></comment><comment><USER26> Wow, that's impressive! I tried FreeDOS and it booted very quickly. Nice to play around with DOS again - it's even got tab completion on the command line these days - fancy!

p.s. By the way archive.org were doing something very similar. You could load up a DOS box from their web site and have the old disk mags and demos running in your browser. All very cool stuff!

</comment><comment><USER27>
        rm -rf /

It just feels so wrong. I know it's doing exactly zero damage but I still feel guilty running it.

</comment><comment> iso-8859-1 Interesting with the VFSync site, nice to see some standardization on this. I used to work on on-demand file systems in Emscripten, but it never worked well, especially because you can't have binary XHR outside webworkers.

[https://github.com/kripken/emscripten/commit/e2046bb8ce2b2cc...](https://github.com/kripken/emscripten/commit/e2046bb8ce2b2ccaaca5328f21a79c6efe15f336)

But I don't understand, why is VFSync superior to something like NFS?

It seems that Bellard is trying to set up a hosting site for VFSync, so that people can easily share images. I dreamt about having something like that, combined with copy-on-write, which would make it possible to make small changes to a big filesystem and share them, without having to upload the full system. Kind of like with Docker. I wonder if that would be compatible with the VFSync model.

</comment><comment><USER28> I couldn't get the Windows 2000 VM to connect to the internet, otherwise I wanted to try and get at least a 1-level recursive reference going... Windows 2000 running in Firefox running in Windows 2000 running in Chrome running on a Macbook Pro.

If anyone has gotten Windows 2000 to connect to the internet, would love to take a stab at it.

<comment><USER29> I was a little disappointed to see there wasn't any internet connection on Windows 2000.

All I wanted to do was to open Firefox, goto [https://bellard.org/jslinux/](https://bellard.org/jslinux/) , and fire up the Linux emulator ...

<comment><USER30> I had working internet in IE5. You have to walk through the internet connection setup for LAN, but it works.

</comment></comment></comment><comment><USER31> Cool stuff! GCC works - I thought it was broken, but it just takes a long time.

[root@localhost tmp]# time gcc hello.c -o hello

real 0m 24.43s

user 0m 2.63s

sys 0m 2.17s

[root@localhost tmp]# ./hello

Hello, World!

<comment><USER32> TCC is faster. It's on the comments of hello.c

</comment></comment><comment><USER33> Windows 2000 plays wonky with my keyboard. Any idea why?

For instance, I open Pinball and, after getting a new game with F2, pressing space (to launch the ball) does nothing except open that menu that pops up when you right click a window.

I open Notepad and I can't type words. If I press "o" it opens the "Format" menu. Which strikes me as odd.

(But really, just let me play Pinball. :D)

<comment><USER34> My psychic debugging skills tell me that the Alt key has gotten stuck down in the emulated machine. Try focusing on the emulator and hitting Alt a couple of times. (This can happen e.g. if you alt-tab away from the window: the emulator sees alt-down, but the alt-up happens in another window and the key stays stuck down in the emulator.)

<comment><USER33> That sounds like a reasonable explanation, thank you. And it works!

Now behold all the work I will not be doing today. :v

<comment><USER34> Yay, glad I could help!

</comment></comment></comment></comment><comment><USER35> Is there a practical use case for this other than "ain't it cool"?

<comment><USER36> Checking for bugs in Emscripten perhaps? A project as complex as an OS is probably good for finding edge-cases.

<comment> mark-r Doesn't this work by emulating the processor instruction set along with a few key drivers? The complexity of the OS itself wouldn't be directly hitting the emulator.

</comment></comment></comment><comment><USER37> How do I install vim?

PS: It has git, but no network.

PS: Upload non-zipped tar balls works. How to access the image (downloading)?

PS: Well, it is way too slow and hangs easily. I guess it is only an interest at this point.

<comment><USER38> Browsers won't let javascript that's running on a webpage access random TCP ports, at the very least you'd need a server component that could act as an IP proxy for you, perhaps tunneling over websockets.

</comment></comment><comment><USER39> Wow, now includes graphics!!! Fabrice Bellard is amazing.

</comment><comment><USER40> Is it using WebAssembly?

<comment><USER41> According to [https://bellard.org/jslinux/tech.html](https://bellard.org/jslinux/tech.html) it's using asm.js since 2015. But at the bottom of the page it mentions wasm, so it's not clear.

<comment> n-gauge I thought it's just a vm?

<comment><USER41> The virtual machine host itself has been compiled to asm.js (or maybe wasm) and runs in the browser.

</comment></comment></comment></comment><comment><USER42> Super slow, but pretty cool....

Finally I can play Solitaire without Ads again.

<comment><USER43> There's a way to get the old windows 7 games working in Windows 8/10, you can find it on the winaero forums.

Personally I just have windows 3.1 installed under dosbox, as it gives me the entire WEP and Crystal Caves. If you decide to do this, I recommend dosbox-x, or other unofficial release, as official releases are old.

</comment></comment><comment><USER44> oh !

\- linux kernel 4.12

\- freedos image \- windows 2000 image (!!!)

and riscv32 emulation for the above listed

<comment> s-macke Not sure about the true meaning of your comment. But if you search for a 32-Bit RISC-V emulator, you can find it here:

[http://s-macke.github.io/jor1k/demos/riscv.html](http://s-macke.github.io/jor1k/demos/riscv.html)

</comment></comment><comment><USER45> Has anyone gotten uploading to FreeDOS to work?

</comment><comment><USER46> One step closer to GNU Hurd hegemony!

</comment><comment><USER47> Was windows 2000 open-sourced?

<comment><USER48> No, see the disclaimer: [https://bellard.org/jslinux/disclaimer.html](https://bellard.org/jslinux/disclaimer.html)</comment></comment>


Why I love Common Lisp and hate Java, part II – code examples - mck- http://kuomarc.wordpress.com/2012/02/02/why-i-love-common-lisp-and-hate-java-part-ii-code-examples/

<comment><AUTHOR> This doesn't feel like the author is trying to actually help Java programmers, but rather is just an excuse for him to patronizingly describe a language to those he considers less fortunate.

I also don't understand the reason for writing the code in an intentionally non idiomatic style. If you want to convince someone why something is better; show them how it would be done properly. The way it has been presented instead insults the audiences intelligence by presupposing that they would be unable to understand this material, that the author has admittedly only recently picked up himself.

I don't think (reduce min list) is going to blow anyone's mind, and writing a function to iterate through the indexes of a list will do more to confuse than to enlighten.

not to mention; we've been here before haven't we - <http://c2.com/cgi/wiki?SmugLispWeenie>

<comment> mck- The purpose of the author of the post was to introduce meta-programming to those ignorant of it.

I see your point of insulting the audiences intelligence, but don't you think to throw in a tail-call optimized recursive function - or (reduce min list) for that matter - would be a moot comparison and distract the reader from the point?

When you try to make a point by comparison, say the weather in Hong Kong versus Toronto, you wouldn't go gauge the weather in HK on a hot summer day at 4pm in the Central District, while in Toronto you go out in the burbs in the middle of a deserted, snowy night?

<comment><AUTHOR> The problem with "introducing" metaprogramming is that you first have to have a solid understanding of the language itself in order to manipulate it. From your examples in the article, I think that anyone who genuinely has no programming experience outside Java is going to have very serious problems understanding even the example you've given. Putting myself in those shoes, I have no idea what defun, let, do, when, setf, nth, length, and funcall are. I can assume they must be function names because they seem familar, but that still doesnt tell me how they should be called, how to give them arguments, what to expect they will return, or even how to know where they're defined.

There's no way to teach someone how to programatically manipulate code if they can't mentally parse the code in the first place. Even if you were to completely teach Lisp as a language before introducing metaprogramming you still run into the experience issue. It takes personal experience to understand where and when to use macros. Like you said yourself in the article, most things can be expressed as a normal function. It takes the experience of writing a lot of code to be able to make the judgement of what actually belongs as a macro, and not just a function.

I understand that metaprogramming is very exciting because it opens up so many doors, but it's easy to forget about everything that you had to already know to get to the point where it becomes useful. Lisp makes it very easy to repeat this problem because the language itself is so small and consistent. It doesn't take long to get to the point where you feel comfortable enough to try writing macros.

It seems that you want to share how great metaprogramming can be and how much it simplifies the work you're doing. That's great, you and I can look at it and appreciate it. But framing it as though it's appropriate for Java programmers as an introduction to Lisp just serves to make Lisp programmers look arrogant, and builds even more resent amongst Java programmers.

</comment></comment></comment><comment><USER2> This is an bad example of macros; they're completely unnecessary here. The only thing you're stepping around is the already reader-macro shortened (function <). If you want to show off macros I think you should do it with something that simply cannot be done without them. Otherwise you end up with these pissing matches in the comments and the discussion diverges entirely away from metaprogramming to talking about characters or lines of code saved, which is kind of not that interesting. And Java vs Lisp is already kind of a tired topic. (Lisper and card carrying Java-hater here)

<comment> mck- I agree, and I am aware of it. I was about to get into another example to show something that only macros can do, but the post was getting too long.

So please hang in there ;) I'm getting there.

Also, there are different levels of audience. Diving straight into complex macro structures might just be ineffective.

Above all, I'm not trying to preach to the converted. You seem to be well aware of the powers of Lisp, so not my intended audience. But thanks for reading and your input anyway ;)

<comment><USER3> I agree with spacemanaki. Your article introduces higher order functions, which is a big enough concept to keep those unfamiliar well-entertained without going a step too far and throwing in a not-particularly-useful macro. The last part of the article is an un-illuminating distraction. If I wasn't familiar with CL I would probably be scratching my head and asking why the hell that ugly #' is needed in the first place.

More seriously though, your macro is dangerous as a teaching example because it introduces variable capture (on i and ans) and unintentional repeated evaluation (of list), which are big gaping traps for the unwary.

Also, try passing #'(lambda (a b) (< a b)) as the pred argument to your macro, and see what happens.

</comment></comment></comment><comment><USER4> Sigh. A good Java coder will also "get itchy" when writing two such similar functions. In fact, they'll probably already have a library that provides predicates that make the same level of abstraction trivial. Sure, the code in the predicate, being a whole class, will be a bit verbose, but that's a separate issue.

<comment> mck- Yes, verbosity is exactly the point. In the comments section, somebody actually wrote it.

<comment><USER4> Verbosity is not as big a deal as operating at the right level of abstraction. It's much more important to adhere to DRY than to save a few lines of typing. Verbosity can harm readability, but not much in this case (I wouldn't use an anonymous inner class, so it's just a case of passing a GreaterThanPredicate).

<comment><USER5> _Verbosity is not as big a deal_

Except that empirical evidence says it _is_ a big deal: code size is the best measure of complexity we have, and error rates grow superlinearly with it.

It's interesting how nearly everyone in the software profession pooh-poohs that finding. What we "know" (i.e., what we're used to) is more important to us than evidence. If we took the evidence seriously, we would work seriously to find ways to make smaller systems.

<comment><USER4> I take your point and I may be underrating the impact of verbosity. However, I think you're not quite addressing my point, which is that violating DRY (by not using high level abstractions) is worse than verbosity.

Is it verbosity that is the best measure of complexity, or is it code size? Yes verbosity leads to more code, but it's not the only thing that does. Operating at the wrong level of abstraction also will, and in a way that makes maintenance extremely difficult.

Unfortunately I've not seen any evidence compares the impact of boilerplate vs duplication.

<comment><USER5> Oh, I see. You're making a distinction between two kinds of code bloat: the kind that comes from verbose languages and the kind that comes from duplication.

I don't know of any way to separate these. Look at it this way: to make small systems, we need both powerful languages _and_ good programmers.

If you don't get to choose what language to write in (as is the case on most software projects), it's still better to write less code. But how can we measure the impact of language choice? I don't think we can. You'd have to write the system twice, holding the other variables constant. But writing the system once already changes everything.

I don't disagree with you about the importance of good abstractions for reducing code size. Where we might disagree a bit is on the influence of programming language on what abstractions get created. It's fashionable in the industry to downplay the importance of language (algorithms matter more, libraries matter more, programmers matter more - pretty much everything is supposed to matter more). But I think the language we're writing in influences the kind of thing we write, and thus the kind of thing we think, all the way up to how we conceptualize the problem. Maybe it isn't the strongest influence at any point, yet it's compounded over every decision made in the lifetime of the system. Once a system has passed the embryonic stage, by far the most important factor in how it develops further is how the existing parts already work. Language seems to me deeply involved in conditioning this trajectory - why the system grows this way and not another. Not every idea is equally likely in every medium. I think we see this in the fact that distinct languages give rise to such distinct programming cultures.

As an aside, this is something the typical way of comparing languages -- juxtapose a known algorithm written in X to the same thing written in Y -- fails to address.

<comment><USER4> Yes, that is exactly the distinction I'm talking about. And I agree this is something that's pretty difficult to measure, probably impossible to measure precisely.

I completely agree also that the language can have an impact on the abstractions you choose. Some abstractions are obvious in one language and obscure in another. I wouldn't defend Java against a better example than in the original post, it certainly has its weaknesses. But I find that most examples just show bad Java code, not that Java itself is bad, which is not enlightening and frankly becomes tiresome.

On a related note, I think a lot of people underrate the influence of tools on code quality too. They might complain Java IDEs are a crutch that merely let you work around the language's weaknesses. I think they're a lot more than that -- when I start working in a seriously powerful IDE for the first time I was surprised how much it influenced my coding. Powerful automatic refactoring just makes it so trivial to do the Right Thing (give something a better name, factor out a method, etc). Beforehand I would have said that all you need is discipline to create code of the same quality without the IDE. Now I'd say even discipline and good intentions have their limits.

</comment></comment></comment></comment></comment></comment></comment><comment><USER6> It can be written more concisely with fold.

      $ ghci
      Prelude> let getFromList ls pred = foldl (\x y -> if x `pred` y then x else y) (head ls) (tail ls)
      Prelude> getFromList [1..10] (<)
      1
      Prelude> getFromList [1..10] (>)
      10

<comment><USER7> Which can be done just as well in lisps. The whole point was to show it in a way that didn't confuse the Java devs.

;Clojure example (defn get-from-list [list pred] (reduce #(if (pred %1 %2) %1 %2) list))

</comment></comment><comment><USER8> That's funny... I posted a few alternate java examples to his blog but it looks like the comment was rejected.

(Perhaps because I said the lisp looked like vomit? but it does!)

int getMax(int... list) { int max = list[0]; for (int i : list) max = Math.max(max, i); return max; }

or if you have a collection Collections.min(list);

<comment> mck- Sorry Elliot, but yes, that is why - you missed the point, just like you missed my Ugly Lisp code beware?

The Lisp looks like vomit, because I wrote it with a Java mind. And for you to say that it looks like vomit, well.. I guess that's why ;)

Your comment did not add any value to the discussion. However, it is emotionally loaded and might antagonize readers; I'd rather keep my blog clean.</comment></comment>


Have Twitter and Facebook killed long-form online journalism? - <AUTHOR> http://betakit.com/2012/04/02/postdesk-launches-long-form-online-journalism-platform

<comment><USER2> The low ratio of paragraphs to ads, piss-poor 'journalist' bloggers, poor pagination decisions, and the sell out of the mass media certainly hasn't helped.</comment>


"10x engineers": Stereotypes and research - <AUTHOR> https://jasoncrawford.org/10x-engineers <comment><USER2> I've worked with many developers over many years.

I am a developer, but only a 1X developer.

But I can say without a doubt that I have met a small handful of people who are many, many times more productive than ordinary developers. Call them 10X if you want, or dismiss the idea that 10X exist, but fact is there are developers who are simply streets and miles ahead of others in terms of productivity.

Why people claim 10X developers don't exist surprises me. Maybe they haven't met any or weren't able to recognise such productivity.

One important thing to take into account when considering extremely productive developers is context:

happy personally productivity +1

unhappy personally productivity -1

deep experience with language/framework/tools/tech productivity +1

average experience with language/framework/tools/tech productivity +0

little experience with language/framework/tools/tech productivity -1

inspired by the mission productivity +1

ho-hum about the mission productivity +0

skeptical about the mission productivity -1

likes the boss/management/fellow team members productivity +1

dislikes the boss/management/fellow team members productivity -1

not impacted by organisational politics productivity +0

impacted by organisational politics productivity -1

provided best tools for development productivity +1

inadequate tools for development productivity -1

knows the codebase productivity +1

does not know the codebase productivity -1

etc etc on and on

<comment><USER3> I think the ability of a programmer to be 10x or 100x or 1000000x depends on the domain. If you are writing a sales tax calculator and need to implement 500 different local state sales tax regulations from a scanned Word document, your best developer is going to be maybe twice as fast as your average one.

But if the problem domain is "write a program to search the Web" or "write a program to allow people to send data-plan messages to each other from even very old feature phones" then the best developers can create millions of times the value of an average programmer.

<comment><USER4> oooh oooh oooh, what an interesting example! I recall an interesting comment on NH a while ago where the dev was describing their refactoring of some law code. The original code was if-then-else, and had the same 'shape' as the horrid law it was interpreting. Our intrepid dev redesigned it to use some kind of truth table instead, and was terribly proud. And then the next week the law was amended and a new clause added deep down in one subclause or something. It turned out that the clever algorithmic approach wasn't actually the most maintainable... So, using a sales tax calculator as an example, its possible for two different devs to take to completely different approaches, and for the betterness of one or the other to be in the eye of the beholder until the code has been maintained in production for a long time, by which time doubtless whichever approach was adopted is deemed to be the worse one as the other has no production experience so might be better ;)

<comment><USER5> Honestly, adding legislature into code is a losing game. Let the developer just deliver a flowchart engine so that even unpaid interns can get to modify the logic.

In my opinion, a whole part of being a productive developer is about empowering people without the technical chops. If someone getting paid 10% of your salary can complete your previous responsibilities, you're technically a 10x developer.

</comment></comment></comment></comment><comment><USER6> The same person might be a 1x or 10x developer, depending on the rest of the team. You also need to include politics, experience, processes and many other related bits which can have an impact on how much a potential '10x' dev can actually contribute (indeed - how is the 1-10x measured in the first place).

I had an 18 month period where I was, essentially, a "10x" developer, and a "1x" contributor. My skills didn't radically change in that time period. What changed was the environments and teams I was in. I wouldn't be a '10x' in every team/environment, and I'm not sure that anyone can.

<comment><USER7> Agreed. I'm a 1x guy, but my knowledge is considerable. Put me in a team where I can share that knowledge and I get valuable. The more bods I can share it with, the greater the value I can bring and it's demonstrably a lot. However I'm not an engine that does the work, more like a lever that others can use to multiply their lifting capacity.

My weakness is I assume everyone around me is as open to (constructive) criticism as I am, and willing to learn from it. This proves often not the case. tl;dr I rub people up the wrong way. This reduces my worth.

<comment><USER8> > My weakness is I assume everyone around me is as open to (constructive) > criticism as I am, and willing to learn from it. This proves often not the > case. tl;dr I rub people up the wrong way. This reduces my worth.

Of course another explanation might be that

\- your criticism is not constructive \- your knowledge might not always be relevant \- you might not have as much knowledge as you think \- the incorporation of your knowledge might not be what's best for the organisation at that point (e.g. the time investment might not be worth it) \- something else

There's no way to tell for us here, but do not automatically assume that the problem is other's willingness to accept criticism (or your ability to make them accept it). It might be the case, but it might not.

<comment><USER7> > your criticism is not constructive

It always is. I never criticise destructively.

> your knowledge might not always be relevant

Then I expect to be told "no, you're wrong, here's why..." and then we can discuss, and one or both of us can learn something.

> you might not have as much knowledge as you think

Always true! However if the other can indicate areas of ignorance, I appreciate that and will go away and learn about it.

> the incorporation of your knowledge might not be what's best for the > organisation at that point

I see my job as supporting the business. I happen to do that with technical tools, but that doesn't change the fact that I'm there to support the company because it pays my wages.

> something else

Probably I'm just insensitive/thoughtless and should learn to think before speaking. But definitely a fair fraction of programmers get defensive about their code.

Anyway, good points, upvoted.

</comment></comment></comment></comment><comment><USER9> It's a nice write up, but the methodology is based on productivity as measured against coding, debugging, and execution speeds - which does really capture how much more impact a good engineer can bring.

The reason is that the impact can't me measured because we can't foresee the inventiveness of a brilliant engineer - if we could foresee then we are that brilliant engineer and are inventing ourselves. For the same reason the act of predicting scientific research is impossible because that would be the very product of research.

There's no upper bound on what a brilliant engineers can do, I've known people that you could not replace with 1000 mediocre engineers working for 10 years and they would't produce the same innovation, creativity and automation.

<comment><USER10> Some of the people I've encountered over the years who were praised for their speed and "productivity" dragged down the project (and/or the rest of the team) by introducing more bugs (and risk) than anyone else. Upfront perception of "productivity" masks the long-term cost, in so many cases.

IMO the true "10X" developer is the one who boosts the rest of the team through mentorship and leadership -- the polar opposite of the "lone keyboard warrior" frequently portrayed as godlike and outweighing any other team member.

<comment><USER11> Yeah, some people are -1x developers. They create more problems than they solve.

<comment><USER12> I think true -1x devs are also rare like 10x devs. To be -1x everything you "contribute" has to be a net negative. They are out there, but I think they're also rare.

<comment><USER13> I don't think so at all. It's really, really easy to slow down a team. The most common -1x behavior I've encountered is committing code that breaks something else, poorly executed "refactoring", misunderstanding the requirements of a task, and asking too many questions.

I've worked with -1x developers and have been one myself. I'm sure lots of other people here have committed these infractions at some point or another. But consistent offenders end up working on low-importance silo projects.

<comment><USER12> We all do that stuff from time to time, but I rarely find someone who is consistently that bad in everything they do. I have seen them, but my point is they are also rare.

</comment></comment></comment></comment></comment></comment><comment><USER14> I've worked with 2 10x engineers in my career. I don't know if they were precisely 10x, but they were the most productive engineers I've ever seen. At the startup I worked at, this was the VP. When he left, the company almost collapsed because he was singlehanded doing so many different jobs that we didn't know about. I would say that was more of a negative than a positive, but just from the sheer amount of work he did, he was definitely more productive than the entire team.

I had another friend who was also extremely productive. He never graduated from college, but he was a programming genius. One day he decided to learn ANTLR. Then he decided to deconstruct our query language for our product into ANTLR and discovered several bugs and inconsistencies in the implementation that made it impossible for ANTLR to parse it. Then I said "Hmm, it would be really cool to use ANTLR to read our XDR files and spit out some code that would implement the migration between versions." He said "great idea!" and accomplished that over a weekend, hand-constructing the grammar by hand. He saved the team probably 1 month of work every release cycle. He was absolutely astounding and the best programmer I ever had the honor of working with.

<comment><USER15> This sounds great. Do you think that their ability to deliver 10x value is based on pure programming skills or is it something else?

I'm managing developers and the ones who are most able to develop software aren't necessarily the most productive ones. I would love to know if 10x developers have simply a different mindset or in essence: What is their core skill that enables them to be so much more productive? Being focused, reducing things to the essentials and managing ones will power and time seems to be a subset of that "10x skill".

<comment><USER14> It's mostly raw passion. You can't get a 9-to-5 worker excited enough to learn something from scratch over a weekend just to do something she thinks is "cool".

There's also a difference between a better programmer vs more productive. I will never be as productive as the two that I mentioned, but I'm pretty good at programming. I have enough experience to know how to develop a feature and a set of code such that it's easy to read, easy to maintain and doesn't have very many bugs. That's just something I've learned over time. Others may be much more productive than me, but I rarely have to revisit features due to bugs. So there are different measures based on what you want from a team.

<comment><USER15> > better programmer vs more productive

That's a good point. Honestly, I would prefer a team of developers that are like you vs. outliers that are by definition hard to find and maybe even more difficult to manage. You sound above-average w.r.t your work ethic which is also hard to find and even more important for a strong development team.

> You can't get a 9-to-5 worker excited enough

Would love to know how to achieve that. I use psychological knowledge in my leadership and may have found a way to give people a way to show their full potential. But I'll make further tests before I write about it. Maybe you already have some well-tested tips - I found this infographic and liked it [1].

[1]: [https://www.visualcapitalist.com/10-proven-ways-to-build- tru...](https://www.visualcapitalist.com/10-proven-ways-to-build-trust-with- employees/)

<comment><USER16> Let me simplify it for you: pay them.

A 50k developer in 1980 should be making 260k today if they just following the growth of the economy.

</comment></comment></comment></comment></comment><comment><USER17> One thing that I think gets lost in this conversation is the 0x engineer. I've met a few people who I'd put into this category and I've definitely met people who think of themselves in the 10x mold who I'd categorize as 0x. Ironically in my experience 0x engineers tend to be really good programmers, but they just don't know how to understand or don't care at a deep level what the business wants so they just build whatever is convenient or enjoyable for them to build.They get away with this on poorly managed projects led by non technical people. Eventually, deep into the project the non technical people start to figure out that the thing they are getting is going to be really poorly suited to their needs and by that time they are over commited and totally screwed. Zero or negative value could be delivered over an extended period of time here.

Compared to the 0x developer, a junior developer who gets a basic understanding of what is needed then solved it with a standard well suited tool is an infiniteX developer.

And there is a huge space in the middle here. But I'd venture a guess that all said and done, deeply understanding the problem space and client or business needs correlates more highly with total productivity than programming wizardry over the long run.

<comment><USER18> And don't forget the -10x engineer. The guy on the team who creates something others depend on, hasn't got the skill to fix its bugs and has the "well there's obviously nothing wrong with my code" personality type.

<comment><USER19> A -10x engineer wouldn't be capable of creating anything from scratch, even if they did they would likely have issues getting buy in from the wider organization for said code to go into production. They would spend their days engaged in the wonton destruction of previously stable applications, wasting the time of competent devs with incessant questions and ultimately creating a net loss in wider organizational output.

<comment><USER16> I've seen -10 x developers put in management positions. The company is now going under for obvious reasons.

<comment><USER17> I read this like 10 days ago and didn't catch the joke. Well played.

</comment></comment></comment></comment></comment><comment><USER20> This is an eternally frustrating topic. The 10x effect is very real, not mysterious ... and also somehow very overhyped.

The original research in the 1970s didn't claim some programmers were 10x average, just that some were 10x some others. That is directly observable in any shop with more than a half dozen people.

There are "10x typists" too. I type 65 words a minute, sad as a typist but fast for an engineer. Our admin types something crazy like 170 wpm. People who never have a typing class can be 15-20 wpm. 170/17 = 10x. I'm sure a real typist would say 17 wpm people "can't type." From that person's perspective it's true.

<comment><USER21> I took an informal survey of engineers and most type 80+ wpm. 65 seems slow if you type as part of your job.

<comment><USER22> I'm typing way faster when I'm typing code than when I'm writing text, because I know most of my IDE's keyboard shortcuts and hence can type 5 lines of code with very limited key strokes.

</comment></comment></comment><comment><USER23> As an engineering leader, I can assure you that 10X engineers most certainly exist. I have seen them at play in the wild... (software and mechanical)

Step back from the tactical metrics and take a strategic view of an individual's contribution to team performance:

\- % of major tasks completed within agreed-upon dates?

\- # of emergency patches / roll-backs due to their code?

\- # hours of management time needed for briefing & QA?

\- % likelihood of exceeding expectations on design tasks

\- % likelihood of expediting a major change successfully

\- Support hours required for their code (zero is ideal)

\- # team hours lost from dealing with random BS / drama

Look at your engineers; you will definitely see outliers. There will be an 80 / 20 in terms of who you should be watching and a handful that you need to let roam free...

</comment><comment><USER24> This is like saying Michael Jordan doesn't exist.

He does exist, and when he was playing, he elevated the whole team. When he was paired with Pippen, it was amazing.

It wasn't just that he scored, but how the rest of the team responded.

10x engineers are like this. They are often good on their own, but pair them with a good team and watch out.

<comment><USER25> In the field of scientific research it's obvious that some people are much more productive than the average/worst people. Einstein research output is easily 10x the output of the average physics prof at the top 50 unis in the world.

The question is simply what percentage of people in a given field are 10x, not if they exist.

<comment><USER26> The question isn't whether they exist, but whether it's just "people" or "people at a given time/place" and in some cases, environment that takes a 1-2x developer and makes them 10x.

</comment></comment></comment><comment><USER27> The whole discussion is derailed anyhow. Most knee-jerk reactions I've read so far seem to deny the existence of any productivity difference between individuals at all or claim that those 10x engineers must be so efficient because they're anti-social assholes.

<comment><USER2> Well the 10X developers I have know have mostly been pretty normal nice people.

One of them wasn't even super interested in development outside working hours - he just came to work, seemed to knock it out of the park and go home. I'm not saying he wasn't interested at all - he was mildly interested outside work.

<comment><USER28> That is probably part of why he is 10x - he works at peak efficency instead of masochistic "long hours mean dedication!" cults which forget the whole damn point is actual productivity.

</comment></comment></comment><comment><USER29> My take on this is, 10X _what_? What is the metric for code that we're comparing? Is it correctness? Speed? Reliability? Fault tolerance? Readability? Intuitiveness of design? Fast development? Someone easy to collaborate with? Excellence at debugging/fixing/refactoring code?

The best programmers I know are 10x better at no more than a couple of these metrics, while every coder expresses a different spectrum of credits/debits across the skill areas than the 'norm'.

Successful code itself seldom serves more than a couple of these metrics well. Fast code is almost never fault tolerant. Or readable. Or easy to refactor.

What's more, the creator of one 10x code is unlikely to easily shift his/her skill set ideally well to produce 10x code on a different project with very different priorities. What's more likely is that the 10x coder's next project will be implemented much like his/her last, regardless of whether the project's design criteria demanded otherwise. So across different projects, the same 10x coder won't deliver 10x results on all, even if s/he performs equally well on the same metrics -- because the design requirements have changed.

Like writers, programmers aren't good at adopting different writing styles for different roles or domains. The skills needed to design and write 10x code for an operating system or math library do not extend to a 10x user interface or social media app.

We really need to stop judging complex systems using single metrics. Oversimplification sucks.

</comment><comment><USER30> Not sure what the proper origin of the debate on 10x was, and if it ever was talking about productivity. In my mind the point about the 10x developers had always been about the fact that some developers' VALUE (not productivity) is 10x. And in that sense it seems to me that the factor 10x is, if anything, an understatement.

A room full of 100 average physicists, or 1000 bad physicists, would never never have been able to make the contribution to physics that one Albert Einstein made.

And I think it's the same with developers. It's not rare to have a team of 10 developers with one individual whose creativity and problem solving ability means that the product that this team is building is better than any competitor's and that none of the problems that arise will stop the team's ultimate success. In this sense it might well be ONE engineer who is the catalyst for a chemical reaction that allows the other 9 to have an opportunity that sheer productivity can end up getting converted into value, the thought experiment being: If you take the team away from that guy, he could still build it by himself (but would take 10x as long, compared to how long it would take the team to do it). But if you take that guy out of the team, then the other 9 could expend infinite resources and would never get anywhere.

<comment><USER31> Your analysis might be sound, but I think your physicist example is off.

[https://en.wikipedia.org/wiki/List_of_multiple_discoveries#2...](https://en.wikipedia.org/wiki/List_of_multiple_discoveries#21st_century)

Ctrl+F Albert Einstein.

Many things in history are discovered very close to one another--suggesting that maybe 100 or so average physicists (not sure what this means though, most people who get to that level are extremely intelligent), would probably work out what Einstein did. People have suggested discovery is "inevitable".

"But if you take that guy out of the team, then the other 9 could expend infinite resources and would never get anywhere." This leads me to believe the physicist example is off here.

<comment><USER32> I think, in Physics, the key is not the discoveries, but the understanding.

It's often a single change in perception that unlocks a raft of related discoveries. Which also makes reasoning about such things hard, because once that perception shift has been identified, previously hard-to-understand things become easy, making the initial discovery look trivial.

That change in perception can happen from a small group of people or from a single person.

</comment></comment></comment><comment><USER33> 10x+ engineers definitely exist within a given language and tooling stack. If they've got it all roughly memorized from top to bottom -- including all the quirks or pain points to avoid -- the ease of expressing a high-level use case in code like "I want the program to do X" becomes much more rapid. They typically also get it right the first time with very few bugs and cover most of the edge cases.

I have yet to meet anyone who is 10x+ across environments and tooling setups. It'd be like expecting a master mechanic from Ford who can do an engine rebuild in half a day to step into a Subaru shop and do the same thing. Highly unlikely. Probably still very competent, but not working at that 10x factor.

The thing is most programmers these days don't get the time to go deep into a given tooling stack. Or they do and once they've begun to achieve mastery the old stack is no longer popular or what the boss wants, or it's on to a new contract with different requirements that prescribe new tools, etc.

<comment><USER34> Absolutely no and this is again another myth. The engineers in top of their game are fairly independent of language and stacks. Learning required tools at accelerated speed is a hallmark of top engineers. Most of them often create new stacks and languages from the scratch. Here are some examples of 10x engineers: [https://en.m.wikipedia.org/wiki/Coders_at_work](https://en.m.wikipedia.org/wiki/Coders_at_work). I don't think they were 10X only for certain language or stack.

</comment></comment><comment><USER35> In my decades of experience I have yet to meet this mythical 10x or 100x programmer, but what I _have_ seen is every once in a while somebody (probably a programmer but not necessarily) is able to take the reins in implementing a new process, or tool, or framework, or whatever, which dramatically increases productivity for the whole team. Conversely, I've seen programmers who don't try to get on board any such possible initiatives and who drag the team down as a result.

So for me, the question isn't how productive one person is. It's how productive a collective can be as they work together, and how fast is the rate of improvement over time?

<comment><USER36> > but what I _have_ seen is every once in a while somebody (probably a > programmer but not necessarily) is able to [..] which dramatically increases > productivity for the whole team.

One of the common alternate explanations of 10x I've seen puts those people into the group: Adding a 1x person to a team results in +1 output for the whole team, while adding a 10x person to a team results in +10 output for the whole team regardless of how much that single person outputs on their own.

The business jargon for this type of 10x-er is "force multiplier". Whether or not they do a lot themselves doesn't matter, they bring the rest of the team up a level to where it's as if multiple people were added to the team.

</comment></comment><comment><USER37> It may be worth emphasizing that the context of this post is a Twitter thread claiming 10x developers were worth actively recruiting even though they don't mentor, keep odd hours, don't communicate, etc. there are certainly exceptional engineers especially in the context of a given company. There are also prima donnas who may be talented but in most cases justify a hard pass.

<comment><USER38> That twitter thread was a train wreck in so many aspects. It favored individual performance over team work, it suggested a lot of supposed "telltale" signs that were so unrelated to actual work that reminded me of phrenology, and it even included some things that are counterproductive or that you really don't want in a development team.

If anything, that thread described cowboy programmers, not 10x engineers.

<comment><USER34> That thread certainly tried to box 10x devs into stereotypes. People like Jeff Dean or Steve Wozniak don't really fit into it (but Linus Torvalds probably does). However, I'd say a many types of work require synthesis from one brain which has obsessively prepared itself over the years. I doubt if any amount of team work by usual musicians or painters could have produce 9th Symphony or The Starry Night.

<comment><USER37> Those things you describe are more or less fundamentally individual endeavors, however. I'll concede that there may be narrow domains where such work can be a net positive within a larger organization but it's not the way to bet.

The fact of the matter is that the tech industry is based _far_ more on collaboration and collective innovation than individual acts of anti-social genius.

<comment><USER34> Yes, that's why I said "many types of work", not all types of work. I don't expect any single genius working alone to put man on the moon or build modern cloud service or entire smartphone OS. However, I also don't expect any committee to invent great programming language or font or new architectural paradigms from scratch.

</comment></comment></comment></comment></comment><comment><USER39> I know a >40x developer. We started trying to get load of that single person, because they were heavily overload, were a single point of failure, and were doing so much that it was impossible for other people to grow new responsibilities.

We replaced part of that person with 2 teams of 20 people doing a small part of the work that this person was doing before. This person is still part of the two teams, in an advisory role, although they started as team leaders, but part of their job was now to grow new leaders that would replace them.

This person is still a >40x developer, still overworked, still doing way too much. Now we are trying to replace another of their responsibilities, with another 20 person team.

I'm pretty sure that once we do that, this developer will still be a >40x developer, just doing other things, that at some point, we'll need to replace.

FWIW, this person just does too much, because all that much must be done, and nobody does it, so they just take on it and do it, really well, and really fast. Most of this stuff doesn't even interest this person. There are some things that interest them, and that's what they want to focus and work on, but they still do all this other stuff.

So "replacing" them is actually something they want, but it takes time to recognize these problems, figure out how to solve them, grow people that can do the job, etc.

When I look at these two teams, and all the work they do, the only thing I can think of is wow, we didn't even know this 40x dev was doing to much. The dev didn't have time to complain, or write reports, or anything, because they were to busy doing all the stuff and putting out all the fires. It was only when they started saying that they would like to focus on this or that, and if we could find someone to take on this or that load, that we realized that no single person could take on these loads.

</comment><comment><USER40> _Additionally, Brooks muses that "good" programmers are generally five to ten times as productive as mediocre ones._

[https://en.wikipedia.org/wiki/The_Mythical_Man- Month](https://en.wikipedia.org/wiki/The_Mythical_Man-Month)

"In one of their studies, Sackman, Erikson, and Grant were measuring performances of a group of experienced programmers. Within just this group, the ratios between best and worst performances averaged about 10:1 on productivity measurements and an amazing 5:1 on program speed and space measurements!" – The Mythical Man-Month

[https://10xfive.com/2018/07/31/origin- of-10x-principle/](https://10xfive.com/2018/07/31/origin-of-10x-principle/)

<comment><USER41> Yeah, the study by Sackman, Erikson, and Grant is exactly the study that originated the concept, and it's the first one listed in Steve McConnell's survey, which I link to in the post.

<comment><USER40> Yeah, but Brooks was the one who first popularised the idea. Mythical Man Month was published in 1975.

And of course he didn't just point to the study, he just used the study to back up what he and many others knew. He also proposed a solution to the problem, the "surgical team", to get out of the idiotic situation of promoting the best programmers to no longer do programming.

</comment></comment></comment><comment><USER42> Why doesn't anyone talk about compensation? Does 10x engineer gets paid 10x as well? Or taking 1x compensation is another feature of the 10x engineer?

<comment><USER43> One way that I approached this was to start my own SaaS product business.

First, it's a good way to test whether you actually are as good and productive as you think you are, since you now have to build an entire software product all by yourself. The ratio of deadwood at your company is going to be either zero or one.

But the nice part is that when you have one person at your company, you get to keep all the profit for yourself. And there's no reason to work 40 hour weeks (assuming you really are mr rock star like you think) so you only need to bring in say $10k/month and work less than 200 hours per year (1/10 of 2080, remember) to see an hourly rate that's more than 10X your 9-5 contemporaries.

Sorted.

</comment></comment><comment><USER44> There are probably a lot of similarities to a "10x line cook"

I've worked at a few restaurants, and you want one of these guys during your dinner rush. With him, it won't even feel like the busiest night of the week.

It's amazing to see them in action. They're fast. They don't putz around. They don't screw up tickets, causing them to re-do their work. They do everything in the right order, keep their grill organized, keep their station in order.

The worst in the kitchen is a lump who gets flustered when they have more than 4-5 tickets at once, has an un-prepared station, screws up every 10th ticket...

</comment><comment><USER45> > The best explanation of the concept and its basis the article "Productivity > Variations Among Developers and Teams: The Origin of 10x" by Steve > McConnell. Here are the key things to know about "10x engineers":

> 10x refers to the difference between the best and worst developers, not the > best and average.

The cited article is using a different definition of 10x engineer than I'm aware of. There's hardly a limit to the worst and I'd believe 100x or 1000x variation with this interpretation. In fact the worst are negative so don't even work on this scale.

<comment><USER46> If you had kept going down the list, you'd see this:

> The studies only compare differences among developers who _actually complete > the task._

> Their figures don't take into account the people (~10% in some studies) who > didn't even finish. Nor can they take into account the real-world cost of > software that is nominally completed but is so buggy, flaky, or hard to > maintain that it has to be rewritten by someone else."

<comment><USER45> I still wouldn't call anyone who eventually completes a task given indefinite time to an unspecified level of qualify to be 1x.

</comment></comment></comment><comment><USER47> I think one factor for 10x productivity is to be in an environment that allows the flexibility to do things in a different way. Most companies prefer tight control so you can't really perform at your best.

</comment><comment><USER48> Listed under "what I believe" the author writes:

> Productivity is not strongly correlated with experience.

This seems contrary to my experience and expectations, although maybe the word "strongly" is what makes it true.

> The inherent traits have a lot to do with intelligence and other thinking > patterns that we can't (or don't yet know how to) identify and teach.

This also doesn't really match up with my experience. I've had employees start as net negative contributors, and then grow into solid positive contributors over time as they learned and developed their skills.

Either way, the explanatory factors seem like the most interesting thing to talk about: what makes one engineer more productive than another?

</comment><comment><USER49> Since it usually comes up in this discussion, Fabrice Ballard: [https://bellard.org/](https://bellard.org/)

<comment><USER2> I'm not going to specifically address Fabrice, but one thing I have noticed is that people who develop almost purely in one language, within a constrained application space, are able to become extremely productive because their knowledge and experience is very focused.

Consider for example someone who does nothing but C programming on a Linux platform. Say for example they build compilers. And say that's pretty much all they do. That person can become incredibly productive because they know the language and the tools extremely well.

Or maybe someone who writes computer games in Java. The only thing they need to know, and the only thing they have done, for many years is write graphics/games code in Java. That allows them to become extremely productive at writing games in Java. I've wondered about this in regards to Notch (Minecraft developer).

Now consider the modern full stack developer - there's a constant, never ending need to learn new things. New languages, new frameworks, HTML, CSS, JavaScript, ReactJS/VueJS/whatever, SQL, some backend language/Python/PHP/Java/C#, cloud operations EC2/Azure, Linux along with the many related toos like debugging etc etc. This sort of developer rarely comes up to mega expert level because they can't - they need to be very good at learning just enough to get the job done in a wide range of things. Likely they will have expert level in some of these technologies, but the point is that being able to dedicate your entire brainspace to only a few major technologies allows super productivity. The full stack developer is unlikely to be incredibly productive in one particular area.

The other thing that destroys my productivity as a full stack developer is hitting problems/roadblocks in some new thing that you need, but don't know well. This sort of thing can take days of problem solving, where an expert in that particular technology might be able to recognise and fix the issue in seconds. A good full stack developer must be very very good at problem solving because if you can't fix problems then you'll never get to write much application code.

On the other hand, a really good full stack developer should absolutely amaze you with their ability to build every single aspect of a large system including back end, front end, deployment and operations.

<comment><USER50> Bellard isn't a 10x, he's more like a 100x.

 _people who develop almost purely in one language, within a constrained application space, are able to become extremely productive because their knowledge and experience is very focused._

He mainly uses C and some JavaScript, but a look at all the things he's written suggests that he doesn't really focus on any one area; the first few projects on his page are:

\- A Javascript engine (discussed at [https://news.ycombinator.com/item?id=20411154](https://news.ycombinator.com/item?id=20411154))

\- Neural networks

\- Image compression

\- Arbitrary precision floating-point

\- JavaScript PC emulator

\- 4G/5G base station software

\- Tiny C compiler

\- A clone of the text editor Emacs

\- Pi calculation

<comment><USER51> What's more impressive than the programming aspect is the sheer depth and breadth of domain expertise.

</comment></comment></comment></comment><comment><USER52> If you want to be perceived as an 10x engineer, be the engineer who starts the codebase, write all the code according to your preferences, and maintain a position as arbiter of what the "right way" to do things is in the codebase as the team grows.

</comment><comment><USER53> I love how programmer Twitter is in total butt hurt denial over this meanwhile Linus Torvalds created git in like a weekend.

I have no problem admitting that guy is vastly more productive than me. I don't understand all the FUD.

<comment><USER54> It's also really dangerous. Maybe I am just really autistic, but I have a real difficulty comparing myself to others, mostly because there isn't enough reliable data available.

The funny thing is: If I don't compare myself at all to other people and just try to "be", I quickly run into problems: People find me condescending because I use words they've never heard.

So, I have to actually think about how smart I am, and how less smart other people are. If I factor this into account, and then engage people with the right vocabulary, I never run into problems. But ironically, this is actually condescending, but the audience never knows about it.

It's less good for me to describe myself as "normal", because actual "normal" people get "deranked". Society's asking me to see myself as the "smart" guy and act accordingly.

Honestly, I prefer to not compare and just be. I understand, or I don't understand, and I have no problem admitting either. I thought this was normal, but it appears, it isn't... in fact, some people would (maybe even literally) rather die than admit they don't understand something, and while most aren't this extreme, this kind of behavior seems to be the norm rather than the exception...

I guess it's more socially acceptable to not give credit and respect to 10x where it's due, because people probably expect these people to be so successful just the way they are, they believe they wouldn't require it.

EDIT: To come back around as to why this is dangerous: I've alienated most everyone of my friends and family in the process to learn the above. It would've really helped me if somebody just told me "You're damn smart, other people aren't, treat them like kids or something." Sure this sounds condescending, but read the above, it would've kept things not just for me, but more importantly, my friends and family, much more comfortable. But it's too late now, already.

EDIT2: In conclusion, talking to people requires active effort from me, and I find it very frustrating and annoying. So I prefer to be alone, and sometimes will be harsh, abrasive and arrogant in a preemptive effort to keep people away from me... My greatest fear is accidentally pushing someone away whom I'd rather not (there are people whose company I do enjoy), but if my logic holds up, that person should understand that...

<comment><USER53> _I understand, or I don 't understand, and I have no problem admitting either. I thought this was normal, but it appears, it isn't... in fact, some people would (maybe even literally) rather die than admit they don't understand something_

This was a costly lesson for me to learn. Your general predicament has been referred to as The Curse of the High IQ (author Aaron Clarey).

Once you realize how much smarter you are than the norm, you realize that talking to someone of average intelligence is the same as someone of average intelligence talking to someone who is mentally handicapped. 2 std dev is 2 std dev.

Anyway I highly recommend Clarey's book.

</comment></comment></comment><comment><USER55> I have worked with -N (negative N) developers before, where the team would be N times more efficient if they were ejected, does this count?

<comment><USER56> Yes. The 10x number says more about your low performers than your high performers.

</comment></comment><comment><USER34> Original Twitter thread: [https://mobile.twitter.com/skirani/status/114930282842006732...](https://mobile.twitter.com/skirani/status/1149302828420067328)

The guy got lit by Twitter crowd for claiming that 10X devs exists, are introvert and often lacks deep empathy.

<comment><USER57> His #3 way to spot a 10x engineer is that their "laptop screen background color is typically black".

<comment><USER34> This might be stab at people who use certain IDEs. Historically many IDEs were said to be designed for "morts" (aka enterprise developers on Windows) and "real" programmers supposed to be using heavily customized Vim/Emacs on Unix. Tools like Visual Studio, Delphi, IntelliJ etc had white default backgrounds which arguably strained your eyes if you worked long hours. In old days, black themed tools is what kool kids did at funky startups and white skinned tools is what tie wearing programmers did at insurance companies (this is obviously not universally true, but I think that was the perception).

For many years, most of these IDEs now have black themes and VS even now has it as default I think since 2015. Like all animals, we collect data (aka set of anecdotes) and build statistical model of word around us. Sometimes these models are hopelessly narrow, have too many false predictions and very quickly gets outdated.

</comment></comment></comment><comment><USER58> I've been a 10x developer at various points in my career. If you connect with the project on a deeper level the code can just flow.

It doesn't last and you end up chasing this feeling.

It's easier to be a 10x developer when its your own project with your perfect setup.

</comment><comment><USER59> There are probably _100x_ and _1000x_ engineers, but their true value can't be easily captured with any metric, at least not on teams and projects of non- trivial size and duration.

The value of an engineer is not _what they did on a given day._ It lies in the architectural decisions they make. Those decisions have ramifications over the following _years._

I have seen "genius 10x engineers" make _egregious_ architecture decisions (both micro- and macro) that have cost thousands upon thousands of employee- hours of lost productivity in the ensuing decades. Those developers were often regarded as "geniuses" and "highly productive" because they spent thousands of hours simply fixing the very problems they caused in the first place! Those same developers have also made a number of decisions that avoided similar pitfalls, thus saving the company thousands upon thousands of lost hours. So were those good engineers or bad ones?

Research that all you want, but I've never seen anybody propose a metric that would even remotely capture this sort of long-term net value created (or harm done) by engineers.

To even begin to answer this question, we'd need to take an extremely long view. We'd need to look at multi-year projects, the architectural decisions made, and somehow correlate them with outcomes. I'm not sure how you'd quantitatively define the success of a software project in the first place, much less map actions to outcomes.

Especially since the evaluation of software architecture decisions really involves comparing _roads taken_ versus _the roads not taken._ Suppose engineer A chose language XYZ for a particular solution. Was it the right call? Well, what you're really trying to evaluate is the choice of XYZ against other choices that might hypothetically be made.

And, on top of that, engineering decisions are typically constrained by outside factors. Engineers often make bad decisions because they're forced to. Perhaps XYZ was an utterly terrible choice for the task at hand, but this engineer had nothing but a team of XYZ developers at their disposal. Management had a hiring freeze in place and deadlines were tight, making the choice of anything but XYZ nearly impossible. So how do you judge the effectiveness of _that_ engineer?

I (like most people reading this) have made some absolutely disgusting engineering decisions in my life simply due to constraints like this. I once hardcoded several megabytes of data into a PHP array instead of using a database like a normal human being. This is, by any reasonable standard, a pretty awful decision. You won't find it in any books.

However, this was a very short-term project. The data was read-only. The deployment environment _had no database available._ And we were facing a deadline measured in hours. (This was before the days of services like AWS / Digital Ocean / etc making it easy to spin up a server stack) After a bit of testing I discovered my kludge didn't perform too badly after the first page load since I guess PHP execution plans get cached somewhere by some app servers. So I shipped it, the site worked, and we got paid.

Was I a 10x engineer there? Or 0.1x? Or 1x? I have no idea, and frankly the question itself seems silly.

</comment><comment><USER60> I personally think high individuality productivity is related to

1\. an ability to avoid screwing around, procrastinating, implementing things that are fun rather than things that are useful.

2\. innate ability to avoid yak shaving or dump it on others.

</comment><comment><USER61> My anecdote: I've been what one might call "10x" on some projects, and 1x on others.

In a position where I built the system/product from scratch, I'm the "10x" person. This is because I know the system's ins and outs better than I know myself. Of course I'm more productive with it. Of course I know exactly where to look when a bug arises. Of course I'm 10x better (across whatever arbitrary productivity dimension) than the new hire. It's not because I'm magical, it's because I was there when there was nothing.

Drop me into a new stack, with a new system, and I'll be -1x for at least a few months. Eventually I'll work my way to productivity, and that's fine. It's expected.

I think developer productivity has a lot less to do with technical skill than many people would like to think. I agree heavily with the article's points that "work environment matters a lot" and "productivity is a combination of inherent traits and acquired skills."

</comment><comment><USER62> If you find yourself to be 10x engineer, you probably need a better team.

<comment><USER63> One way to do this is to find a better job.

Another way to do it is to put more effort into improving the quality of your current team. In the long run, that's a skill you're gonna need.

</comment></comment><comment><USER64> "Do not tolerate brilliant jerks. The cost to teamwork is too high." —Reed Hastings, CEO, Netflix

<comment><USER65> But what if they are brilliant team players?

I have to agree with another user named Amatecha higher up the thread who says:

>"IMO the true "10X developer" is the one who

>boosts the rest of the team through mentorship and leadership -- the

>polar opposite of the "lone keyboard warrior" frequently portrayed as

>godlike and outweighing any other team member."

<comment><USER64> The problem here is with the definition. Most people don't define a 10x developer as someone who is a team player and more like the lone ranger you describe.

See this recent discussion on twitter: [https://twitter.com/skirani/status/1149302828420067328?s=19](https://twitter.com/skirani/status/1149302828420067328?s=19)

</comment></comment></comment><comment><USER66> I've met some wizards. I'm not one of them and that's fine. I don't do any tech stuff once I'm home (I mean until my kids get into Lego mindstorms and such).

But I think being a 1x dev is pretty darn good when a lot of devs I've met are like 0.5x or even the occasional -x.

<comment><USER67> Literally the first bullet in the article's body says that there is no such thing as a <1x engineer. What am I missing in what you are writing?

<comment><USER66> I don't agree with the bullet. I've worked with people in the past who basically just didn't do their job if nobody was watching and would find ways to stretch their projects out to fill extra time. And I've met people who actively harmed the work environment and code. It's amazing how some people can get away with things and not be fired.

Which leads me to the feeling that if you come to work and consistently do an adequate job, I'd love to work with you. I don't need wizards on my team, I just need the basics.

</comment></comment></comment><comment><USER68> Maybe it's not that there are 10x engineers, it's just that there are so many 0.1x engineers

<comment><USER69> As stated in the article, 1x refers to the least productive engineer – not the average one.

</comment></comment><comment><USER70> I don't consider myself a 10x developer, but I can be more productive than my peers on day to day tasks. The difference between me and them is automation. Stuff like executing scripts on the database, scaffolding new business objects I just execute a PowerShell script and its done. I focus a lot on productivity and keeping frustrating or boring tasks to a minimum and automation helps me achieve that.

But that only helps me so far, in areas where they have more business knowledge, unless it's an obvious thing, they have the upper hand, while I read docs to understand what is required or what might be wrong they already know.

</comment><comment><USER71> [https://dynamicecology.wordpress.com/2014/01/23/william- shoc...](https://dynamicecology.wordpress.com/2014/01/23/william-shockley-on- what-makes-a-person-write-a-lot-of-papers-and-the-superstar-researcher- system/)

This is specifically about academia and publishing, but I don't think that's all that different from a lot of development practice. It presents a useful way to think about engineer productivity that doesn't get caught up in meaningless slogans.

</comment><comment><USER72> if the 10X is a comparison between the best and worst developers then there are 100X developers out there, during the early 2000s I once worked with a guy who spent a day wondering why his VBscript that instantiating an ActiveX object worked in Internet Explorer but not in Netscape. Yes, I or the other programmer in the office were more than 10X this guy, but I don't think it made us 10X developers.

</comment><comment><USER73> Experience counts.

Some people seem to have 10x and more motivation to practice and learn than others. Why is it so difficult to agree that this is a major advantage? Because you can't have it delivered on a silver plate at any price?

From my experience; the biggest difference is knowing what corners may be safely cut, which code doesn't even need to be written.

</comment><comment><USER74> I've hired a lot of engineers over the years who were better at engineering than me, but I can get 10x done faster because it's my vision/passion and I work 80 hours a week vs their 40 because it's my company (I am also full stack). So I say I understand the product and market 5x better and put in 2x the hours, which totals 10x.

<comment><USER75> Do you worry that your team will see you working 80 hours/week, conclude that they have to work that much too, and burn themselves out? Even if you can work that much without burning yourself out because it's your passion/vision/company, most people can't. It seems to me that employees often look to folks at the top to set the culture and feel obligated to follow along, even if that culture includes unsustainable work hours.

<comment><USER54> No, it's his company, if anything, I'd enjoy working for him. But only if my work contract clearly states that I need only put in X hours per week into work.

I worked in a startup, became the right-hand of the boss and owner, and then he'd slip into work only once per week. I did this for three years, my pay was... ok, but I didn't see the ability to develop any further or earn any more and eventually quit, and started my own company a few months ago.

I would've really appreciated if he would've done more, so that I could have focused more on actual technical work and not administrative or management work. Mabye I would've even stayed, I don't know.

Then again, I am very different than most people, so maybe this is just a very lonesome opinion...

</comment></comment></comment><comment><USER76> Modern engineering is a social endeavor.

</comment><comment><USER77> I do not discount the existance of more productive developers. But I do believe there's an ideological framing similar to the ceo cult personality. Also probably related to why you didn't consider a positive impact of organizational politics on productivity.

</comment><comment><USER78> you'd need a reasonable definition of "productive" before we can start measuring the spread between the least and most productive engineers. any reasonable definition wouldn't constrain the spread to a single order of magnitude. the real spread is infinite. I think people's intuition is backwards on this as well - if some engineer takes 10x less time to solve some (non-trivial) problem I'd expect some combination of underengineering andor oversimplicification which could potentially set you back by more than 10x your overall "productivity".

</comment><comment><USER69> The only way to become a 10x+ engineer is by enabling others to increase their productivity by mentoring, improving tooling and the overall environment, and making sure your team follows the big picture.

</comment><comment><USER79> Just here to link to [https://yosefk.com/blog/?s=10x](https://yosefk.com/blog/?s=10x)

</comment><comment><USER80> People are reacting badly to the idea of 10* engineers because it goes against the idea that we are all equals and can all make it.

I see it as a political stance, For some reason a lot of developers seem to believe in the idea of "equality of outcome" which means that we should all have similar outputs and if for some reason some of us are more productive it must be because of some systemic prejudice.

I personally believe that there are excellent developers out there. Some of them are brilliant jerks but some of them are also brilliant developers without being jerks. Those later ones are the real 10* engineers.

<comment><USER81> This is also how I perceive the 10x discourse. The Twitter thread - much more so than this HN thread - was quite revealing of this political/worldview aspect. Liberal views tend to encourage us to think that we are all equal and have the same opportunities. And obviously the idea that there are coworkers out that literally "10 times" as productive goes against this and receives a hostile reaction.

</comment></comment><comment><USER82> Not sure if I've met a 10x developer.

But there are 1/10x devs and managers for sure :)

...bring one into your team and overall productivity drops by an order of magnitude.

<comment><USER83> Dont forget -1x developers [http://wiki.c2.com/?NetNegativeProducingProgrammer](http://wiki.c2.com/?NetNegativeProducingProgrammer)

</comment></comment><comment><USER84> There are definitely engineers who can solve the same problem using 10x the memory and cpu cycles of a really top-notch engineer... are these the mythical 10x engineers?

</comment><comment><USER85> I believe in Gaussian distributions.

</comment><comment><USER86> > we can't say anything more than "roughly order of magnitude"

This, to me, is the core of it. "10x" is just a shorthand for the compound interest of good decisions. Nobody would be surprised to learn that teams with good managers are 10x more productive, or that factories with a good safety culture have 0.1x the accident rate.

Where it gets confusing is that developers have substantially more scope for compounding than their roles suggest. They're generally not managers, executives, or cultural leaders, and yet their decisions seem to compound anyway. Why is this?

For the answer, I encourage you to watch the YouTube channel "Primitive Technology", where a brave and often shirtless soul spends weeks and months constructing tools and structures from scratch. He does so with a deftness and skill that is captivating, yet the stuff he makes is, by modern standards, totally useless. It's not his fault; it's just that technology compounds, and no amount of individual skill can ever make a skyscraper out of wattle and daub.

Only in software, the most questionable of all the engineerings, do we build the processes and tools we depend on while we're using them. If you showed up to a job site talking about making your own concrete mixer to get the building done faster you'd be laughed right back to wherever you came from. Yes, in cutting-edge applications and prototype manufacturing it's different, but almost every other area of engineering takes stuff that works and uses it to make more stuff that works.

To be clear, this isn't a long-form argument for "we should have stuck with Rails", nor is it "software is just getting started, give it some time". Rather, I believe that software development is essentially and unavoidably compounding. Every design decision, abstraction, function and data structure is a piece of your foundation that you build and then stand upon to build some more. We're creating abstract machines, and when they become concrete enough to rely upon they no longer need software developers.

Which is why it's so ridiculous to imagine this 10^x developer who crushes code 24/7, laughs in the face of process or documentation, and communicates only via a colourful aura of cheeto dust and misanthropy. It's an adolescent fantasy of expertise, no different from Doctor House or Detective Batman. Sophomoric macho bullshit. The 10x isn't the person, it's the compounding effect of good decisions.

Peter Norvig is a great developer, but try airdropping him into some dumpster fire codebase that's 99% finished, riddled with bugs and way behind schedule. Is he going to grab his wizard hat and 10x his way out of it overnight? No. He's going to have to slog through the mess like anyone else. His expertise doesn't result in faster typing, but in better decisions. Decisions that work well now, but enable even better work later.

Most importantly, this kind of compounding doesn't just apply to you, but to the people you work with and the environment you work in. To enable that, you need communication, leadership and generosity. Here's the man himself, describing his work at Google [0]:

> I've varied from having two to two hundred people reporting to me, which > means that sometimes I have very clear technical insight for every one of > the projects I'm involved with, and sometimes I have a higher-level view, > and I have to trust my teams to do the right thing. In those cases, my role > is more one of communication and matchmaker--to try to explain which > direction the company is going in and how a particular project fits in, and > to introduce the project team to the right collaborators, producers, and > consumers, but to let the team work out the details of how to reach their > goals.

Or some quotes from his essay "Teach yourself programming in ten years" [1]:

> Talk with other programmers; read other programs. This is more important > than any book or training course.

> Work on projects with other programmers. Be the best programmer on some > projects; be the worst on some others. When you're the best, you get to test > your abilities to lead a project, and to inspire others with your vision. > When you're the worst, you learn what the masters do, and you learn what > they don't like to do

> Work on projects after other programmers. Understand a program written by > someone else. See what it takes to understand and fix it when the original > programmers are not around. Think about how to design your programs to make > it easier for those who will maintain them after you.

Or check out his lavishly documented walkthrough of approaches to the Travelling Salesperson Problem[2] (just one of many similarly educational "pytudes"[3]). Or the leading AI textbook he co-wrote[4]. Or the online AI course he co-developed[5]...

That's what real 10x looks like. Not some myopic ubermensch who divides the world into "code" and "dumb", but a thoughtful decision-maker who treats great work as a garden to grow, rather than a race to win.

[0] [https://www.quora.com/What-does-Peter-Norvig-do-exactly- at-G...](https://www.quora.com/What-does-Peter-Norvig-do-exactly-at-Google- What-is-his-day-to-day-like-and-what-kind-of-work-does-he-prioritize)

[1] [http://norvig.com/21-days.html](http://norvig.com/21-days.html)

[2] [https://nbviewer.jupyter.org/github/norvig/pytudes/blob/mast...](https://nbviewer.jupyter.org/github/norvig/pytudes/blob/master/ipynb/TSP.ipynb)

[3] [https://github.com/norvig/pytudes](https://github.com/norvig/pytudes)

[4] [http://aima.cs.berkeley.edu/](http://aima.cs.berkeley.edu/)

[5] [https://www.udacity.com/course/intro-to-artificial- intellige...](https://www.udacity.com/course/intro-to-artificial-intelligence --cs271)

</comment><comment><USER87> mmh. my keyboard looks like his :D

</comment><comment><USER88> This is coming up b/c of the 10x engineer tweet from an Accel Ventures VC. This is how they defined a 10x engineer/what to look for in order to spot them:

\- They hate meetings (doesn't everybody?)

\- They keep irregular hours

\- They use a black desktop background

\- They wear out the i, f, x keys on their keyboard instead of a, s e (the latter 3 are apparently correlated to sending lots of emails somehow)

\- They know every line of code and can therefor immediately trace back any bug in prod to the exact line of code

\- They are full-stack engineers, code is code so they can do everything but they won't touch UI

\- They convert thought into code by caffeine fueled code binge sessions in which they implement any product feature over the span of 4-6 hours

\- They rarely if ever need to rely on documentation

\- They are always using the latest and newest tech

\- They are poor mentors and interviewers b/c: They always think "It takes too long to teach or discuss with others, I would rather do it myself."

\- They don't hack things, they write quality code and know exactly how the code needs and will evolve over time

\- They rarely job hunt or move out of a company

[https://twitter.com/skirani/status/1149302828420067328](https://twitter.com/skirani/status/1149302828420067328)

I can't even begin to comprehend how some of these are in any way an indicator of a 10x anything.

Keep that in mind as you read through this post.

<comment><USER2> What garbage.

EDIT: utter and complete garbage

Surely this is not written by a developer?

Update: [https://www.linkedin.com/in/kirani/](https://www.linkedin.com/in/kirani/)

The person who wrote that tweet is not a developer.

The conclusions this VC has drawn about what makes developers productive is _Voodoo_.

He's observed people who he thinks are productive, and gathered together his observations/prejudices/misjudgements about those people and concluded that the Voodoo is the magic. I'm sure there's some great analogy out there about drawing conclusions about how something works based on misunderstood observations, but I can't think of that analogy.

<comment><USER88> Correct, they're not a developer. They doubled down on it too: [https://twitter.com/skirani/status/1150019060467240960](https://twitter.com/skirani/status/1150019060467240960).

</comment></comment></comment><comment><USER89> Somewhat OOTL but why was 10x engineers trending on twitter? I've seen a few references to it but no idea why it started trending

<comment><USER90> This tweet[0]. People then made endless parodies of it.

[0] [https://twitter.com/skirani/status/1149302828420067328](https://twitter.com/skirani/status/1149302828420067328)

<comment><USER89> Jesus.

</comment></comment></comment><comment><USER91> One reason to deny the existence of 10x developers is that some organizations intentionally handicap their developers such that processes prevent anybody from being productive. This can occur when the organization, or the developers compromising it, do not trust the developers. It is hard to trust anything when a body of developers is functionally illiterate in their primary programming language or when the primary development goal is always achievement of easy.

<comment><USER92> > One reason to deny the existence of 10x developers is that some > organizations intentionally handicap their developers such that processes > prevent anybody from being productive.

So because some developers work for awful organizations, 10x developers cannot possibly exist anywhere? Not following that logic.

<comment><USER91> That is absolutely not what I said. You took part of what I said out of context to fulfill a maximum strawman.

<comment><USER12> That's how I read it to. I don't think he straw manned it??

</comment></comment></comment></comment><comment><USER93> > Why people claim 10X developers don't exist surprises me

Personally I've observed HN to have a pretty strong culture of tall poppy syndrome, which would explain that.

<comment><USER94> Really? On hacker news?

Certainly on Reddit, but HN seems significantly more meritocratic.

<comment><USER93> For everything that's wrong with Reddit, I don't think it's tall poppy problem is as bad as it is on HN.

Most stories about exceptional achievement on HN will be met with all sort of anecdotal derision. "10x developers don't exist", "10x developers are sociopaths", "10x developers are 'brilliant jerks'"... other common themes are that people must have exploited others to achieve their success, or that their success is unreasonable because they made too many personal sacrifices to achieve it, or that their success is unfair, because not everybody has the same social skills as them, or whatever other 'privilege' people want to nitpick in order to deride them.

<comment><USER95> This is probably a function of the target audience. If most people on HN work at SV-ish companies where all developers are at a very high level, then the biggest observable difference between them is how obnoxious they are.

</comment></comment></comment></comment><comment><USER96> 10X engineers deliver tech debt 10X faster than your average engineer.

If there are some out there who truly are of unparalleled execution ability, focus on testing, testability, reusability, modularization, componentization and simultaneously lifting up the team, I've never met one in my years in the industry. I don't think even if you spent 100 hours a week, you could achieve all of these things at once, so you have to compromise somewhere, and I know where people usually compromise. And that's before factoring in work-life balance haha.

In the early days of your company sometimes you do need people to just bang shit out, and eventually, you'll pay 10 1X engineers to clean it up. That period in a company's lifecycle is limited and the 10X'ers are outgrown very fast. Often they don't understand why but to their peers it's painfully obvious.

<comment><USER87> you describe net negative engineers not 10x engineers.

<comment><USER96> I'm describing the engineers I've encountered who have been dubbed by others in the business as "10X".</comment></comment></comment>


Gopli – Database backup between remote or local hosts, written in Golang - <AUTHOR> https://github.com/timakin/gopli <comment><USER2> Does this sync / delete tables in the correct order so that there are no foreign key constraint issues? Also, does it diff rows to tell what rows have actually changed or does it attempt to update every row every time? Does it handle cases where primary keys are not configured?

Percona's pt-table-sync has quite an extensive man page and explains how it accomplishes these. It may be a good reference to update your README.md to explain how the syncing works in gopli: [https://www.percona.com/doc/percona- toolkit/2.2/pt-table-syn...](https://www.percona.com/doc/percona- toolkit/2.2/pt-table-sync.html)

<comment><AUTHOR> This reference looks nice. Thank you for your comment. Gopli's README is so plain and just a kind of overview. So I should write down the process of syncing and notice about key-constraint and so on.

</comment></comment><comment><USER3> This looks nice, but what is wrong with: mysqldump -h 127.0.0.1 --database production | mysql -h yyy.yyy.yyy.yyy staging -f ?

<comment><USER3> Is this for replication or maybe updating developer environments with fresh data? If the latter, maybe some more options like "WHERE date <= curdate() - interval 30 day"

</comment></comment><comment><USER4> MySQL only. Would be nice to know without delving into the code. ;)

<comment><USER5> Exactly, I just read through it myself to realize it's a wrapper for the 'mysql' command. `management_system` needs to be documented.

<comment><AUTHOR> Hi, Thank you for checking my code and README. I'll take an effort to let Gopli work on other data storages. So please wait for that

<comment><USER6> In the meantime it might be wise to simply update the readme.

</comment></comment></comment></comment><comment><USER7> This is actually pretty interesting -- Makes me think one could actually move the problem of sharding/replicating a database completely out of the database layer itself, and instead just run the database through the add on?

Has anyone done this or know about any tools that take this approach? I mean like, instead of worrying about distributing, sharding, or replicating postgres/mysql/any-other-sql-compliant-db , why not just write mysql-dependent but a implementation agnostic layer that sits on top and does the coordination/sync bit? Sprinkle a little etcd and call it a day?

<comment><USER2> I've seen analytics services do this - periodic sync of multiple data sources into their analysis service, which is usually BigQuery or some OLAP system.

I don't think it would be a good idea for replication though. General syncing is super intensive, you have to diff everything between the two databases. Major DBMSs all have better replication processes that avoid this, in MySQL's case the slaves are synchronized using the master's binary log so they replay transactions that happen on master. It's far faster and allows for near real- time synchronization.

<comment><USER7> What I was suggesting was moving the binary log replaying OUT of the database layer, which is where I imagined the efficiency losses might arise.

I'm still fleshing this out in my head, but the idea is like: maintain a log of SQL commands that were to be done on the database at a layer right _before_ the database layer. then, as writes come in, they go to the layer-before, get replicated/distributed, and get copied to every node, and you could get easy (?) multi-master support. At the very least, you could do master-slave replication at the same cost that databases are doing it now, minus any inefficiencies from not using the specialized binary log each DB uses today. I guess the pre-database layer would do more gossiping than anything (in a multi-master type setup).

The real benefit here is that you would then be able to distribute any database that just supported SQL instantly (and even some that don't, with a properly written adapter), by composing it with another external (rather than internal) management process. There might even be some efficiencies to be gained if the databases are sharded -- localized data might go to the wrong node, but live there long enough to get reads and stuff like that.

I guess maybe instead of saying I'm thinking of building a layer on top, I'd rather pull binary log replication out of the databases themselves, and find some close-to-SQL log of commands that all DBs can share.</comment></comment></comment>