Ask HN: What features would you like to see in the next generation of DBMSs? - <AUTHOR> <comment><USER2> Having used RethinkDB[0] for a few things over the last year or so, I'm convinced that it represents "what's next" for DBMSs.

* The community is great.

* Documentation is amazing, and up to date.

* Examples actually work.

* Installation is simple. Runs on multiple platforms.

* Clustering is easy. Sharding is easy. Management is easy.

* Built for today's needs, not for what we were doing 30 years ago.

If you haven't yet taken a look at RethinkDB, do yourself a favor and spend a couple hours dinking around with it. You may just be impressed.

* [0] [http://rethinkdb.com/](http://rethinkdb.com/)

</comment><comment><USER3> How about one that supports a fast serialisation format such as Flatbuffers [1]? In the communications protocol. Json is so 2015 ;).

[1], [https://google.github.io/flatbuffers/](https://google.github.io/flatbuffers/)

</comment><comment><USER4> High flexibility. I want to have just indexes to do the queries I need to do with insane speed, effective compression of all data. More of a "database construction set" than an actual database.

Oh yeah, and something that is a cross between SPARQL and SQL 1999.

<comment><USER5> >I want to have just indexes to do the queries I need to do with insane speed

couchdb seems to fit this requirement really well. You explicitly create indexes on the fields that you want to query. The index creation part is slow but once it's done your queries are really fast. All data is compressed with snappy[1].

couchdb also encourages you to split your data across multiple databases. Effectively you can have thousands of databases all managed by a single couchdb server instance. You can move your data in temporary databases and "purge" them when they are no longer required. It's all really cool once you get a hang of how to use this feature. Although if you query across databases you'll have to "join" the result set within your application.

You should give it a try, you'll really like it :)

[1][https://github.com/google/snappy](https://github.com/google/snappy)

<comment><USER6> So is the index cached then?

<comment><USER5> Indexes are cached on disk. As the new data comes in that satisfies the query function,that you have written,it will automatically be appended to the index that has already been built up to that point.

The query results are retuned with Etags that you can cache like you do any http resource.

[http://stackoverflow.com/questions/4952429/couchdb- supports-...](http://stackoverflow.com/questions/4952429/couchdb-supports- caching)

</comment></comment></comment></comment><comment><USER7> Some tool for data upgrading/rolling back when doing updates. We got so many nice tools for programs, its about time to have a DB that could do as such... and versioning?

<comment><USER5> The documents that are stored in couchdb are versioned _as long as_ you don't replicate/ don't compress the database. You can query for all the _revs of a document. You can make api calls to give you an older _rev of the document.

The _revs themselves follow a pattern that is semi-human readable.Like so:-

`1-some_uuid,2-some_uuid,3-some_uuid` and so on.

The versioning functionality is not comprehensive but you certainly have the building blocks.

</comment></comment><comment><USER8> Automatic restful APIs endpoints, ideally with watch (long poll and WS) paths too.

<comment><USER5> Have you looked at couchdb? It has an http api and you can "watch" the modifications made to the database using a "_changes" feed that has options for longpoll, eventsource and a "continuous poll".

Web socket is not supported out of the box yet but you can always put couchdb in front of a proxy and send the changes feed over the socket connection once it's established.</comment></comment>


Any good hackathon during the Christmas holiday?  - <AUTHOR>

My recent concept died and I want to move on. I am kind of looking for some new ideas for me to work on and I am definitely open to suggestions as to how I could absorb some brand new ideas that are interesting to solve.
I live in Canada but I am willing to travel to the US after my finals and hopefully I want to attend some hackathon event during this christmas holiday (after 22nd) to meet some cool hackers...
Can anyone recommend me some events that I can attend? <comment><AUTHOR> to be clear I am on the east coast (Toronto).

</comment><comment><USER2> I don't know of any organized events going on, but if you want to come to North Carolina, gimme a shout. I'm off work from about 7 hours ago, until January 4th, 2011. I intend to spend basically the entire time (barring a trip to the coast to visit my family for about 2 days) hacking away on my projects and consuming potentially dangerous quantities of caffeine. :-)

<comment><AUTHOR> Thanks for the offer. I am afraid NC is a bit far for me. I was thinking somewhere around NY to be the furthest.

Anyway we should discuss some great ideas together sometime. Do you do IRC?

<comment><USER2> Yeah, I hang out on #startups on Freenode with the other gomers from here. :-) I'm usually on as 'mindcrime' there.</comment></comment></comment>


Introducing Conversations: A Whole New Way to Work - <AUTHOR> http://blog.pixelapse.com/post/82107545402/introducing-conversations

<comment><USER2> is there a way to use this without putting my designs on your servers?

<comment><AUTHOR> Thanks for the question! Unfortunately, we only offer a hosted service at the moment.</comment></comment>


Hindenburg Omen (Occurred Twice this Month) - <AUTHOR> http://en.wikipedia.org/wiki/Hindenburg_Omen

<comment><USER2> The consensus here was that this was drivel the first time it appeared (2 or 3 days ago).

<comment><USER3> I would tend to agree, although the Hindenburg omen might easily be explained by publicity like this submission on HN. Investors read about the Hindenburg omen and the fact that it has occurred twice this month. They assume that a crash is imminent, so they start selling stocks, which causes the market to collapse.

In this way what is drivel could be made reality because too many people don't believe it to be drivel. There are some really interesting analysis of stock market dynamics such as this in the book "The Wisdom of Crowds" by James Surowiecki

</comment></comment><comment><USER4> Not that I'm a finance expert, but I'm uncomfortable with an indicator the parameters of which were optimized by backtesting half a century ago.

On the other hand, a confirmed Hindenburg Omen signal occurred on Jun 16th 2008, which would be a timely warning for the following events. On yet another hand, no word on how many such _confirmed_ signals were false alarms historically.

<comment><USER5> Doesn't it say in the article that in the past, 24% of the Hindenburg Omen were followed by a stock market crash? From the article:

 _Looking back at historical data, the probability of a move greater than 5% to the downside after a confirmed Hindenburg Omen was 77%, and usually takes place within the next forty-days. The probability of a panic sellout was 41% and the probability of a major stock market crash was 24%._

While the past performance is quite compelling, the criteria seem to be extremely arbitrary.

<comment><USER4> The "Conclusions" section of this wiki page is rather uninformative... See [http://www.safehaven.com/article/3880/the-past- performance-o...](http://www.safehaven.com/article/3880/the-past-performance- of-the-hindenburg-omen-stock-market-crash-signals-1985-2005) (from external links)

There's a table with all 22 confirmed signal clusters 1985-2005 and the following "crashes" along with their delays. Although the phrase "25% of these signals were followed by a market crash" is pretty impressive, when you look at the actual definitions of crash (>15% drop), and the delays (up to 4 months) it doesn't look as exciting anymore.

</comment></comment></comment><comment><USER6> Sometimes humans too good at seeing "patterns" for our own good.

<comment><USER7> There's a word for that:

<http://en.wikipedia.org/wiki/Paredolia>

</comment></comment><comment><USER8> I've never seen anything that makes me think 'Technical Analysis' of stock market indicators is anything more than a pseudoscience. It's fun and spooky, but is it any better than throwing darts?

<comment><USER9> No, it is generally complete BS. It survives on human gullibility, and the observation by stock brokers that promoting technical analysis is good for trading volume. A good book that addresses this _A Random Walk Down Wall St_.

That said the particular "omen" they are talking about does have some merit. Normally there is a broad correlation between different stocks, so you don't typically get both lots of highs and lows at the same time. (This correlation is why the DOW, which just tracks a small number of indicator stocks, is correlated with much broader sets of stocks.) But you can get lots of highs and lows at the same time if the market is very volatile. Volatile markets sometimes go down very sharply. Of course causation goes the other way. Fears that the market could drop cause volatility, and sometimes those fears are proven right.

That said we can and do measure market volatility directly. (For instance every options pricing model does that.) And that gives a much more fine-tuned predictor of risk than this "omen".

</comment></comment><comment><USER10> Funny that probably a lot of people make a living thinking in such a way, forming complicated theories of superstition.

<comment><USER11> It sounds like the beginning to one of those 80ies barbarian movies (I'm thinking of The Beastmaster, which is a must-see):

"On the 10th day, the Omen foretold the coming Prophecy that the Evil Emperor would be slain by a Child born under the Star of the Omen"

<comment><USER10> Would be fun to see a SciFi movie based on this idea :-)

</comment></comment></comment><comment><USER12> That's a hilariously small sample size for a 'model'.

</comment><comment><USER13> This is right up there with the similarities between Kennedy and Lincoln.

</comment><comment><USER14> Let's assume the big crash will happen before the end of this year: what do you do? As an angel? As an entrepreneur trying to raise money? As a funded startup with a limited runway?

</comment><comment><USER15> OMG, let's go looking for Ms and Ws in stock price data</comment>


Djbsort: A new software library for sorting arrays of integers - <AUTHOR> https://sorting.cr.yp.to/ <comment><USER2> The problem with AVX2 accelerated code (and much of AVX) is that unless you have a lot of it to run you end up with a substantial speed hit that comes from the cost of switching to a different power bin (which often takes 1 or 2ms!) and then running at a lower clock speed.

This often still ends up being an improvement over scalar code (at the cost of higher power usage), but for occasional workloads that don't need to do multiple milliseconds of AVX instructions you tend to have better results from 4-wide vectors, which don't have this cost.

<comment><USER3> You also have penalties from context switches, although you can reduce their impact by performing them in a lazy fashion. AVX512 is even worse, of course.

<comment><USER4> You need a context switch to use AVX?

<comment><USER3> No, unless you have a special setup where one process on the whole machine can use AVX. That might make sense for special, controlled environments.

What I meant is that there are more registers to shuffle around at context switch time, when multiple processes use the extensions.

<comment><USER4> Ah got it, thanks.

</comment></comment></comment></comment></comment><comment><USER5> The main feature of the algorithm is to sort fast in constant time (for fixed n) for cryptographic purposes [0].

[0] [https://ntruprime.cr.yp.to/ntruprime-20170816.pdf](https://ntruprime.cr.yp.to/ntruprime-20170816.pdf) P. 48

</comment><comment><USER6> The library doesn't have a license on the site or in the tarball. From djb's previous writing and software, he probably intends it to be license-free software, which is an uncommon situation worth investigating before use: [https://en.wikipedia.org/wiki/License- free_software](https://en.wikipedia.org/wiki/License-free_software)

(Trying to describe this neutrally because I've seen enough bickering about it over the last ~20 years and don't have strong feelings about it.)

<comment><USER7> Three of the source code files contain a notice that they are in the public domain:

* cpucycles/mips/cpucycles.c

* cpucycles/cortex_vct/cpucycles.c

* cpucycles/cortex/cpucycles.c

Regarding the missing license:

I guess if you download the software from his website you are not allowed to distribute it yourself. Is that correct?

<comment><USER8> > Three of the source code files contain a notice that they are in the public > domain:

Making it legally dodgy to dangerous in mainland europe, either way certainly not reliably licensed.

<comment><USER9> I hear people raise this concern a lot, but I think it is without foundation. If something is in the public domain in the U.S. then anyone can use it for any purpose, including releasing it under whatever license they want. Of course, any constraints imposed by that license will be unenforceable since any user of the software can claim to be using it under the terms of some other license or, of course, as part of the public domain. But if you think you need it licensed, you can have it licensed.

<comment><USER10> Anybody under U.S. jurisdiction can follow U.S. law.

However if me and my company and everything is in Europe and I use/redistribute the code in Europe I must follow applicable European law. If that doesn't accept that form of public domain the author (rights owner) could sue me and it were upon the judge, who sensible they are. (This is mostly theoretical - if the author decides to put it in public domain per U.S. law they most likely don't want to restrict to U.S.)

For an example see the recent case about project Gutenberg [https://news.ycombinator.com/item?id=16511038](https://news.ycombinator.com/item?id=16511038)

<comment><USER9> That was a completely different situation. In that case, the material was still under copyright in Germany. In this case, the material has been placed in the PD _by the original author_ and so no one in the world can possibly have any legal claim on it.

But if this really concerns you, I would be happy to provide you -- or anyone else -- with a licensed copy of any of DJB's code for a modest processing fee.

<comment><USER8> > In that case, the material was still under copyright in Germany.

Which is exactly the case of djb's work here.

> In this case, the material has been placed in the PD by the original author > and so no one in the world can possibly have any legal claim on it.

Wrong. djb _and any possible heir of his_ does, because you can't place things in the public domain in mainland europe.

> But if this really concerns you, I would be happy to provide you -- or > anyone else -- with a licensed copy of any of DJB's code for a modest > processing fee.

Unless djb specifically gave you license to do so, your "licensed copy" is worth exactly as much as the original public domain dedication is. As far as european law is concerned, you have no rights to the work, and thus certainly don't have the rights to relicense it.

<comment><USER10> > Wrong. djb and any possible heir of his does, because you can't place things > in the public domain in mainland europe.

However a court could interpret this as a royalty-free license, at least until the moment they start sueing.

<comment><USER8> That is true, but there currently is no precedent for _that_. Until someone becomes the "sacrificial lamb" by 1. taking the risk and 2. being sued for it, anyone intending to build a business is understandably wary and unlikely to touch public-domain-dedicated assets.

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment> Ono-Sendai The median sort time for sorting 1048576 elements wth djbsort is 61467822 cycles: [https://sorting.cr.yp.to/speed.html](https://sorting.cr.yp.to/speed.html)

On a say 3.6 Ghz processor that would be around 17ms. So the number of elements sorted per second would be around 61.4 M elements/s.

My parallel radix sort can sort floats (a little harder than integers) at around 165 M elements/s: [http://forwardscattering.org/post/34](http://forwardscattering.org/post/34)

A serial radix sort should still be similar or faster to djbsort.

<comment><USER11> My single-threaded floating point sort is also twice as fast as djbsort:

[http://stereopsis.com/radix.html](http://stereopsis.com/radix.html)

This one uses 11-bit radix to save memory bandwidth. Without all the floating- point stuff it would be faster.

<comment><USER12> The point of djbsort is to be constant time while not being super slow.

Of course a fast directly implemented radix sort will be faster, especially if you sprinkle SIMD on top.

</comment></comment></comment><comment><USER13> Why might the installation instructions require the creation of a new user specific to the sorting program? Purely for security of the normal user given that the installation is using a wget / shell script process? [https://sorting.cr.yp.to/install.html](https://sorting.cr.yp.to/install.html)

<comment><USER14> This is bizarre. Why not just give users link to the source?

[https://sorting.cr.yp.to/djbsort-20180710.tar.gz](https://sorting.cr.yp.to/djbsort-20180710.tar.gz)

And it's certainly not about good security practices:

* The page teaches users to paste stuff copied from web into a terminal, whereas many terminals are still vulnerable to this: [https://thejh.net/misc/website-terminal-copy-paste](https://thejh.net/misc/website-terminal-copy-paste)

* The page teaches users to use su to lower privileges, whereas many (most?) su implementations are vulnerable to tty hijacking.

<comment><USER15> "The page teaches users to use su to lower privileges ..."

In the example, he could have used his own utilities for dropping privileges (setuidgid, envuidgid from daemontools).

If I am not mistaken, busybox includes their own copies of setuidgid and envuidgid, meaning it is found in myriad Linux distributions. I believe OpenBSD has their own program for dropping privileges. Maybe there are others on other OS.

Instead he picked a ubiquitous choice for the example, su.

It is interesting to see someone express disdain for the version.txt idea. I had the opposite reaction. To me, it is beautiful in its simplicity.

As a user I like the idea of accessing a tiny text file, version.txt, similar to robots.txt, etc., that contains _only a version number_ and letting the user insert the number into an otherwise stable URL.

This is currently how it works for libpqcrypto.

[https://libpqcrypto.org/install.html](https://libpqcrypto.org/install.html)

I would actually be pleased to see this become a "standard" way of keeping audiences up to date on what software versions exist.

By simplifying "updates" in this way, any user can visit the version.txt page or write scripts that retrieve version.txt to check for updates, in the same way any user can visit/retrieve robots.txt to check for crawl delay times, etc.

It is not necessary to "copy and paste" from web pages. Save the "installation" page containing the stable URL as _text_ , open it in an editor, insert the desired version number into the stable URL.

Save the file. Repeat when version number changes, appending to the file.

I like to keep a small text file containing URLs to all versions so I can easily retrieve them again at any time.

</comment></comment></comment><comment><USER16> (Kind of a tangent, but if you're into sorting check out:

"Generic top-down discrimination for sorting and partitioning in linear time"

[https://www.cambridge.org/core/journals/journal-of- functiona...](https://www.cambridge.org/core/journals/journal-of-functional- programming/article/generic-topdown-discrimination-for-sorting-and- partitioning-in-linear-time/B85E48EFC0B4D2BDDDE9A3885094FDD7)

Abstract: "We introduce the notion of discrimination as a generalization of both sorting and partitioning, and show that discriminators (discrimination functions) can be defined generically, by structural recursion on representations of ordering and equivalence relations. Discriminators improve the asymptotic performance of generic comparison-based sorting and partitioning, and can be implemented not to expose more information than the underlying ordering, respectively equivalence relation. For a large class of order and equivalence representations, including all standard orders for regular recursive first-order types, the discriminators execute in the worst- case linear time. The generic discriminators can be coded compactly using list comprehensions, with order and equivalence representations specified using Generalized Algebraic Data Types. We give some examples of the uses of discriminators, including the most-significant digit lexicographic sorting, type isomorphism with an associative-commutative operator, and database joins. Source code of discriminators and their applications in Haskell is included. We argue that built-in primitive types, notably pointers (references), should come with efficient discriminators, not just equality tests, since they facilitate the construction of discriminators for abstract types that are both highly efficient and representation-independent.")

<comment><USER17> I've tried so many times to frontpage that, but I've failed. Maybe we should have another go?

Nothing in djbsort's approach is inapplicable to another sorting algorithm, so maybe we can hope for better primitive support for discrimination sort implementations (or at least american flag sort implementations). I seem to recall reading that discrimination sorts are inherently content-independent.

<comment><USER16> It was probably you I heard about it from! Submission upvoted. ;-)

<comment><USER17> Something something definition of insanity. ;)

</comment></comment></comment></comment><comment><USER18> > Other modern Linux/BSD/UNIX systems should work with minor adjustments to > the instructions.

I can report that I got it to build and run on slightly out of date FreeBSD by deleting all of the -m32 variants, and deleting all of the -march=haswell variants. I haven't looked into whether this is down to the version of GCC that comes in ports and the version of Clang that comes in base, or something else. No _other_ changes were needed to the build process, though.

        JdeBP /package/prog/djbsort % /tmp/djbsort/command/int32-speed
        int32 implementation int32/portable4
        int32 version -
        int32 compiler clang -fPIC -Wall -O2 -fomit-frame-pointer -fwrapv
        int32 1 72 72 72
        ...
        int32 1048576 1979077401 1979993070 1983745962

</comment><comment><USER19> Create a user and env to run a one-off build + application. DJB cracks me up. He may have the right thing in mind but this type of prophylactic approach is no longer proof against anything.

</comment><comment><USER20> The 2.5 cycles/byte compared to the 32 cycles/byte that Intel managed pulled off seems like an improbably large improvement over the current state of the art? Is this real?

<comment><USER21> The cycles/byte for all the sizes listed in the table on the Speed page[0], courtesy of copy-paste and a one-liner in the Python REPL:

      size      cycles/byte (based on median)
      1         6.0
      2         3.375
      4         11.625
      8         9.625
      16        7.984375
      32        7.515625
      64        6.0390625
      128       4.31640625
      256       2.4873046875
      512       2.29443359375
      1024      2.5048828125
      2048      2.77893066406
      4096      3.0791015625
      8192      3.89566040039
      16384     5.23690795898
      32768     6.35472106934
      65536     7.55914306641
      131072    9.23189163208
      262144    10.789557457
      524288    12.4885950089
      1048576   14.6550707817

[0] [https://sorting.cr.yp.to/speed.html](https://sorting.cr.yp.to/speed.html)

</comment></comment><comment><USER22> Direct download link (if you don't want to run the script)

[https://sorting.cr.yp.to/djbsort-20180710.tar.gz](https://sorting.cr.yp.to/djbsort-20180710.tar.gz)

</comment><comment><USER23> I liked this bit, using the fastest compiler for each primitive:

> ./do tries a list of compilers in compilers/c, keeping the fastest working > implementation of each primitive. Before running ./do you can edit > compilers/c to adjust compiler options or to try additional compilers.

<comment><USER24> It is sadly necessary; 30%+ performance regressions from, say, gcc 4 to gcc 6 are not uncommon w/ vector intrinsics.

</comment></comment><comment><USER25> beware, Linux only. Needs the usual BSD/macOS patches for HW_CPUSPEED and CLOCK_MONOTONIC and do away with for linux/perf_event.h.

Unfortunately I have no idea how he deals with patches, I don't think he does.

</comment><comment><USER26> My job involves a lot of packaging/cross-compilation, and djb's libraries always seem consistently hostile to the lowly packaging engineer.

Would it really be all that much work to package in autotools or CMake? Why do I need his special-snowflake build system with its hard-coded assumptions about system paths?

I know that the cult of djb will downvote this into oblivion, but seriously, what is the rationale for a build flow that involves:

      1. Downloading a text file
      2. Parsing it to get a URL
      3. Making a new user
      4. Symlinking the user's HOME directory into the build tree
      5. Run an extremely non-standard build system.
      6. Hope you're not trying to cross-compile, because good luck with that.
      7. Guess at where the files came out (hint: it probably won't be in FHS locations)
      8. Copy the output yourself once you find it.

Would it really be that much harder to give us a git repo and a ./configure or a CMakeLists.txt?

<comment><USER27> I've been using DJB's stuff for ~20 years and I don't like his recent build systems either. Without defending it, I'd just like to offer a theory on why he packages things the way he does.

Back in the day, his build processes were atypical but still much more "normal" than now. He also released his software without licenses. During this time of heavy software development, DJB was concerned about people screwing around with the internals of his software, hurting security or reliability, and then blaming the software rather than the modifications. So his build systems were, I think, designed to lead to the result he wanted, where software behaved and was administered in the same way on various platforms.

In the mid-2000s he re-licensed existing software as public domain and began publishing all _new_ code as public domain as well. Around this time, build systems began to get more wonky. Also, his public work that garnered the most attention shifted away from software toward cryptography. He did some attacks on existing crypto and authored Curve25519, Salsa20, etc.

He's also been putting out a _tremendous_ volume of work in multiple categories. I bet he'd rather work on this stuff than on user-friendly build systems.

So given these points, I think the explanation for his unfriendly build systems is

      A) a very strong aversion to people modifying his stuff where he gets blamed if modifications do harm;
      B) a shift away from software development, where people generally care more about build systems anyway;
      C) a huge level of productivity which results in very atypical Pareto principle choices/tradeoffs;
      D) his public-domain licensing.

Given these 4 points, I think DJB is unwilling to take time away from crypto and other work and put it into build systems he doesn't enjoy that will take more time upfront and more babysitting down the road. Fewer people will package it, but the software is public domain and competent people can just add their own build system. This squares with his available time and interests.

So, I don't like his build systems either, but I think I understand where they come from.

<comment><USER26> Appreciate the response! Interesting to read.

The Libsodium guys wound up doing exactly what you're suggesting, because of the impossibility of trying to package NaCl as-is.

So they essentially had to re-do/duplicate all of his build work just to make it packageable. And now there are two competing implementations (three if you count tweetnacl). And a bit of a confusing mess in the documentation department.

It seems a little selfish for djb to take the "works on my machine" attitude, because it means that a bunch of other people have to reverse-engineer all that stuff just to make it portable.

But I guess OTOH it's his software, so it's his choice. And maybe he doesn't care whether people choose to use his stuff or not, as long as he's publishing.

<comment><USER28> conveniently, almost nobody uses NaCl as-is due to its more or less never having been patched, and nobody uses TweetNaCl, so there is de facto one implementation.

<comment><USER29> Does it need to be patched?

</comment></comment></comment></comment></comment><comment><USER30> I haven't read the algorithm here yet nor all of the comments, so this may have already been covered, but in many cases when it comes to sorting integers and such you don't really need to _sort_ them at all - you just need to _count_ them.

<comment> ur-whale Yeah, except that when the integers are very large, that tends to fail. Also, as pointed out by many others, the point of this work is to sort in constant time to avoid side-channel attacks. I doubt the histogram sort (which I think you're referring to) has this property.

</comment></comment><comment><USER31> That's pretty cool, are there any bindings for e.g. Go out there?

</comment><comment><USER32> Am I the only one who thought the name had something to do with Djibouti?

<comment><USER33> Probably one of the few. On HN, a search for "djb" results in about 60 submissions where "djb" is in the submission title. There are about 6 for Djibouti.

Also, the airport code "DJB" is for Sultan Thaha Airport. The Djibouti–Ambouli International Airport code is JIB.

</comment></comment><comment><USER34> It's nice when things fit in RAM. But very often they don't. When you want to sort arbitrary-sized binary records on disk look no further than bsort: [https://github.com/pelotoncycle/bsort](https://github.com/pelotoncycle/bsort)

<comment><USER35> When sorting on disk, there are actually limitations in sorting linear time. Has to do with being able to read blocks in memory and write them back to disk. As everything is in blocks, you are limited to O(n/B log(n/B)) with B being the amount of items per block. For more speed, a merge sort that keeps in mind the block size works quite well. Search for external sorting.</comment></comment>


European Parliament publishes a corrigendum to the GDPR - <AUTHOR> https://www.technologylawdispatch.com/2018/06/privacy-data-protection/european-parliament-publishes-a-corrigendum-to-the-gdpr/ <comment><USER2> Hold up, there are _26 "official" language versions of the GDPR_? How does that work in practice? It would seem that no matter how well researched there is a possible meaning mismatch when you cross languages.

<comment><USER3> > EU law currently applies to 27 countries and is available in 23 languages > which all carry equal status. In practice, this is achieved though > translation and by the work of the DGT (Directorate General Translation), > which hosts the largest translation service in the world. But from a legal > point of view, translation is institutionally ‗non-existent' and EU > languages are all equal and authentic. The issue has been given attention in > the last two decades mostly from scholars, linguists and translators > (Correia 2003, Kjær 1999, Koskinen 2000, Ńarčević 2001, Tosi 2001, Wagner > 2000), thus raising awareness on the paradox of translation and the lack of > a proper EU language policy and legal culture

Source(pdf): [https://pressto.amu.edu.pl/index.php/cl/article/download/698...](https://pressto.amu.edu.pl/index.php/cl/article/download/6984/6983)

<comment><USER4> The specific part about interpreting laws says

> The Court of Justice arbitrates on Community law and pursues a particular, > but no less rigorous application of multilingualism. ... each authentic text > is considered independent for the purpose of interpretation by the courts > and despite being translations, judgements and other documents are deemed to > be authentic only if they are the language of the case. On the other hand, > as none of them prevails and they all have the same meaning, whenever > linguistic divergence or ambiguity arises, the European Court of Justice > consults all the texts of a given instrument on a routine basis. > Interpreting the intended meaning of the single instrument is given priority > over any linguistic discrepancies and this is why revisers and translators > of the European Court of Justice need to be lawyers.</comment></comment></comment>


Workout app with 100.000 people on waiting list launched - <AUTHOR>

Hello,
I'm CTO of madbarz.com - a workout app that has finally launched after reaching milestone of almost 100.000 people on waiting list.
Now in less than a day we have already 15.000 registrations!
Check it out at: http://madbarz.com/ <comment><AUTHOR> UPDATE: we are now on 25.000 :)</comment>


Ask HN: Help! My First Impressions Suck.  Any advice? - <AUTHOR>

I want to do more tech.  I want to be near people who love tech.  I live somewhere where tech isn't much.
My problem?  My first impressions suck.  I don't think I can change that.  Like some people have a hard time loosing weight, I have a hard time changing my first impression.
For the people who know me well, when they reminisce, they tell me how awful a first impression I gave them but due to location and how we were forced together, they completely changed their mind to enthusiastically recommend me.
From what they tell me, I sound like an idiot when I first talk to them.
Because of this problem, I'm having extreme difficulty passing the telephone interviews (yes, the easy part!) of companies in tech hubs.  It feels like I could hear the disappointment in their voice when they talk to me.
What can I do if my first impression suck?  How do you continue the interview after failing the phone interview?  How do you get a job someplace far away? <comment><USER2> Something I learned at my consulting firm.

If you are asked a question and the answer has three parts, A, B, and C, and you know A and B but you don't know C, lead with what you know, before asking about details of which you are unsure.

"So how many HVAC systems are installed in Houston every year?"

"Well, Houston has about 5 million people, which makes about 2 million households, so if half of household live in a house there are about 1 million homes in Houston....

Now, as to how many HVAC systems are installed every year, how long does the typical HVAC system last?"

"About 25 years"

"Okay then, that makes 1 million divided by 25, or 40,000 HVAC systems every year..."

</comment><comment><USER3> Why do you think you suck? Are you curt, derogatory, pessimistic or judgmental? Do you mumble instead of talking whole sentences?

In face-to-face interaction, a smile, even a superficial one is a good thing. In general, showing curiosity, interest in what the other person is doing puts you in their good books. Joke if you can, it lightens the weather.

Listen before you talk. Be kind, even to an asshole. Don't judge someone too harshly, you are not in their shoes. And people can see that through if you do that.

I don't know what else to say unless you tell us more about yourself.

</comment><comment><USER4> If you are truly unable to naturally connect in terms of good first impressions you might want to approach this like an optimization problem. I recommend Nolan's book on elicitation ([http://www.amazon.com/Confidential- Business-Secrets-Getting-...](http://www.amazon.com/Confidential-Business- Secrets-Getting- Keeping/dp/097213560X/ref=sr_1_7?s=books&ie=UTF8&qid=1391254677&sr=1-7&keywords=elicitation)) as a good start, only read segments 1&2 unless you want to start a competitive intelligence firm. That said, as someone who used to have some degree of confidence issues with new contacts, go apply to 50 jobs you don't give a shit about in the slightest, that you wouldn't accept whether or not you got the job. If you live in a city with lots of consulates/embassies, try BSing your way into a party.

When it comes down to it the ability to bullshit won't necessarily help you secure a job, but it will remove a lot of the barriers previous to where that decision is made and get you closer to that meeting/phonecall/interview where that decision might be made. Bottom line, get used to making first impressions. I bet your first website sucked (probably far less than my first site sucked but it still _probably_ sucked), but the way to get better at most things is to do it more.

</comment><comment><USER5> You're not giving us much to go on.

Do you say "like" a lot? That can do it. Try to avoid saying "like."</comment>


Building Beautiful Apps from Ugly Code - <AUTHOR> http://prog21.dadgum.com/108.html

<comment><USER2> This reminded me of this image from Code Complete: [http://www.civilnet.cn/book/program/Code%20Complete%202th%20...](http://www.civilnet.cn/book/program/Code%20Complete%202th%20Edition/images/0735619670/graphics/24fig02.gif)</comment>


Designing and building a distributed data store in Go [pdf] - <AUTHOR> https://fosdem.org/2018/schedule/event/datastore/attachments/slides/2618/export/events/attachments/datastore/slides/2618/designing_distributed_datastore_in_go_timbala.pdf <comment><USER2> Funny I built the same kind of distributed timeseries system for Digital Ocean, called Vulcan. Also did a few talks on it. It was based around prometheus, I've seen at least 4-5 other major ones that are in production at different places, like Heroku has their own internal one also. Go seems to be the leading language for these next gen databases.

[https://www.youtube.com/watch?v=likpVWB5Lvo](https://www.youtube.com/watch?v=likpVWB5Lvo)

[https://www.slideshare.net/MatthewCampbell7/distributed- time...](https://www.slideshare.net/MatthewCampbell7/distributed-timeseries- database-in-go-gophercon-india-17)

</comment><comment><USER3> video: [https://www.youtube.com/watch?v=Su2WqMHVoAA](https://www.youtube.com/watch?v=Su2WqMHVoAA)</comment>


Interactive Photo Mode for Programming - <AUTHOR> https://medium.com/@annakornakiewicz/interactive-photo-mode-for-programming-d3796940c96d <comment><USER2> How does this work? It's a visual programming language that uses photos to represent python code?

<comment><AUTHOR> This is not exactly visual programming language. I would say rather mixed language. Only some variables, statements and data structures have photo representation. Still there are instructions with letters and numbers.</comment></comment>


Ask HN: Can a software engineer help change the world? - <AUTHOR> I am sorry for the weirdness of the question but I am relatively young and want to make an <i>actual</i> impact on the world, something science/tech related one day. Will becoming a software engineer be a step in the right direction? <comment><USER2> > want to make an _actual_ impact on the world

So in your view, there's this mass of people who accomplish nothing useful and then a few who change the world?

> Will becoming a software engineer be a step in the right direction?

You need to realize that every single one of us who _works_ , regardless of profession, changes the world. Even the guy flipping burgers at McDonalds.

If the guy wasn't flipping burgers, you'd be hungry and you wouldn't be able to do your super duper important job. Every single person in the economy contributes and shares in the accomplishment of everyone else. It's a large machine and every piece is important. Every piece, every person, makes a contribution and has an _actual_ impact on the world.

Don't choose your profession based on some pre-conceived notion of how important a job is. That's just a cover for where on the social ladder a job falls. Yes, if you're a lawyer or a doctor, you'll be more respected than a burger flipper. Better paid too. But that's all it is, social perception and its consequences.

Choose a profession based on your ability and your capacity to engulf yourself in the work. If you have the capacity to be a software engineer, you enjoy the work, and you can make beautiful things with your skill-set ... by all means. But leave the pretentious bullshit of "making a difference" behind. We all make a difference, regardless of the profession.

</comment><comment><USER3> I hate to admit this as an engineer, but the people who make a difference are working at the _strategy_ level and not the _implementation_ level, which is where most of us engineers spend our time. No world-changing startup ever failed because "we couldn't get the code right" \-- they fail because they build things the world doesn't want, or they fail to market the things they build to people who want them.

The second difficult-to-admit thing is that software is only good at solving problems where analyzing information and manipulating data are the solution. Mobile banking apps are doing tremendous good in developing countries -- but the benefits of things like mosquito nets, water purification, and vaccination are not software problems. I'm pretty doubtful that apps will do much to alleviate poverty, although I'm excited and encouraging of the people who are trying.

<comment><USER4> While what you are saying isn't entirely without merit, isn't that like saying that the success of the Apollo missions lie with the bureaucrats rather than the engineers and astronauts?

<comment><USER3> I'm saying that man wouldn't have set foot on the moon without John F. Kennedy declaring it would be so, but man would probably still have walked on the moon if the Apollo engineering team were slightly less skilled. (Probably would have taken longer and had more deadly failures, so thank God for the highly- skilled people who worked together to make it all possible, but engineering skill adds incremental improvement because efficacy becomes more dispersed the further down the pyramid you go -- that's just the nature of human organizations, the implementers are fungible.)

<comment><USER5> You don't think a bunch of astronaut deaths and costly mission failures would've eventually eroded public will?

<comment><USER3> A bunch of astronauts _did_ die at the very beginning: Apollo 1 blew up on the launch pad. There was outrage in Congress but it died down because LBJ (a huge NASA supporter) used his (in)famous strong-arm political leadership to push forward. Again: strategy and leadership were more important than implementation. LBJ was more important for pushing Apollo forward than NASA's engineering success (or lack thereof).

[http://en.wikipedia.org/wiki/Apollo_1#Political_fallout](http://en.wikipedia.org/wiki/Apollo_1#Political_fallout)

</comment></comment></comment></comment></comment><comment><USER6> 1\. Don't try to boil the ocean. You can't change the whole world at once, you can only affect your sphere of influence. You first need to expand your sphere of influence.

2\. The easiest way to do that is money. (ex: Bill Gates, Elon Musk) Barring that, you need to write an awesome book, or have something published (Stephen Hawking, Albert Einstein), or get elected (Barack Obama). Chances are, those will lead to increased wealth also.

3\. Use that to introduce the change you want in the world.

The thing is, the most difficult part is expanding that sphere. You're one person of 7 billion. That's not even a one in a million chance of impacting the world. The odds are against you. That said, you can make a change in your community, maybe even your city. The key is, you have to start right where you are and impact what you can.

If Albert Einstein had asked 'should I become a patent clerk if I want to make an actual impact on the world?' people would have flat out said NO. The thing is, it did give him steady income and time to think, and thinking is what he did best.

Now of course you can shortcut those steps by being a martyr for whatever cause you want (Nathan Hale, Mahatma Gandhi), but then if you have to ask, you're already doing it wrong.

</comment><comment><USER7> Whatever your career, if you earn money then effective altruism is an evidence-based approach to improving the world:

[http://en.wikipedia.org/wiki/Effective_altruism](http://en.wikipedia.org/wiki/Effective_altruism) [https://centreforeffectivealtruism.org/](https://centreforeffectivealtruism.org/)

I suspect this isn't what you were thinking of - but it's something you can do that doesn't prevent you improving the world directly through the software you make.

<comment><USER8> Yes. Effective altruism is the right answer.

This EA organization seems especially relevant to OP's question: [http://en.wikipedia.org/wiki/80,000_Hours](http://en.wikipedia.org/wiki/80,000_Hours)

</comment></comment><comment><USER9> I'm going to say no. It isn't because software engineering is unimportant. It is because you don't know what impact you wish to make. You asked the wrong question. You ask "Can a software engineer help change the world?" when the question is really "How useful is software engineering to a person who will change the world?" Perhaps you can see the difference.

The people who stand upon the ends of our levers are there because they have already decided the direction in which they wish to push.

You cannot guarantee that any skill will help you until _after_ you have decided what you want the world to do for you. Once you have decided upon your end, the means to it should become more clear.

</comment><comment><USER10> My take on this: it's already hard enough not to screw the world any more that it already is. My favorite example: adding "[this project] shall be used for Good, not Evil" to your code will bring infinite arguing back and forth[1].

I try to focus on the small things. There are alternatives to owning the Bill & Melinda Gates Foundation. Choosing not to work for Zynga, or refusing to help the NSA's data collection is more than enough for a start. Write an app that makes life in the 3rd world easier, and you are already ahead of the curve.

[1] [https://news.ycombinator.com/item?id=5138866](https://news.ycombinator.com/item?id=5138866)

<comment><USER11> >I try to focus on the small things

Great attitude. I feel the same way too. There is a line[1] in "Hey Jude" (a popular Beatles song) that I quite like:-

"Don't carry the world upon your shoulders"

No one can change the world all by himself but little things go a long way.

</comment></comment><comment><USER12> I think as an engineer, you'll learn to 'learn how to build'. Then you can learn to build software, or hardware, or anything.

Then as a software engineer, you'll earn a decent amount of money. And money is what you will need to sponsor your dreams of changing the world. And getting started in software isn't too hard, you just need a computer and internet. Worst case, you can always fail and start again (minus health). So out of any career, I think a Software Engineer one is a pretty good bet. You will at least get a pretty good set of tools to help you out. Then it's up to you and your determination.

All that said, I don't think there's a single 'career' that will make you change the world. Nothing and nobody will make you change the world, it will come from you.

</comment><comment><USER13> I think that depends on what you mean by " _actual_ impact on the world."

Some people might advise you that you can't do much, but that's not true.

Some people might advise you to try to do something uber-ambitious, but most uber-ambitious projects fail.

Some people point out that almost whatever you do, your labor contributes to society as a whole. That's _very_ true.

I want to add: change the world _into what?_ In what _direction?_ Your question is like asking, "Can I travel far away by train or automobile?" Depends where! Are there roads or tracks? Do you intend to construct roads or tracks? Why are you picking out some destination in particular?

</comment><comment><USER14> I think it's inevitable that you will need software if you're going to use science/tech.

Software did change the world and is continuously doing that. The jump from even 30 years ago is spectacular.

This trend will continue in the foreseeable future.

But notice that there's a slight difference between 'changing' the world and 'making the world better'.

The first means doing thing differently, which doesn't necessarily mean that it is in everyone's best interest. Change can also be a bad thing. The second means giving humanity tools which they can use themselves to solve some of their existential problems.

When you solve one of these problems you can either: profit immensely from it by being a monopoly (change the world) or share your solution with the world and contribute to everyone (make the world better).

Then there's you very personal desire to 'make an actual impact'. You have to very carefully examine _why_ you want it to be _you_ and not someone else. If you're looking to get rich, you can do it in a lot of other ways. If you want a place in history, you can also do it by going into politics or science or art. If you want to improve the condition of the human race, then ask yourself - are you willing to still do it even if you get no reward or recognition for it ?

I guess that's more than you asked for, but I think it's important to consider these things early as they will haunt you for the rest of your journey. Good luck.

</comment><comment><USER15> "Make an impact", "change the world".

You can certainly do that working at NSA (or FSB).

If you want to do _good_ , then becoming a software developer (or accountant, or statistician) is not enough in itself.

It is what you then choose to do with your knowledge that determines that. And luck.

However, beware [1].

[1] [http://en.wikipedia.org/wiki/The_road_to_hell_is_paved_with_...](http://en.wikipedia.org/wiki/The_road_to_hell_is_paved_with_good_intentions)

</comment><comment><USER16> Sometimes I think we software engineers are really arrogant. Can a plumber change the world? Can a teacher? Can an architect? Can a civil engineer change the world? Yes they all can, but you'll only hear a software engineer ask a question like that.

Anytime I see an article saying something along the lines of "Big data can cure cancer," I just laugh. All occupations help the world in their own way. Software just happens to be really trendy right now.

<comment><USER17> This is a legitimate question because, as opposed to other professions such as the ones you mentioned, software engineers very often do not change the world at all. A lot of software projects fail, have a very short lifetime, replicate something that already exist or do something that may not seem very useful (eg. the plethora of "social" apps). In all these cases, the impact of software engineering is actually rather negative: wasted natural resources, wasted time, etc. Even when the software is innovative and has users, we can often question its usefulness for humanity. For example, high-frequency trading or mass destruction weaponry are fields where the question of meaning may arise. I think the question of meaning is important to software engineers because it's not obvious at all that what they do is meaningful.

<comment><USER16> I think you illustrated the reality of software engineering, not the perception. The perception is that all these startup entrepreneurs and software engineers are ninjas and rockstars. Software Engineers are curing cancer and ending terrorism with big data, all while becoming millionaires over night.

I believe the reason this question is asked, is not because of the stark reality of software engineering, but because of the perception.

</comment></comment></comment><comment><USER18> You have to look at this in a different light. Making a difference as Software Engineer is possible. But digging deeper into the root of how its possible is not a career related move. Many people make tremendous contributions everyday (i.e. See Github) in many fields everyday. However, your determination and drive as an individual will deter your actual affect on the world. To put it simply:

 __ _We choose to become leaders. We choose to change the world. No one is_ _born_ * into it __*

In the massive wealth of technology and information we have today, it is very much possible to become successful as an engineer. But much too often do people outweigh the failures of the many with the successes of the few. Think of how many startups have failed before one actually ___succeeded_ __. It is ungodly and daunting to look at the statistics but that should never stop someone. It should drive you as a professional in not becoming one of the many and deciding to fight to become one of the few.

TL:DR; I do not believe that being a software engineer alone gives us all the capability of changing the world. It must be coupled with drive and determination before it becomes an option

</comment><comment><USER19> I'm an engineer working for Change Heroes (www.changeheroes.com). Every Sunday night all I can think about is how excited I am to work on the product the next week. I've never felt this way before.

Change Heroes helps raise money for charities and non profits. Our impact includes 217 funded projects so far including schools, wells, and aid packages. Yes, as an engineer, you can make a huge impact on the world.

</comment><comment><USER20> Here's an opportunity to make an impact using software development [0]:

> International Development Design Summits (IDDS) Lahore is a hands-on design > experiences that brings together people from all walks of life to create > low-cost, practical innovations to improve the lives of people living in > poverty.

(you can apply before July 1st, and participate in Lahore, Pakistan).

The impact may be small, but certainly significant for many (things that have come out of these summits are water purification and monitoring technologies, sanitation technologies, healthcare information through SMS, and many more)

[0] [http://www.idds-lahore.com/](http://www.idds-lahore.com/)

</comment><comment><USER21> You won't be able to have a large impact as just a software engineer, but you can do a little bit. If you really want to change the world in a big way, you have to be a businessman and leverage the power of others' efforts.

</comment><comment><USER22> This 'Google Ideas' post from the front page appears to be an example of what you're talking about.

[https://www.google.com/ideas/](https://www.google.com/ideas/)

</comment><comment><USER23> Anything changes the world, some inconsequential, some not, some for better, some for worse, some neutral. If you mean make a positive impact on the world though:

Probably one of the most direct ways to help is to work for a company whose impact you believe will help change the world in a positive way.

E.g. online learning, green tech, or whatever you personally are passionate about.

Another example: in the last 5-10 years there's been a lot of work done in helping consumers understand their power and water usage. This lets us make smarter choices and reduce our energy and water usage.

</comment><comment><USER24> As a software engineer, you might not be able to change the world, but you'll at least get to rebuild it ... everyday, when you come in the morning and git pull latest.

</comment><comment><USER25> The boring answer is that as long as you are making money, you can donate some to charity.

Making lots of money and donating it to an effective charity is usually a more efficient way to change the world for the better than choosing a sub-optimal career trying to change the world by yourself.

If you think you can be a good software engineer, you should go for it.

<comment><USER26> > Making lots of money and donating it to an effective charity is usually a > more efficient way to change the world for the better than choosing a sub- > optimal career trying to change the world by yourself.

Been thinking about this as well, would love to hear any further thoughts on this.

Following that same strain of logic, if I want to build a successful business, why not choose an optimal career and then hire someone to build the business for me?

<comment><USER25> > if I want to build a successful business, why not choose an optimal career > and then hire someone to build the business for me?

That's what angel investors do, right? First they make lots of money and then they invest on other people's business.

</comment></comment></comment><comment><USER27> Yes. I feel software engineers at drchrono make a difference every day by giving amazing software to doctors who use it to treat more patients and make more money doing it (which will keep them from rage quitting medicine after using other EMR software which is just terrible.)

</comment><comment><USER28> Here is a (hardware engineering) example of how a group of volunteers are helping make the world a better place:

        http://www.unite-to-light.org/

(developed lights with solar-powered storage for students without access to electricity).

</comment><comment><USER29> I didn't used to think so.

I'm less cynical now.

It all depends on where you want to work, and what you want to do.

</comment><comment><USER30> AI research/implementation -> AI -> Better AI -> Cataclysm or utopia.

Either way the world is changed.

Just an example.

</comment><comment><USER31> No, problem solvers that can implement solutions can change the world.

</comment><comment><USER32> Certainly, if you use your skills to make an actual impact.

</comment><comment><USER33> Asking this unironically, and seeing unironic answers, is painful...painful to read.

<comment><USER34> Why is that?

<comment><USER33> C'mon dude, look at the site you're on.

<comment><USER35> Can you elaborate? I don't have the slightest idea what you are trying to imply and it seems I'm not alone.

</comment></comment></comment></comment><comment><USER36> Of course you can! It'll all depend on what career choices you make. If you become a software engineer to then go develop corporate Java software for Big Corporation Inc, then you won't have much of an impact. Join a startup with a valuable vision and a good product and you definitely will.

<comment><USER36> I don't get why my answer is being downvoted. I guess I didn't write a super philosophical, clever reply that'd "change this 'ere lad's life forever". I wasn't going for that either.

I just wanted to give my perspective, as someone who's done both, regarding 'working for a product you love' and 'working for a big company because you get paid well'.

If anyone got personally offended because they think they can be a corporate Java developer and change the world by contributing to a charity... well, I guess it's pretty clear the OP wasn't asking about that. Anyone can do that, no need to be a software engineer.</comment></comment>


If management isn't a promotion, then engineering isn't a demotion - <AUTHOR> https://charity.wtf/2020/09/06/if-management-isnt-a-promotion-then-engineering-isnt-a-demotion/ <comment><USER2> This is great in principle, but it's only a starting point - there are huge issues that are left unaddressed.

The biggest issue is that it presents status as if it can be conferred independently of power, or as if we can change the dynamics purely by changing the way we think about this.

Managers have power for many reasons - they generally do have a significant influence over subordinate's career prospects, and they have access to all kinds of organizational information that ICs simply can't obtain, at the very least because they can usually spend time in conversation with other managers or in meetings with people at higher levels in the organization.

Engineers obtain power through indispensable or highly valued domain expertise or institutional knowledge.

Both forms of power are real and confer status.

If you move from management to engineering and your power as a manager is not replaced with corresponding power as an engineer then _you have been demoted in status_ , irrespective of how the organization claims to value people.

Organizations that want to deny this, even with the most positive intentions end up gaslighting their employees. I.e. saying things like "Our organization doesn't see these moves as a demotion", when manifestly the individual's power and status has changed in a way they and everyone else can directly perceive.

<comment><USER3> ICs largely wield informal power within organizations, managers have real formal power, codified in organizations charts, rules and regulations. Whenever there is conflict between the two, almost always formal power wins over informal. Not the least because managers control salaries and hire/fire decisions.

<comment><USER4> The key for those with informal power is how to leverage it to leverage the formal. Get managers to trust you, and you can have both.

<comment><USER5> That's still nowhere near the power of official management.

At a bare minimum a manager can summon their underlings, they will come and listen because they depend on him for promotions and have to make a good impression.

Try to get your teams or coworkers to do some tasks as an engineer. You will quickly realize you have no power.

<comment><USER6> I mean, this varies team by team and company by company. To spit an anecdote into the bucket, I asked some teammates to help me take care of a problem, and they did - and I don't _think_ I hold a particularly large cache of informal power.

<comment><USER5> Send a meeting invitation and see how many people show up. Preferably a videocall or a presentation, something not too attractive to come to.

Then do the same thing as a random employee. See the difference in how much of your team doesn't show up and absolutely nobody from the other team across the corridor comes.

<comment> yjftsjthsd-h > Then do the same thing as a random employee. See the difference in how much > of your team doesn't show up and absolutely nobody from the other team > across the corridor comes.

And then do it not as a random employee, but as the senior employee who isn't technically anyone's manager but who is known for having saved the last 3 major projects by pointing out pivotal details and coordinating with other teams, and observe that everyone who's worked with that person shows up regardless of the org chart. YMMV, but let's compare apples to apples.

<comment><USER5> > and observe that everyone who's worked with that person shows up regardless > of the org chart

Too bad it's only a handful of people.

Well, maybe less than a handful because most of the people you saved in the past 3 years have moved on to another company or another role. They're not here for you anymore.

However the manager got that power instantly since the minute he joined, by virtue of the position. It's not an apple to apple comparison indeed.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER7> I think the focus on "managers have power" is overstated. Places I've worked (as both an IC and a manager) there's also greater responsibility and higher ups hold the managers much more accountable than the ICs. I know many people who simply do not want the responsibility or accountability. Maybe the work is not better or worse, easier or harder, but often there is more pressure/stress.

<comment><USER8> > managers have power

Respect, too. I've lost count of how many times I've been stuck waiting for somebody in the network group to grant me access to some resource I need to do my job until somebody that they actually respect (that is, a manager) tells them that they have to.

<comment><USER9> Isn't this normal? If they're not your manager, it's not their job to make sure you can do your job.

Though arguably the manager should have arranged for an easier process if this happens more than a few times.

<comment><USER10> Too much bureaucracy for access to internal resources (api access, repository access, requisition of servers, google doc permissions, w/e) slows down ICs and is a sign of bad corporate culture.

There's a difference between needing to ask for access to shared resources and asking for someone else to do work. So many corporations silo there teams so deeply that PRs from outside members aren't even considered. Collaboration isn't given anything more than lib service.

<comment><USER11> If a company gets too big, they have to start worrying more about internal adversaries. That's why you see those kind of things.

<comment><USER12> Nope. Those kinds of things are almost entirely related to control and power, not real security.

If you're legitimately concerned about internal adversaries you start by finding and catching them. Once you've set up effective mechanisms for doing that you can start to introduce stricter controls around sensitive data/operations based on what you've learned about how your internal adversaries operate. You use a light touch when doing this, because you understand that every extra control you put in place introduces inefficiency into the organization.

</comment></comment></comment></comment></comment></comment><comment><USER13> IC = individual contributor. I was trying to figure that out till they defined it halfway through the article.

<comment><USER14> Thanks for that, I had seen the same acronym on that article from today of a (I think former) Salesforce staff engineer describing her experience there, and of course that I didn't know what it meant.

</comment></comment><comment><USER15> In my opinion the best way to deal with the issue of hierarchy would be to make management an assignment, not a job. Someone takes on the role of "manager" for an upcoming project, at at the conclusion of that project they go back to engineering.

Apparently this is how Valve does things. I've heard Gabe Newell say in an interview (that I can't find right now) that often times the management role is given to the newer employees. They see it as a big honor at first, but by the end of it they're pretty happy to go back to engineering and let someone else deal with the responsibilities of management.

<comment><USER16> Maybe this is why Valve can't seem to actually finish a project to save their lives at this point!

<comment><USER15> They actually _have_ admitted that that's an issue, although I'm not sure what they did about it. Recently they shipped the Valve Index and released Half Life: Alyx, both of which are excellent. They also released Artifact (it did terribly, but they did release it), and a UI overhaul for Steam along with a lot of new features. So in recent years they've been doing fairly well I think.

</comment></comment></comment><comment><USER17> I think one thing that's missing from this article is what engineers at higher levels of IC status (up to VP) do. When a Sr. Engineer is choosing between a Manager and a "Principal Engineer" type role, I have to remind them that PEs also don't do much depth coding. They write proposals, discuss architecture, sit in meetings. Ultimately, the same mechanisms that Managers use.

There's certainly SOME PEs that go deep in some areas but the majority I've worked with are broad owners of technology in large organizations. The difference between the two roles often has to do with what balance of time you want to spend influencing others for broad goals vs. building up jr. folks to accomplish your team's goal.

In either case your authority is derived from whether people "under" you want to work with you. People who can't get a set of people to go a certain direction won't last long in either role.

<comment><USER18> This article was on HN yesterday and I like the way that four different types of staff engineers are described:

[https://news.ycombinator.com/item?id=24437715](https://news.ycombinator.com/item?id=24437715)

<comment><USER19> On the same topic, [https://staffeng.com/guides/staff- archetypes](https://staffeng.com/guides/staff-archetypes)

</comment></comment></comment><comment><USER20> One fix could be to give managers a title that is of lower rank than an engineer e.g. a manager's official title should be "support" and ensure that every part of their role is about supporting the engineers they are responsible to.

<comment><USER21> That kind of reminds me of how military dogs usually outrank their handler: [https://science.howstuffworks.com/military-dogs-outrank- hand...](https://science.howstuffworks.com/military-dogs-outrank-handlers.htm)

I've heard that if you abuse the dog, you can get charged with assaulting a superior officer.

</comment></comment><comment><USER22> They're promotions if you get paid more. They're demotions if you get paid less.

People refuse promotions all the time because they don't want the role/responsibility and prefer other stuff besides money. Doesn't mean it wouldn't have been a promotion.

<comment><USER23> You get more power as a manager because you have influence over engineers and their careers, even the ones that get paid more than you unless they have been with the company for a long time and have built relationships with the higher ups.

<comment><USER22> If your compensation isn't higher when switching ladders, then it doesn't matter how much influence you have. It's a demotion.

To be clear, that's not necessarily bad.

Imagine your boss coming to you to congratulate you on your promotion, and then mentioning that your pay will be reduced by 10% because now you're management. How'd you feel about them calling it a promotion?

</comment></comment></comment><comment><USER24> It doesn't matter how you define what is or isn't a promotion.

The real issue is that _there isn 't one, single right answer_ about what role is right for everybody.

It is not the case that being a manager is strictly better in every way for every person. That's the assumption that people need to let go of.

<comment><USER25> > That's the assumption that people need to let go of.

Its not an assumption, its a fact that managers everywhere on average get paid more than IC.

Unless you are working for a non profit or some cutting edge change the world stuff. Why would you work for less.

<comment><USER26> > Why would you work for less.

Because I am more than comfortable on my engineer salary and being a manager would make me miserable.

<comment><USER25> Sure but by that logic no two jobs can ever be compared to each other. Someone can always say because I am comfortable doing X than Y, thats not something anyone can respond to.

</comment></comment></comment></comment><comment><USER27> Seems like people instinctively associate status with having control over other people. Maybe engineers could fight back by calling their programs cute names or putting hats on servers? Cloud computing is not helping though.

</comment><comment><USER28> One further complication is when a move back to engineering is coupled to a change in employer.

As an interviewer, seeing somebody whose last job was in a management role apply for an IC position is a warning signal for me. Do they just apply for that position to get a foot in the door (especially for a move to a FAANG), and will they immediately angle for a move back to management?

So this is generally an area where I would ask follow up questions (and probe a bit harder in the technical interview, to assess whether any rust has crept into the technical skills). It's not an automatic red flag; we've hired former managers for IC roles and it worked out, and leadership ambition can be a positive in any role, but it's worth some exploring.

</comment><comment><USER29> > The work done by a database engineer is different from the work done by a VP > marketing, or a director of database engineering. It is not inherently > better or worse, easier or harder, more or less deserving of praise and > admiration. It is simply different.

The big problem is that it is the VPs and directors who are getting together and deciding what everyone should be paid. And, lo and behold, somehow, they almost always decide that VPs and directors should get paid more than database engineers.

<comment><USER30> Eh, market forces are more influential than this. You can't simply decide how much to pay engineers, any more than you can decide what your rent is going to cost.

<comment><USER31> The rule of thumb is "the closer you are to the money, the more you make".

People rarely demand what they are worth since they usually value security over maximizing income, but if you are familiar with the numbers, then you are in a much better negotiating position.

People should always keep applying to jobs, even if they don't intend on accepting them just to keep a pulse on the value of the labor they are selling.

</comment></comment></comment><comment><USER32> I am sure this is going to sound silly, but I honestly still have trouble figuring out what value is contributed by managers and executives at all.

I can't get past the gut feeling that essentially the model hasn't changed for thousands of years: wealthy owner, boss with whip, wage slaves being watched over.

I mean I imagine a group of people who are actually competent and intelligent working together. Some people would have more seniority and power, but everyone is actually a useful worker there involved in the day-to-day work.

I feel like most managers I have had have spent the majority of their efforts trying to pretend that problems don't exist and reduce the amount of actual useful engineering work in order to push _anything_ out faster and cheaper. Or just basically telling people to hurry up.

That's why I think we had the Boeing crapshow. The guy at the top was actually good at doing what management actually is for most companies, like I described above. He avoided doing real engineering and got a product out. And then hundreds of people died. And still they tried to blame the engineers.

It actually comes down to a class struggle for me. Parasites without integrity have managed to work their way to the top while feeding off of the honest workers below.

Sorry I know it's kind of extreme but I am just being honest.

<comment><USER33> You need management because:

\- you need project management. It is an administrative work.

\- you need people management. You can leave that to HR, they will do a very poor job, so it's better to have a person from the team fill that role, the more senior and experienced the better. This is admin work

\- you need to simplify large organizations into smaller, manageable teams that can focus on items they can swallow. You need team leaders (for the 2 reasons above) and good communicators to keep the team in sync with the rest of the company

\- when you have many teams you need "teams of teams", that is one or more management levels.

In any army you don't have management, you have a chain of command. You have generals with a strategy and a vision, you have officers that lead portions of the army, you have NCOs that lead small teams and you have specialists or riflemen that do the grunt work. They all have a role and in most cases you find the higher the rank, the higher the competence, otherwise nobody would follow in battle an incompetent officer (or even shoot him in the back, it's easier in a battle than getting rid of a bad manager in the office).

This hierarchical organization came from real needs and stays there because the need is still there. You can argue some hierarchies gets corrupted, in theory all do, but a hierarchy is generally better than no hierarchy whenever the group is large enough.

<comment><USER34> Tangential on hierarchies and meshworks by Manuel De Landa: [http://netbase.org/delanda/meshwork.htm](http://netbase.org/delanda/meshwork.htm) "To make things worse, the solution to this is not simply to begin adding meshwork components to the mix. Indeed, one must resist the temptation to make hierarchies into villains and meshworks into heroes, not only because, as I said, they are constantly turning into one another, but because in real life we find only mixtures and hybrids, and the properties of these cannot be established through theory alone but demand concrete experimentation. Certain standardizations, say, of electric outlet designs or of data-structures traveling through the Internet, may actually turn out to promote heterogenization at another level, in terms of the appliances that may be designed around the standard outlet, or of the services that a common data- structure may make possible. On the other hand, the mere presence of increased heterogeneity is no guarantee that a better state for society has been achieved. After all, the territory occupied by former Yugoslavia is more heterogeneous now than it was ten years ago, but the lack of uniformity at one level simply hides an increase of homogeneity at the level of the warring ethnic communities. But even if we managed to promote not only heterogeneity, but diversity articulated into a meshwork, that still would not be a perfect solution. After all, meshworks grow by drift and they may drift to places where we do not want to go. The goal-directedness of hierarchies is the kind of property that we may desire to keep at least for certain institutions. Hence, demonizing centralization and glorifying decentralization as the solution to all our problems would be wrong. An open and experimental attitude towards the question of different hybrids and mixtures is what the complexity of reality itself seems to call for."

</comment></comment></comment><comment><USER35> Interesting to compare this with practices in other professions, eg Doctors, Architects, Scientists and so forth.

Certainly management happens in those areas, but I believe that as your career progresses you would naturally be expected to take on more management activities - there isn't a dual track in quite the same way.

In more recent times the waters appear to have become muddied somewhat, with the appearance (and glorification) of administrators, project managers, 'CEOs' of various kinds.

<comment><USER36> Most of those people are independent entrepreneurs or senior ICs, not managers.

</comment></comment><comment><USER37> This reads more like an attempt to extract "powers" that managers have and give them to staff / principal engineers, but I think this is a bad mistake.

The people in those roles should influence those things, sure, but not "own" them, under some faulty idea that management doesn't lead or own strategic and technical roadmaps or execution.

An engineering manager is someone who takes inputs from their immediate team or staff, as well as cross-functional stakeholders like product managers, directors, staff/principal engineers, and more. Then you synthesize that into team or org investments: projects, roadmaps, staffing, resources, tools.

An engineering manager is not someone who "just executes 1-1s and fills out HR feedback cycles."

They are more like a staff engineer or architect who takes an expanded point of view to include team dynamics, career growth, budgets, strategy and resources into the solution design process.

This often just _is_ a promotion above staff or principal IC roles because it usurps and supersedes a lot of that work while also involving additional new work managing career growth, hiring, training, resource investment, etc.

I'd put product manager and engineering manager at the top of the promotion pyramid (excluding director / VP / executive roles). Staff and Principal ICs are next, then the rest of ICs.

</comment><comment><USER38> If one group has firing power, and the other does not, they are not equal, regardless of what's written down.

<comment><USER39> It's not that simple.

First line managers rarely have the power to simply fire on a whim. There has to be cause, and it needs to be evidenced, and there will be multiple people involved in the decision.

If you're a more senior engineer, you may report to a manager of managers, and line managers may be your peers organisationally, and probably not paid as well as you are.

Engineers also get to make a different kind of decision to managers; and it's often the longer term, higher leverage, more strategic kind of decision. Line managers are much more operationally focused and rarely have the bandwidth or tasking for strategic decisions.

</comment></comment><comment><USER40> Managers do have some formal powers engineers don't. They are three: they have access to budget; they perform the management function which assigns resources to problems. Since demand on resources almost always exceed available resources, they must assign priorities well. They also have high input on hiring choice.

Doing this well requires,

\- requiring internal service suppliers to be (internal) customer driven. Too often however things we'd never put up with an external supplier we'd pay money for are tolerated and even protected by management. Internal suppliers are required for use. It's hard to fire them.

\- cross functional management: reduce silos. Again my area and your area might be "clean" but users of both of our services are often left to die in the crummy alley way between our groups. That's a major source of frustration for engineers.

\- promote openness and honesty: there's too much happy talk and PC correctness at the office.

\- supply chain management: engineers are often confronted by too many point solutions that don't integrate. See above points. Managers need to cull and combine supply chains time to time.

\- be customer in not supplier out. Older companies often have organizational norms where what can be produced is more a function of what internal politics allow or company culture tolerates than what customers want. Managers need to be on top of this.

\- top management needs to give TLC to middle management perhaps the toughest job. Here they (middle mgmnt) cannot micromanage while they are held accountable for broad corporate goals like lean organizations, top line growth, reduce expenses, and better expense control. They neither are hands on nor are too far from the real work to be too abstract.

\- teamwork (see schutz's human element the best single read on this) namely that ridgid individuals are the single fastest way to break teams. Today's work depends more than ever on good teams.

When managers do above well it's a pleasure to work with them. It's a difficult task.

</comment><comment><USER41> If you think there are a lot of bad managers now wait until they're explicitly paid less and aren't accountable for their team's impact (what happens when you shift all accountability of technical strategy to TL).

</comment><comment><USER42> Solution: take away some of the formal powers of management - for example the ability to assign resources on personal decisions. A manger wants X to happen so spends Y of her budget to hire a team to do it.

That is power.

Simply make all such decisions open, transparent and democratically elected - in other words have employees vote.

We live in a world where we value democracy but spend our working lives in autocratic dictatorships.

Yes Coase and efficient allocation, but really I think information revolution is changing that equation.

Democracy in companies now! Struggle brothers !

<comment><USER2> I upvoted you because the point about spending our working lives in dictatorships is very valid, but I don't think for a second that voting would work.

<comment><USER42> But ... let's look at GE.

It's basically worth nothing at net now. And it was the great poster child for brilliant management decisions - that take any industry and run it using GE managers and allocate capital the GE way and it will win.

Turns out not so.

This is the lesson of history - great Kings and Emperors really do make great world beating decisions. But anything less than great leads to disaster whereas parliaments don't do great but they do keep growing and managing well for centuries.

We need to stop relying on great CEOs and kings and look for ways to run a company well.

PIRC has lots of research on the central founder risk to long term value.

but just ask this - imagine Elon Musk retires tomorrow. Do we want to hope we hire his clone or do we want a strong driven community of engineers to keep knocking out battery improvements for 2 decades.

I know which one i want my money in, and which one will make better headlines.

<comment><USER2> Everything you just said is fine, however, nothing in your comment in any way suggests or supports the idea that voting would work.

Agreeing on a problem is not the same as agreeing on a solution.

<comment><USER42> Ok - so we agree on a problem.

Just to ask, what solution have humans found to replace dictators and kings apart from (weakly or strongly) elected parliaments ?

Rome had two consuls (or three), but I am not sure that's a big step beyond one.

I just think we need to find the right ways for democracy to evolve not just chuck it in the bin :-)

<comment><USER2> Again, no argument from me, but also no argument from you that direct democracy would work as a form of corporate governance.

Can you find even one example of where it has? The closest I am aware of is Mondragon, but Mondragon has almost no direct democracy - voting is for people rather than decisions.

<comment><USER42> As a brit I have recently learnt to be wary of direct democracy (ala referendums) but frankly I will take any form of democracy as a bastion against the mono-culture.

Perhaps we can take heart from the media - [https://www.ft.com/content/afe3386e-d3bb-11e4-a9d3-00144feab...](https://www.ft.com/content/afe3386e-d3bb-11e4-a9d3-00144feab7de) \- The Guardian's editor was appointed following an indicative vote from staff.

But I would suspect that appointed leaders and elected policies will be a much easier sell than the other way round. Any leader can happily say they always agreed with a policy forced on them. It's much harder to say you agree with them firing you.

And I shall have to do some more research. Thank you for pushing

<comment><USER2> One argument would be that working for a business and not finding another job is inherently vote for the leadership.

<comment><USER42> That sounds rather weak - the fact I have not left the country does not imply I have voted for Boris Johnson as PM.

Although Trump could then claim that 11M card-less Mexicans who have not left the USA in November were actually voting for him, which would give him California in the Electoral College.

In the end I do not see a way to imply someone's vote - one needs to actually count their vote in the normal way.

I respect your position but I remain convinced that _some form of_ democracy is needed in corporate life.

<comment><USER2> You must realize that leaving the country is not even remotely comparable to changing job. My argument may be weak for some other reason but this is not a valid one.

It's also worth mentioning that there _are_ democratic workplaces.

They are called cooperatives.

Another example: [https://en.m.wikipedia.org/wiki/John_Lewis_Partnership](https://en.m.wikipedia.org/wiki/John_Lewis_Partnership)

Relevant questions seem to be - why are they not more successful or widespread?

Also, have you considered working for one?

<comment><USER42> Yes good point on the co-op.

Relevant questions seem to be - why are they not more successful or widespread?

The co-op store (similar food john lewis) sprang from the mid victorian co- operative movement where different models where tried out by wealthy philanthropists who essentially gave back huge amounts of their wealth to their workers

I think that last sentence shows why it is not more widespread - perhaps we simply need to force companies to become co-op/ partnerships at a certain state of growth?

<comment><USER2> By last sentence - you mean where I ask whether you have considered working for one?

<comment><USER42> Ok second to last :-)

But this seems interesting for the actual last sentence

[https://electricembers.coop//pubs/TechCoopHOWTO.pdf](https://electricembers.coop//pubs/TechCoopHOWTO.pdf)

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER43> This hit close to home.

A manager at a previous company kept saying that his role was to support engineers and that management was a separate track, that he wasn't my superior (probably from some internal training BS). He acted completely the opposite way. He obviously wielded more power by having more information, and, to add insult to injury, played tech lead as well.

Suffice it to say, over the years almost all the original ICs have left his team, and he's still wondering why he can't get the team to run.

Good riddance.

</comment><comment><USER44> Does anyone do the pendulum? How well does it work?

<comment><USER45> I've done it. Like many people, I moved to the management track when my first kid was born. I realized that moving up as a manager required actually competing with the other managers and being better than them. It wasn't going to happen. As the low level project manager, I got the low level projects.

I moved back to IC when they finally posted an opening to backfill my job. I went to my boss and said: "I can be a mediocre manager or a great engineer, you choose."

Later on there was a big reshuffling of the department, and I was asked if I wanted to be a manager. I said yes. More reshuffling thinned my team down to 2 reports, and I kind of oozed my way back into IC by just doing IC work for people when I wasn't managing.

All of these moves were promotions in both pay and title.

One thing you learn by being through a few annual cycles at your business is how it _really_ works. For instance, things like performance reviews, pay increases, promotions, etc. This knowledge will serve you in any career.

<comment><USER8> > project manager

In every organization I've ever worked in, project management was very different than management management: project managers never ended up actually "managing" people as in hiring, firing, budgeting, setting direction for, etc.

<comment><USER45> Quite true. I was a people-manager the second time around. And as an entry level manager I had very little actual discretion over those things. I'm just giving myself as an example of someone who went back.

</comment></comment></comment></comment><comment><USER46> There is another aspect of it: when you want to change from management to "lower" status jobs, the person hiring you will have a hard time understanding your decision. "You understand you're overqualified for this post, right"? They get suspicious even if you're a perfect candidate.

</comment><comment><USER47> This is an interesting instance of two HN tropes in conflict: Managers are able to 'create' more (by improving IC productivity) than any individual contributor (and thus more value), unless you believe in 10x engineers.

<comment><USER48> A senior IC who is a technical lead or area lead can do the same without formally leading anyone.

I know of quite a few teams where there is a high level manager and an equal level IC who report to the same director. They're both responsible for the same scope of work, (and this is oversimplifying) just one is responsible for the tech and one for the people.

<comment><USER49> Yeah, I'm the technical lead and report to a director.

My contributions are 50% personal, 50% "team acceleration" (ie: ensure your teammates are able to get things done quicker including pairing, code walkthroughs, documentation, last minute team changes to ensure a project succeeds)

</comment></comment></comment><comment><USER50> If your company is like mine, then they'll combine the jobs and pay you the lower of the two salaries while selling it as a great opportunity.

Simultaneous promotion and demotion.

</comment><comment><USER51> I really hate these kinds of articles, primarily because they are trying to sugar-coat reality here.

The fact of the matter is this, management _is_ a promotion, because you get paid more and you gain access to more powers. If it were _not_ a promotion within the company, no one would take it up.

Management is a simple deal, you give up power _outside_ of the company for power _within_ the company. It is far easier to join another company as an Engineer than it is as an Engineering Manager.

However, I agree that it should not be. It should not be there to create a sense of hierarchy. This is why, I believe that you should rotate your managers _within_ the team. Back at my old company, we had people who would go into a Team Lead position for a certain project, and then someone else would take over.

Pay was already high, so no one really wanted to take up lead because it meant more responsibility.

This is how power needs to be treated, you should not have it for too long. This is why we have term limits.

<comment><USER52> I've never seen "team lead" as a management role. Team lead is more of technical tie-breaker role on technical decisions. It's not a management role if you can't fire someone.

<comment><USER53> I am a team lead and I'm definitely a manager - 80% of my time is managing people, not making technical decisions, that's more for technical directors(who in turn do not manage anyone). And why would managers be able to fire people though? I can recommend that someone be let go to HR, but ultimately that's not up to managers. The process takes at least 6 months unless for gross misconduct anyway, it's far too complex to be up to a single person.

<comment><USER54> This sounds a lot like differences in individual companies' organizational structures—I bet there are some where a "team lead" is more of a "first among equals," while others, as you describe, are primarily managing their group of technical specialists of whatever stripe. Similarly, in some organizations, managers tell HR, "We should fire this person," while in others, they tell HR, "Hey, I just fired this person."

<comment><USER51> I've never ever felt anything other than a first-among-equals as a lead. I've learned so much from my team, and I'm proud of where my former team members are now. I dont intend to be anything else should I become a manager again,</comment></comment></comment></comment></comment>