Ask HN: Recommended resources to learn the Linux kernel and OS theory? - non-entity So recently I did a couple of minor patches on the FreeBSD and NetBSD kernels and played with some linux kernel. It was the first time in a a few years, I've been excited about programming.
Unfortunately I'm still completely lost. It seems that there's so much to learn and every kernel update breaks whatever you just finished writing. I see terms in unfamiliar with and often find myself googling basic OS concepts.
Are there any recommendations reading materials to get a better grasp on OS theory and / or the Linux / UNIX kernels and programming for them? <comment><AUTHOR> The OSDev wiki can probably teach you anything you need to know about "OS theory" (and practice, mostly on x86): [https://wiki.osdev.org/Expanded_Main_Page](https://wiki.osdev.org/Expanded_Main_Page)

Depending on your style of programming, I'd recommend maybe not working on Linux if you are trying to learn kernel development and OS internals.

Have you looked at Haiku ([https://www.haiku-os.org/](https://www.haiku- os.org/))? We have a _very_ well organized and commented modular-monolitic kernel, and a pretty active development team with a wide range of experience levels. I'm more than happy to help you (or anyone else) learn OS development!

Some example Haiku kernel code:

* "load_image_internal()", which is responsible for creating processes: [https://github.com/haiku/haiku/blob/master/src/system/kernel...](https://github.com/haiku/haiku/blob/master/src/system/kernel/team.cpp#L1679)

* a recent change by me to replace a global lock with two local ones: [https://github.com/haiku/haiku/commit/37eda488be1c9fee242e8e...](https://github.com/haiku/haiku/commit/37eda488be1c9fee242e8e4bf6ca644dd13441d8)

<comment><USER2> > I'd recommend maybe not working on Linux if you are trying to learn kernel > development and OS internals.

Could you also tell why ?

<comment><AUTHOR> The Linux kernel is really, really large, and changes extremely rapidly, to the point of breaking such "experiments" (as the original poster already noted.) It's also rather loosely commented and, aside from a handful of core systems, not so well documented.

The BSDs do much better here; they have excellent manpages for almost all kernel APIs, manuals for how the internals work, and very good source code commenting and organization practices. Haiku is not as good as the BSDs on manpages and manuals, but that's the only real way we differ from them (in this respect.)

</comment></comment></comment><comment><USER3> One important part about learning how the Linux kernel works is understanding the details of the system's interface the kernel provides. In my opinion, there is no better book out there than the "Linux Programming Interface" by Michael Kerrisk: [http://man7.org/tlpi/](http://man7.org/tlpi/)

It provides extremely detailed information about everything going on in Linux, as well as example programs and exercises to help you further your knowledge. While it doesn't get deep into kernel theory like Tanenbaum's books tend to do, it will provide you with a greater understanding of how things work, IMO.

<comment><USER4> £50 on kindle! Looks like a great book, but that's punchy. (Maybe that's standard for a text book like this, not something I would normally buy)

<comment><USER5> It's a huge book (1552 pages), much larger than even your average OS textbook. It's more like a Linux encyclopedia.

</comment></comment></comment><comment><USER6> Many folks here are recommending reading up on production grade operating systems which might be a very steep learning curve.

I'd recommend starting with an academic operating system to nail the fundamentals down, once you have a solid foundation you can then use all the excellent links provided by everyone else and ramp up on production grade kernels.

I have linked to multiple course syllabus'es below from excellent institutes, pick whichever feels most comfortable.

Harvard with MIPS based OS/161 (I learned this personally): [http://www.eecs.harvard.edu/~cs161/syllabus.html](http://www.eecs.harvard.edu/~cs161/syllabus.html)

Berkeley EECS 162 (John Kubiatowicz is amazing and is behind RISC V) [https://inst.eecs.berkeley.edu/~cs162/sp19/](https://inst.eecs.berkeley.edu/~cs162/sp19/)

MIT OCW: [https://ocw.mit.edu/courses/electrical-engineering-and- compu...](https://ocw.mit.edu/courses/electrical-engineering-and-computer- science/6-828-operating-system-engineering-fall-2012/lecture-notes-and- readings/)

</comment><comment><USER5> Yes, there is just so much, and it is not thoroughly documented. "BPF maps" in particular are a) a special feature of a special feature that's Linux- specific, not anything that generalizes across OSes, and b) a feature for userspace anyway, not for kernel internals. But even longstanding kernel- internal features aren't well documented (e.g., the other day I was trying to figure out what struct file's f_version does, and I think there's genuinely no docs for it.) So honestly I think the answer there is to not feel bad about not knowing everything.

The basic trick of dealing with the kernel is becoming comfortable working in a large codebase most of which you don't understand, and figuring out how to find what you need. Honestly, git grep is one of the best tools here. Get some practice finding some specific thing and where it's implemented, e.g., find a syscall (git grep SYSCALL.*foo) and trace what it calls. Find the definition of a structure inside include/ and see who uses it. Get comfortable with the kernel's OO-ish system of operations structs, and get some practice tracing both "this function makes a generic call, here's a sample driver that implements it" and "this is an implementation of a generic function, here's the syscall that calls it."

Beyond that, reading [https://lwn.net](https://lwn.net) 's articles is invaluable, partly for the clear prose coverage and partly for the breadth of what they talk about. (You don't need to pay unless you care strongly about this week's updates - you'll learn plenty from reading articles a week behind - but support them if you can, they're an important resource.) Again, you're not going to follow exactly why e.g. Google wants a new syscall for "restartable sequences" on your first read, but you'll get a sense of what is involved in adding a syscall, how various concurrency models work, what other kernel features are relevant, etc.

What sort of OS concepts are you finding yourself Googling? I will say that actually doing a college OS class is what made things like virtual memory management click in my head. It's an intensive approach but writing code in a much smaller kernel than Linux is a valuable way to understand concepts without being drowned in real-world optimizations and edge cases and portability.

<comment> non-entity Perhaps I should've said more. I do often end up googling basic OS stuff, especially related to memory (software interrupts still confuse me though, I understand the concepts, but never the implementation), but even more so have been my ignorance of hardware level concepts. For example trying to patch the code for a network card the other week, I learned what watchdog timer is, I've never heard of that prior.

<comment><USER5> Working through an academic OS class where you implement interrupt handling is super helpful for that - hardware interrupts and software interrupts are similar but not _quite_ the same and actually writing the interrupt handlers makes some things clear.

I can endorse MIT's 6.828 (I've taken it and taught a version if it), which (at least before this semester, they seem to be redoing things) has a lab about setting up interrupt handling: [https://pdos.csail.mit.edu/6.828/2018/labs/lab3/](https://pdos.csail.mit.edu/6.828/2018/labs/lab3/) It's entirely possible to complete the labs on your own, and doing labs 1-3 is worthwhile, but it's also a significant time commitment.

Watchdog timers fall in the category of "there is so much" \- they're not crucial to how network drivers work, but drivers in a production OS have to care about every feature that might interact with them. :(

</comment></comment></comment><comment><USER7> My somewhat off-beat recommendation is "Lions' Commentary on UNIX 6th Edition with Source Code". The sources to a 1976 version of Unix written in an archaic dialect of C and targeting a long-dead CPU architecture are obviously not of immediate relevance to the modern world, and may well not be to your taste unless you already have some sympathy with the idea of retrocomputing. But if you do: 6th Edition is small enough that you really can read through the whole set of sources and understand pretty much how the whole thing is put together; the basic bones of the design are not so far away from Linux, so it gives you a workable conceptual model of what the shape of the Linux sources are and what the various important parts are; the commentary is really good, explaining the inscrutable but nudging you to figure things out yourself too; and overall it is practice in reading a large volume of somebody else's code, which is something you're going to be doing all the time if you work on Linux or any modern kernel. You probably also want a more theoretical and general book too, of course.

<comment><USER8> And you can witness the glory of swtch() and the original "You are not expected to understand this" comment.

Highly recommended.

</comment></comment><comment><USER9> "Operating Systems: Three Easy Pieces" is a more modern book to OS concepts, explained with real source code as opposed to algorithms, and is regularly updated.

[http://pages.cs.wisc.edu/~remzi/OSTEP/](http://pages.cs.wisc.edu/~remzi/OSTEP/)

(Online version above, you can also order a print copy).

<comment><USER10> I took this class (although not from him). it's a good textbook and was pretty much sufficient.

</comment></comment><comment><USER11> If you are still interested in FreeBSD, "The Design and Implementation of the Freebsd Operating System" is a fantastic book. It's a great guide in how a UNIX system and especially FreeBSD works and written by some of FreeBSD's most respected developers.

<comment> non-entity I actually did get a copy of that! Unfortunately, I only have the ebook at the moment, which I find difficult to read

<comment><USER12> If you want something more hands on you can try "FreeBSD Device Drivers: A Guide for the Intrepid" by Joseph Kong. It is much easier to read, and my guess is you ran into it while making those BSD patches.

He also has a short book on building rootkits for BSD, which is a very fun read and demonstrates how things like system calls work.

If you want a guide to reading actual kernel code, here is a great resource explaining where some of the foundational pieces are. With a tool like cscope, reading the FreeBSD kernel is actually pretty easy.

[http://www.watson.org/~robert/freebsd/reading/](http://www.watson.org/~robert/freebsd/reading/)

Hope that helps!

</comment></comment></comment><comment><USER13> If you want to understand Linux systems programming, I've never found anything better than the book _The Linux Programming Interface_. For Linux internals, _Linux Kernel Development_ by Robert Love is good. For something more removed from current systems, _The Design of the UNIX Operating System_ and _Operating Systems: Design and Implementation_ are good.

</comment><comment><USER14> Not _very_ Linux specific but Tanenbaum's _" Modern Operating Systems"_ is an excellent source for OS theory and very readable as well. It's an expensive book, but I found it worth its money.

<comment><USER15> Seconded. I would add "Structured Computer Organization", as a preamble book. I read both as if they were novels - they really are interesting and written in such a way that can hook you just as a good story

</comment></comment><comment><USER16> Operating Systems: Three Easy Pieces [1]

XinuOS [2]

MIT's xv6 OS [3]

[1] [http://pages.cs.wisc.edu/~remzi/OSTEP/](http://pages.cs.wisc.edu/~remzi/OSTEP/)

[2] [https://xinu.cs.purdue.edu/](https://xinu.cs.purdue.edu/)

[3] [https://github.com/mit-pdos/xv6-public](https://github.com/mit- pdos/xv6-public)

<comment><USER17> +1 for xv6. The commentary book is really well written, and is a straightforward introduction to a lot of important ideas.

[https://pdos.csail.mit.edu/6.828/2014/xv6/book- rev8.pdf](https://pdos.csail.mit.edu/6.828/2014/xv6/book-rev8.pdf)

</comment></comment><comment><USER18> OSTEP is the best book on Operating systems easily. [http://pages.cs.wisc.edu/~remzi/OSTEP/](http://pages.cs.wisc.edu/~remzi/OSTEP/)

<comment><USER19> Not sure if it is the best book overall, but I used it for one of my college classes and I think it is a great book for learning much more about operating systems. And it is available for free online so my college student wallet loved it!

</comment></comment><comment><USER20> Not Linux, but I'd recommend this lab from MIT.

[https://pdos.csail.mit.edu/6.828/2014/overview.html](https://pdos.csail.mit.edu/6.828/2014/overview.html)

I completed this one, but they may have more recent versions. They give you some initial code that you need to complete, as well as tests to check your work before going to the next step. It covers a lot of material. It takes some patience and it's not for complete beginners but it's a great fun project. Not sure, but I'd say it took me 1 or 2 weeks full time.

</comment><comment><USER21> I recommend starting with something smaller.

Here's something I put together some years ago - it's about 5,000 lines of code and supports a number of key Unix features. There's a PDF of lecture notes accompanying it.

[http://adelaideos.sourceforge.net/](http://adelaideos.sourceforge.net/)

Once you've gone through this you'll be in a better position to tackle the Linux kernel and books on OS architecture. My personal favorite is "The Design of the UNIX Operating System" by Maurice J. Bach. It's an old book but explains things well, and I believe it was one of the references Linux Torvalds relied on when creating Linux.

</comment><comment><USER22> Years ago in school, I decided to figure out how the Linux kernel works. So I've downloaded the sources and open them with a notepad++. After an hour I've figured out that I have no idea what am I reading in the code, close the notepad++ and never open it back. Buy the sources are still awaiting in some old directory at my home's PC :)

</comment><comment><USER23> This looks like a good start: [https://0xax.gitbooks.io/linux- insides/content/index.html](https://0xax.gitbooks.io/linux- insides/content/index.html)

</comment><comment><USER24> The O'Reilly book Understanding The Linux Kernel is fantastic. Starts you off in the deep end with memory management/addressing and fans out from there. Notably excludes networking, as the book is long enough without it.

<comment><USER25> Any suggestions for Linux networking? I have Understanding Linux Network Internals but it's a bit dated.

<comment><USER24> I think it comes down to just understanding IP and TCP in general. Which I don't really (IP) or do sorta (TCP).

As far as kernel architecture, the rest of the system can work with the assumption "writes go into a buffer, reads come from a buffer, and the networking subsystem does the right thing in between", plus some stuff about DMA optimizations.

</comment></comment></comment><comment><USER26> [https://github.com/s-matyukevich/raspberry-pi- os](https://github.com/s-matyukevich/raspberry-pi-os)

Matyukevich page on GitHub has lessons with accompanying code to develop a Linux OS for raspberry Pi. The author goes into kernel, processor initialization, interrupt handling, scheduler, implementing syscall, and virtual memory.

I like his approach for these reasons 1\. Minimal workable code, 2\. Points you to entry function in linux repo 3\. Line commentary of those entry function

Note that it's also a somewhat short read as he focuses on the practical implementations instead of the theory.

I myself was hoping that the author continue developing the chapters on file system, drivers, and networking but seemed to have been on hiatus.

</comment><comment><USER27> Linux Kernel Development By Robert Love and Operating Systems Concepts by Silberschatz are great resources.

</comment><comment><USER28> This book has been immensely helpful to me, it's "Writing an OS in Rust" [https://os.phil-opp.com/](https://os.phil-opp.com/) and I can't recommend it enough.

Also, Redox OS is an OS written in Rust and has a ton of wonderful documentation on it that's been helping me get into systems development: [https://www.redox-os.org/docs/](https://www.redox-os.org/docs/)

</comment><comment><USER29> For the Linux kernel, there is an excellent book called "Linux Core Kernel Commentary" (which is based off the SVR4 Lion's Commentary book), which has the source of the kernel (an old version) printed out and then a commentary on all the bits (code and data structures). It was an eye opener for me.

As for OS theory, "Operating Systems Vade Macum" and "Operating Systems Design and Implementation" are my favorites.

</comment><comment><USER30> A classic: The Design of the UNIX Operating System by Maurice Bach. [https://www.amazon.com/Design-UNIX-Operating- System/dp/01320...](https://www.amazon.com/Design-UNIX-Operating- System/dp/0132017997)

I learned more about OS from this book than any academic material used at the university and gave me a good foundational handle on *NIX's.

</comment><comment><USER31> Join us in /r/kernel. As a mod I'm always helping folks with questions like this, and our community frequently beats me to the punch.

<comment> non-entity Didn't know that existed. Will definitely check it out.

</comment></comment><comment><USER32> The Linux kernel is unfortunately pretty far from UNIX nowadays but if you want to understand the basic concepts behind UNIX I think Lions' Commentary on UNIX is still the best resource out there:

[https://www.amazon.com/Lions-Commentary-Unix- John/dp/1573980...](https://www.amazon.com/Lions-Commentary-Unix- John/dp/1573980137)

</comment><comment><USER33> Zhao Jiong released this excellent work in English not too long ago: [http://www.oldlinux.org/download/ECLK-5.0-WithCover.pdf](http://www.oldlinux.org/download/ECLK-5.0-WithCover.pdf) If you know Chinese it was originally done in that language, details are on [http://www.oldlinux.org/](http://www.oldlinux.org/) It heavily comments / explains everything in the linux kernel v0.12. Quite some time ago, but I agree with the assessment on its relevance for an "inductive understanding". The growth since then is largely in new subsystems, new drivers, but maybe not so much in core theory...

If you're more into the core theory and less about Linux in particular, a fun project-oriented approach to get more familiar with the basics would be to get a little development board for an ARM CPU and try to make a multi-tasking (real-time) OS for it from scratch.

<comment><USER34> Thanks, I haven't this one, it seems interesting.

</comment></comment><comment><USER35> I've semi-seriously been considering starting my own OS project to circumvent some of the IO latency issues with VR and AR applications.

The big metric is motion-to-photon latency, which is a measure (in milliseconds) of how quickly a user's head motion is reflected on the display. Anything greater than ~10ms is considered insufficient, as you've missed a whole frame of rendering for VR headsets that run at 90hz refresh rate.

Current XR applications need to wait for the USB or Bluetooth stack to return the head pose before modifying the view, then wait for the graphics stack to reflect that change. There are many ring transitions along the way. This leads to an absolute minimum motion-to-photon latency that can't be broken just because of OS overhead. This eats into your frame budget, as every ms of context-switching out of the IO drivers leaves you fewer ms to render a frame in time to get it back into the kernel for context-switching back to the graphics driver.

</comment><comment><USER36> For learning the basics about how an operating system works, I recommend Remzi and Andrea Arpaci-Dusseau's "Three Easy Pieces", which is freely available at [http://pages.cs.wisc.edu/~remzi/OSTEP/](http://pages.cs.wisc.edu/~remzi/OSTEP/). Both Remzi and Andrea Arpaci-Dusseau are great writers and are prolific researchers in the areas of operating systems and file systems, and their work is top-notch.

For learning the basics about how the kernels of Unix-like systems are designed, I recommend the following books:

\- The Design of the UNIX Operating System (1986) by Maurice Bach. You could also compare this book to the source code of Version 7 Unix, which is freely available under a BSD-style license. Many of the basic design principles of Unix-like operating systems have not changed dramatically over the years, and so studying the design and implementation of older, smaller implementations of Unix-like operating systems can aid with figuring out the code base of Linux, which is huge.

\- UNIX Internals: The New Frontier (1996) by Uresh Vahalia. This discusses some more modern features that weren't part of AT&T Unix.

\- The Design and Implementation of the FreeBSD Operating System (2014) by Marshall Kirk McKusick et al. I've met "Kirk" in person a few times at USENIX FAST conferences; he's not only a great writer, but he is also very friendly and approachable. This is a well-written text describing FreeBSD 11, including modern features such as ZFS.

There are some good Linux kernel books (I'm partial to "Linux Kernel Development" by Robert Love, which helped me out nine years ago when I was writing a device-mapper device), but what I've found during my experience working with the Linux kernel is that the Linux kernel changes much faster than the books describing it get updated. Nevertheless, the best thing to do is to get a solid understanding of how Unix works, then get a basic understanding of the Linux kernel, then start diving into the source code, which is ultimately the definitive documentation of the Linux kernel.

When I was working on that device-mapper device, I found that perusing the Linux kernel mailing list was excellent for learning about how relevant portions of the kernel changed over time.

Good luck and have fun! I haven't done much kernel-level work lately, but I miss working at that level!

</comment><comment><USER37> The Minix OS was designed as an academic, teaching OS: [https://www.minix3.org/](https://www.minix3.org/)

There is a textbook that goes along with it: [https://www.pearson.com/us/higher- education/program/Tanenbau...](https://www.pearson.com/us/higher- education/program/Tanenbaum-Operating-Systems-Design-and-Implementation-3rd- Edition/PGM228096.html)

The author famously had interactions with Linus Torvalds when he was starting Linux!

And, Minix is __inside __every modern Intel CPU ([https://www.zdnet.com/article/minix-intels-hidden-in-chip- op...](https://www.zdnet.com/article/minix-intels-hidden-in-chip-operating- system/)).

<comment> non-entity Oh yes, I've been curious about breaking into the Minix in intel chips.

</comment></comment><comment><USER38> If you're looking for something specific to Linux, that is less about the kernel, and more about how the whole operating system is strung together, I can highly recommend: [http://www.linuxfromscratch.org/](http://www.linuxfromscratch.org/)

</comment><comment><USER8> The old standards.

Stevens to start. If you don't already know Stevens' thoroughly first, I wouldn't go deeper. The kernel is not separable from the userland-facing API.

I'm not convinced you actually need to / want to understand actual internals. But assuming you do, next is Tanenbaum (maybe start with the newer "Modern" book).

"Operating Systems: Three Easy Pieces" also seems well regarded, although I haven't read it myself.

The FreeBSD book is great, if you're specifically interested in BSD. There's also a video lecture you can buy on McKusick's site. I had the pleasure of an on-site presentation and it is absolutely fantastic.

</comment><comment><USER39> This is a somewhat different take, and quite possibly impractical.

The main thing I've done over the years to get reasonably 'good' at the Linux kernel (and quite a few other things) has been to read every technical article appearing in [https://www.lwn.net/](https://www.lwn.net/), every week, in full, even if I didn't really understand the material. I've been doing that since 1997, and it's paid off quite well.

Also: please buy a membership, even though you don't have to. It's well worth supporting.

</comment><comment><USER40> I've been making my way through this course [https://classroom.udacity.com/courses/ud923](https://classroom.udacity.com/courses/ud923)

It's about 9 years old I think so might be too out of date for you, but I've been really enjoying it. I think they have an advanced version of the course as well if you're already familiar with the basics

<comment><USER41> That appears to be the Intro to Operating Systems course from the Georgia Tech OMSCS program.

Udacity also hosts the GT Advanced Operating Systems course: [https://www.udacity.com/course/advanced-operating-systems-- u...](https://www.udacity.com/course/advanced-operating-systems--ud189). AOS is not a "how to" course, but rather a tour of the challenges operating systems may face (distribution, fault tolerance, scale) and of academic papers attempting to solve them (mostly from the 90s and 00s). As such, the material might not be a good fit for the original post, but this commenter found it fascinating.

The reading list is here: [https://www.udacity.com/wiki/ud156-readings](https://www.udacity.com/wiki/ud156-readings)

</comment></comment><comment><USER42> These are some good resources on the things that go into creating your own kernel:

      - http://www.brokenthorn.com/Resources/OSDevIndex.html
      - http://www.osdever.net/bkerndev/Docs/intro.htm

For learning about Linux specifically:

      - https://0xax.gitbooks.io/linux-insides/content/index.html

</comment><comment> no-dr-onboard In school we used Tannenbaum's MINIX book (Operating Systems Design and Implementation). It was generally a pretty good introduction to "OS Theory", directly references the source code in the back. Useful for a desk book if you're constantly elbows deep in writing syscalls.

</comment><comment><USER43> In the past I did this: [http://eudyptula-challenge.org/](http://eudyptula- challenge.org/)

Not sure if they are accepting new applicants, but you can try. Also, you can try to find the assignments online.

<comment><USER44> The assignments are online in various places. Here is a copy I made from someone's Github (not linking GitHub b/c it had answers too): [https://pastebin.com/cUXuHcDc](https://pastebin.com/cUXuHcDc)

</comment></comment><comment><USER45> From my previous comment - [https://news.ycombinator.com/item?id=18451260](https://news.ycombinator.com/item?id=18451260)

</comment><comment><USER46> From not too long ago: [https://news.ycombinator.com/item?id=19225268](https://news.ycombinator.com/item?id=19225268)

</comment><comment><USER47> In addition to OSTEP, Linux From Scratch [1] is also a good resource.

[1] [http://www.linuxfromscratch.org](http://www.linuxfromscratch.org)

</comment><comment><USER48> [http://inst.eecs.berkeley.edu/~cs162/sp19/](http://inst.eecs.berkeley.edu/~cs162/sp19/)

</comment><comment><USER49> [https://github.com/0xAX/linux-insides](https://github.com/0xAX/linux-insides)

</comment><comment><USER50> Beyond the exceptional mentioned texts dedicated to the subject; my favorite resources are:

lwn.net

github.com/torvalds/linux

</comment><comment><USER51> Start with something smaller, such as lk.</comment>


Ask HN: People who rejected a FAANG offer. Why? - <AUTHOR> Or put another way, what was more appealing about a non-FAANG offer that made you reject a FAANG offer? <comment><USER2> I've rejected multiple interview requests for FAANG jobs because I do not wish to study strange-toy-programming-quiz-tricks for three months just to be able to say I work at Hooley-Soft.

<comment><USER3> Have you tried giving it a shot without studying beforehand to see how it is yourself?

<comment><USER4> I tried that for an SRE position at Google. You can get through the first couple interviews, but if you don't study the _exact_ material they give you-- and they'll give you a lot--and aren't able to recite it exactly as described, you get turned away pretty quickly.

They feed you all this nonsense about wanting to see creative solutions in the interviews, but if you study the material they give you, all the answers are there. It might as well be a certification.

"You did well initially but didn't study the material we gave you. Re-apply in 1.5 years." No thanks; I was just keeping my options open and don't have time for all that nonsense. If you have a decent offer, tell me; otherwise, stop wasting my time. And, of course, what happens 1.5 years later? They call and email you incessantly asking you to re-apply.

<comment><USER5> Can you post the list of study material?

</comment></comment></comment></comment><comment><USER6> I rejected a Google offer in 2012, and they've been bothering me ever since. They seemed really surprised that I didn't feel all warm and fuzzy about their lowball offer. I ended up going to Blizzard and then SpaceX, and I don't regret anything. My big thing is that while the money might have been better at a FAANG, I've done a lot more interesting work I can talk about (and some I can't). But at Google, and Microsoft where I worked for about 10 years, there are frankly so many people (many quite talented) that it's hard to get something interesting or useful to work on. It's just a lot of politics and empire building. Also, the mobility especially for women is pretty bad. We always joked the best way to get a raise was to leave and come back. It was funny because it was so true.

<comment><USER7> > there are frankly so many people (many quite talented) that it's hard to get > something interesting or useful to work on.

I always say it's like picking stocks. The chances that you've found a truly great opportunity on a project at Google that hasn't been found by 5 more qualified people aren't nothing, but they're pretty small.

<comment><USER6> While that's true, and I get the feeling this is coming from the perspective of someone already working at google and shopping around for new projects, this actually brings up another point I hate in particular about Google's process.

When you're interviewing you are interviewing for a role but not with a team. So you don't know what you will be working on, and it seemed like when coming in, new hires always seem to get to be put on whatever. I talked to a couple of the managers who wanted to work with me, and it felt like these projects were kind of silly. Talking with people on the inside, it seems like the way to find something useful is to get in, then transfer to a team you want. But why would a qualified person want to go through all this craziness when they can just interview at a different company and meet the people they will work with, and talk about what they are working on? Like a normal job? It seems like Google's process is highly skewed toward "great for Google, shaft the candidate." Which they can do, because everyone wants to work for Google, so why not only hire kool-aid drinkers?

And just to bring up the empire building again, if every great opportunity has so many people wanting it, it tends to lead to toxic behavior and culture due to unhealthy competition.

</comment></comment></comment><comment><USER8> Fulfilling life. Half the salary of FAANG is more than enough to live on, and I love my job, and I love my family, and I get plenty of time for both.

Alternative would be twice the salary, retire early, I guess. And I think if I did that, I'd be spending my time the exact same way I am right now. Only after a few hellish years having my soul drained by making people click on more ads.

<comment><USER9> Wow, it seems people who don't work at FAANGs have a weird impression of what goes on at them... Autonomy is not guaranteed at a startup, just as lack of autonomy is not guaranteed at a FAANG.

<comment><USER8> I have plenty of friends at FAANGs, and I've visited plenty of times. I am not sure why you'd presume I have a weird impression.

I'm also not sure why you're presume autonomy brings fulfillment. All job offers I had gave plenty of autonomy. I'd take genuine community over autonomy in most cases. People are social creatures.

I'm also not sure why you'd presume I work at a startup. I work at a large and ancient not-for-profit many times older than FAANG. I work on an internal startup-like project. What they historically did was super-labor-intensive, and you can do it better integrate digital technology and big data. People can do more, better. I'm trying to do that.

To be clear: (1) I don't have an NDA, and I can openly talk about what I do. I don't need to keep secrets. (2) My work will help people in significant and visible ways. (3) It's technically fascinating (and I can publish research papers on anything I do).

That's not an abstract Google/startup/tech transform the world: ("At BILLINGLY DOT COM, we are changing lives by TRANSFORMING HOW PEOPLE DO BILLING"), but having direct, positive impact, potentially on millions of lives. Right now, I'm doing pilots, so it's impacting a few individuals, but if those are successful, the organization can take this to scale.

There are surely such projects at FAANG too, occasionally, but they're rare, often highly political, often incompetently done, and it's not the offer I had.

</comment></comment></comment><comment><USER10> I'm in my mid 30s, and have had a healthy, fun career as a comp. engineer. I thought of applying to one of the FAANG companies recently, but the idea of whiteboard/hackerrank Code tests was a severe turn off, and put that idea away. I don't have the energy to put into preparing for weeks like a code monkey to give a shot at a test and mostly never using them at the job. I'd rather spend more time playing with my baby boy.

I'm not saying learning data structures and algorithms are useless. Only that testing for them using timed tests with toy puzzles is silly and not for everyone.

</comment><comment><USER11> I was offered significantly more money from a non-FAANG than any of my (three) FAANG offers were willing to give me, with better liquidity. At the end of the day, that's all I cared about. I'm very happy with where I landed, and have no plans to leave in the near future. My present company has an academic engineering culture that prioritizes outcomes over process, while encouraging curiosity and autonomy.

<comment><USER12> Was it from a unicorn?

<comment><USER13> Hard to get good liquidity from a unicorn. The description sounds like trading to me.

<comment><USER14> Yes, the GP username is also a hint!

<comment><USER11> No it isn't, I just like calculus/analysis. I don't work in trading, my job is developing simulations in operations research.

</comment></comment></comment></comment></comment><comment><USER15> I never got an offer but I've rejected interviews (won't take part of the code challenges culture).

At one of them I had a strong recommendation for a specific project which allowed me to skip the initial screening but I declined in an interview as I didn't want to move to the USA and they weren't remote friendly.

I am quite happy where I am in my career and industry ! Feel that I have a huge impact in the business at a global scale. Starting over as just-another- sw-engineer-in-a-large-company doesn't seem attractive.

<comment><USER16> Similar story here, I would receive emails/phone calls every half-year inviting me to interview with Google.

My initial question was always "Can I stay here in Scotland?" to which the reply was "You will move to either Ireland, or Switzerland." At that point I said "Please stop calling/emailing me"

At some point they stopped calling/mailing, looking over my email archive it has been ten years+ since I've heard from anybody at Google.

<comment><USER17> I had a similar thing about a decade or so ago. Thought I might be able to wrangle a (part) remote gig with them. No dice.

There's no way I was going to commute to London every day. Nightmare. They kept coming back to me though.

In my case I think it was more likely a combination of poor process, and a clueless recruiter, rather than a deliberate retargeting of me that resulted in the ongoing calls/emails.

</comment></comment></comment><comment><USER18> I realized FAANGs are not the holy grail of tech, and that it's possible to get excellent TC while working on interesting projects with more autonomy and possibly better WLB as well at smaller, faster organizations. Granted, a downside is that most equity is unlikely to be meaningful (except for the lucky), but I found a far more exciting path. Though I could certainly see the appeal in the more innovative arms of some companies (like Alphabet's X) to do bleeding edge work...

I also have ethical issues with spending my existence working for some of them (Facebook is the grimiest offender by far), and the line blurs quite a bit into questionable territory for some of these companies.

</comment><comment><USER19> I've turned down feelers (politely) because I like where I live and don't want to be forced to move to $expensiveBigCity.

</comment><comment><USER20> (I'd rather share this anonymously, so this is a throwaway account.)

I'm a software engineer and technical manager with just shy of 20 years of experience. I've never received a job offer from a FAANG company, but I've said "no thanks" to Google recruiters a number of times over the last ~10 years, and several years ago I did an initial remote interview with Netflix before declining their offer of an onsite interview. Here are my own personal reasons against working for a FAANG company:

Facebook/Google: Surveillance capitalism at its finest; working for one of these would be using my talents to further the greatest privacy-violating data collection regime (with virtually zero government oversight) the world has ever seen. I'd rather give up programming entirely and work retail.

Amazon: less objectionable than FB/Goog, but only slightly. They treat their non-engineer employees like complete crap, and I've heard that the quality of life even on the engineering side is very much dependent on the org you end up in and your manager.

Apple: Their privacy game is much stronger than any of the above, and I use their devices daily. Their walled gardens and locked-down devices mean they are a far cry from the hacker-friendly Apple I remember as a kid in the 80s. Their culture also seems very secretive and rather cult-like which leaves a bad taste in my mouth (though maybe this is improving since Steve's death?)

Netflix: by far the least objectionable FAANG, many fewer privacy concerns and the product is good IMHO. However, they have a hard policy against remote work, at least for the team(s) I was looking at, and I don't want to live anywhere near SF Bay. This is the only reason I ended the interview process with them.

Aside from all of the above and even if you disagree with everything I said, FAANGs are just so huge that the incremental value you add by joining is likely to be tiny. I've spent the last ~10 years working at startups to mid- size companies (from 5 up to 200 people) and I feel that my work is much more impactful than it could ever be at a FAANG.

<comment><USER21> I'm not sure why this is getting downvoted since it seems to be a pretty genuine and relevant post.

<comment><USER11> It's probably getting downvoted because it's not actually relevant. I'm not sure why there are so many comments here talking about refusing to interview or "rejecting" recruiter advances. The OP was very specific in wanting to hear from people who rejected an _offer_ for another offer.

Instead a lot of these comments are from people who seem to have an axe to grind against the companies themselves. It's fine to have ethical or practical disagreements with FAANG companies, but it's not relevant to the OP's question about why someone would go as far as receiving an offer and then decline to accept it.

<comment><USER22> To be fair, the narrow question would have been far more compelling 10-15 years ago, when these companies were both less-huge and had largely unvarnished reputations as being the best large companies for software developers to work. 10 years ago, sour grapes would have been a lot more obvious and valid answers would have been quite interesting. Today, not wanting to work for Google or Facebook is rather mundane.

</comment></comment></comment></comment><comment><USER23> On one occasion, Google offered a reasonable amount of money to do something I had basically no interest in. On another occasion, Google gave me a 4-day exploding offer for way below market. So I didn't work for Google so far

</comment><comment><USER24> Didn't want to sell my soul to a heartless company that exploits workers. I'd rather be part of the solution and build products and services for unions, non-profits, and movement organizing.

</comment><comment><USER25> Rejected a Google offer in 2008 to join a company called DataDomain. I had just finished my PhD and my co-advisor was in Data Domain so it wasn't a very hard choice. At that point, Google also didn't assign you a team before joining so the choice was between join an unknown team at Google VS go and work for someone I knew and respected. Also, Google paid well but wasn't as crazy as things today.

Learned a lot at DataDomain as it scaled from a reasonably small team to a large enterprise and eventually got acquired. Financially though, would have fared much better in Google (assuming I had stuck around).

</comment><comment><USER26> I didn't even apply. I sort of wish I did. I don't know that I would have even gotten an interview, but my god... my current job just seems so boring.

</comment><comment><USER27> I turned down interviewing with one of this companies because they didn't want to disclose the salary, not even a salary range.

Also, the interviewer explained that I had 3 weeks for studying a list of computer science concepts for the interview.

I already have a Master's degree and I am working, I don't need to study to proof my profesionality.

</comment><comment><USER28> I'm loving the comments of this thread. Gives me more courage that cool tech and software does exists out of FAANG World.

I'm just a new grad soon to be graduating whereas my peers are always talking about landing positions at FAANG while missing out on other opportunities. I think its important to look at things from different perspective.

<comment><USER29> I never worked for FAANG, but I have been bothered by their recruiters for years and I politely reject them every time. To be fair, not just FAANG, other household name BigCorp too.

Reason being is I used to work for a small company and had to do everything myself, then I moved to a bigger company and I did a fraction of stuff compared to my previous company, but for a lot more money. It was a bit of a shock and it got me thinking. So I figured the larger the company, the more corporate the environment is; you are more restricted as to what and how you are allowed to do. I don't like that, so I moved back to a smaller company for less money and never even considered working at a BigCorp again.

You can make a case for both, depending on what you prefer. If you like prestige/cult/money and want that stamp on your CV then work your way up to FAANG. If you're curious, easily bored and like implementing solutions your way then work at a small company. Cool tech does exist outside FAANG. I can look back at each company I worked for and proudly talk about solutions I developed and implemented, most of which are still in use today and can be seen or used by customers/users. I'm not sure if I could have such an impact on a business working at BigCorp.

Sure you don't get paid as much and you can't brag to your friends and family you work for FAANG, but it feels really good when you suggest a solution to your manager/boss, then research it, build it and implement in and they TRUST you fully to do all that. You're part of the entire process and there's a certain satisfaction with having such freedom.

<comment><USER28> I totally agree with you. Plus the satisfaction you get when owning the entire process. I totally agree working at a large corp you are like the tiny part of process

</comment></comment></comment><comment><USER30> Honestly, slightly more appealing kool-aid.

I interned at one and turned down a grad-job offer. I never was proud of myself for working at a big-tech company that is infamously awful to its more replaceable staff. The pay cut (at least at this point in my life) is worth not feeling like I'm actively contributing to a global evil.

</comment><comment><USER31> It was straight out of college. I'd have had to move across the country, and wanted to stay closer to home. The startup that I joined instead also appeared to be offering work more aligned with my technical interests, though in practice it didn't work out that way and I left after less than a year.

</comment><comment><USER32> Everytime I was interviewing, some other company out bet FAANG offer. There are many companies in the bay area, which can match/give you more than FAANG. Fb being the only exception. But then, you are working 2X more for 1.5X the salary. So its not really worth it.

</comment><comment><USER33> I did not get an offer, but I did reject interviews to work in the support team at Google Cloud. Moral implications aside, the job required me to move from Brazil to the US, and I had no interest in living in the US.

No regrets about my decision.

</comment><comment><USER34> I wanted to be a digital nomad instead.

Edit: it was Twitter though, so not FAANG, although I'd also interned at Amazon previously.

<comment><AUTHOR> Being a digital nomad is something that I've considered as well. How does your experience as a digital nomad compare to a FAANG offer financially and in terms of growth?

<comment><USER34> It is not good financially or for career growth (although it doesn't have to be terrible if you keep it short and keep your skills up)

If you enjoy working full-time at a FAANG, or if you've never tried it, you shouldn't pass up the opportunity.

I already knew I didn't enjoy it, and I didn't want to waste any more of my youth which I felt I had already wasted studying hard at a nerdy male- dominated college with non-stop internships. So that's why I did what I did.

</comment></comment></comment><comment><USER35> \- I don't like to work at a public company, it's very stressful to work in an environment where every day you are waking up and reading stock market news.

\- I had already worked at Microsoft for 2 years in 2009-2011, there isn't much value addition to your resume after you have worked at one such big tech company.

\- I was hired by the Android TV team, which is not a product I use and not my area of interest. It was mildly disappointing that they only gave one option in the team matching process.

\- I am not a very suburban person, and commuting from SF to MV every day seems appealing at the outset but gets very cumbersome after a few weeks. Google doesn't tell you whether the team will be located in SF / MV when you interview, you are allowed to choose when you go through team matching.

In summary, I had made enough money already and my resume had Microsoft brand (though it was not a FANG in 2009). Touchwood, I have been able to get through all the recruiter filters whenever I have applied and I think that's the only advantage of working at FANG.</comment>


Nestle Embeds GPS Trackers in KitKat Bars for Promotional Campaign - <AUTHOR> http://betabeat.com/2012/09/misguided-nestle-campaign-assumes-men-like-gps-in-their-kitkats/

<comment><USER2> > But Americans need not worry about a candy company following them home. GPS > devices will only be embedded in the wrappers of British candies like > "KitKat 4 Finger, KitKat Chunky, Aero Peppermint Medium and Yorkie Milk," > whatever those are.

The US has strong regulations about packaging non-food inside of food. Putting a lollipop on the end of a flashing LED[1] is fine, but crossing the border with toys embedded in chocolate will get you detained by customs and fined $2,500[2].

[1] [http://www.windycitynovelties.com/11433p/light-up-candy- ring...](http://www.windycitynovelties.com/11433p/light-up-candy-rings.html) [2] [http://ca.news.yahoo.com/blogs/daily-buzz/seattle-pair- detai...](http://ca.news.yahoo.com/blogs/daily-buzz/seattle-pair-detained- smuggling-canadian-contraband-kinder-surprise-204518469.html)

<comment><USER3> I like the Aero bars, and every time I visit Canada with someone I hunt down a Kinder Surprise so we can eat something that's illegal in the US :)

</comment></comment><comment><USER4> This sounds just bizarre. Even beyond the privacy issue, there is a PR debacle waiting to happen (man found eating Kit Kat with mistress).

<comment><USER5> I'd assume that the device will be obvious once the wrapper is open. Such a tracker requires batteries with enough power to driver a transmitter, and a reasonably efficient antenna. That means it has to be a significant physical size.

If fact, it wouldn't surprise me if the only thing in the wrapper is a golden coloured tracker emblazoned with the word's "you won!", and the winners task is to wait for Nestle to come to him and deliver the 10000 pounds and missing chocolate bar (tough luck if you had the munchies).

If fact, it would be quite fun evading the "Nestle police" for as long as possible, leading them on a merry chase across the UK.

</comment></comment><comment><USER6> The best outcome of this campaign would be for someone to identify the tracker chip without opening the chocolate bar. Then post it somewhere else in the world.

</comment><comment><USER7> We just lure all the Nestle employees into a dark alley... and then we hit 'em up for more candy bars!</comment>


How to use the hacker news effectively (what is your idea?) - <AUTHOR> How to use this site for business or for some other purpose ? <comment><USER2> I would say the format of hacker news is pretty clean and it gets out of the way. So you can at the least check the front page every couple of hours to find cool new stuff.

You could also post stuff and ask questions. Among the answers to the questions asked on here you can find a lot of gems and you can usually expect the same if you ask one yourself.

<comment><AUTHOR> Great explanation .... thank you JobsandJobs

</comment></comment><comment><USER3> I join irc://irc.freenode.org/#hn ;-)</comment>


Show HN: Cashcash – Accounting app for visual people - <AUTHOR> https://cashcashpro.com/#/ <comment><AUTHOR> Hi everyone,

Here is yet another personal finance app with, I hope, a fresh perspective on how to handle cash flow.

In my daily life, I have to use well-crafted data visualization software to analyze and search in our production log. Back home, I always felt frustrated to not be able to do the same with my personal financial data so I created this app (my first one so far).

I've also added extra stuff into it like spreadsheet import, automation rules and budgeting capabilities. I think now is a good time to get other feedbacks than my own brain.

Don't hesitate to ask me anything,

Cheers,</comment>