Btrfs RAID 5/6 Code Found to Be Very Unsafe and Will Likely Require a Rewrite - <AUTHOR> http://phoronix.com/scan.php?page=news_item&px=Btrfs-RAID-56-Is-Bad <comment><USER2> Bryan Cantrill expresses this[1] in a manner I certainly agree with. Software that is designed/ intended to be reliable should not go through large periods of instability only to be written off as "prepubescence".

[1][http://www.youtube.com/watch?v=79fvDDPaIoY&t=18m28s](http://www.youtube.com/watch?v=79fvDDPaIoY&t=18m28s)

<comment><USER3> Bryan Cantrill is such a great storyteller. "My therapist says I need to talk about the firmware revision numbers, just to let it out so it doesn't corrode me." He then shares a story about a firmware bug where the polarity would be reversed of the disk read/write head. :)

<comment><USER4> Completely agree.

I often just search for videos by him and just listen to him present a topic or talk about anything. I learned about things normally I wouldn't go and read about otherwise -- history of Sun, ZFS, dtrace, containerization (Joyent, zones legacy, KVM, comparison with Docker).

In that respect he is like David Beazley (of Python fame). I could just listen to David talk all day and not get tired of it.

</comment></comment></comment><comment><USER5> It is amazing how stubborn Linux people are. They constantly refuse and reject superior engineering that was done outside of their realm. (yeah I was pointing at ZFS, but that would be just one of many)

<comment><USER6> This is Linux's legacy. Rejecting already solved problems only to try reinvent the wheel their way and often their wheel is broken and horribly engineered if engineered at all. See btrfs v zfs, kqueue vs their current poll of the month method, dtrace vs system tap, the list goes on a mile long.

It's so frustrating to watch this. To see all these projects being developed on Linux where it's easier for others to simply reimplement the idea from scratch than try and remove all the Linux only code from them.

The Linux world only thinks about itself. In a spectacularly selfish way.

<comment><USER7> Linux's 'legacy' is a policy of accepting code before it is known to be perfect. I don't know anyone who runs BTRFS and I certainly would not do so myself. Getting accepted as part of the kernel code is not an imprimatur of some sort.

I'd much rather this than the *BSD policies.

Oh and talking about FreeBSD, it's hardly bug free. Look at these Golang issues and tell me it doesn't indicate something wrong in the FreeBSD kernel threads/fork/exec code.

1\. [https://golang.org/issue/16136](https://golang.org/issue/16136) 2\. [https://golang.org/issue/15658](https://golang.org/issue/15658) 3\. [https://golang.org/issue/16396](https://golang.org/issue/16396)

<comment><USER6> I'm not sure it indicates anything. They are all incredibly vague and no one seems to have any idea what's going on.

</comment></comment></comment></comment><comment><USER8> I've never really understood the rationale behind integrating RAID into the filesystem. It seems like a giant layering violation, and unnecessary given that mdadm already exists and has many years of testing behind it.

The btrfs FAQ [1] says that "unlike MD-RAID, btrfs knows what blocks are actually used by data/metadata, and can use that information in a rebuild/recovery situation", but is that really a good enough reason to reimplement the entire RAID subsystem from scratch? And couldn't TRIM/discard provide the same benefits?

For me, the clincher is that mdadm has a very stable, well-defined on-disk format, which is a huge bonus if you hit a bug or make a mistake. I once almost lost a personal RAID10 array by recklessly trying to add an extra drive with no backup, and without fully understanding what I was doing. I was able to recover all of my data by hacking together a Python script to reassemble all of the blocks in the correct order. I can't imagine how much effort that would have taken if I had to build something that understood the full details of btrfs's data structures.

[1]: [https://btrfs.wiki.kernel.org/index.php/FAQ#Case_study:_btrf...](https://btrfs.wiki.kernel.org/index.php/FAQ#Case_study:_btrfs- raid_5.2F6_versus_MD-RAID_5.2F6)

<comment><USER9> Integrating RAID into the filesystem solves a lot of problems, one big one being data integrity. When you have a dumb RAID layer unaware of the data on disk it will just replicate or shard it between disks, when you request that data back and you have replicas it has no way to verify that data is correct, if one replica is bad but the other is good your filesystem has no way to correct for it since it just sees one big block device.

Moving storage pooling functionality to the filesystem is the right call for anything with server workloads, ZFS's self-healing functionality wouldn't work without it (and this is precisely why I use FreeNAS instead Unraid or other alternatives, I care about the long-term integrity of my data, especially my VMWare images and photo collection).

<comment><USER8> Enforcing data integrity is great, but I don't see why it has to happen at the filesystem level. There are any number of ways you could solve it while still keeping the RAID functionality at the level of unstructured block devices. I can think of three just off the top of my head:

#1: Do checksumming at the block level. Within each block, the RAID driver reserves a few bytes at the end for a checksum, and verifies them before returning data to the FS. (This would only work if the filesystem supports non-power-of-two block sizes.)

#2: Similar to #1, but pack the checksums for multiple data blocks into a separate dedicated checksum block. (This adds some extra read latency in the worst case, but caching could mitigate it.)

#3: Let the filesystem handle checksumming, but extend the block device API to provide feedback to the kernel if the checksum is invalid. I think you only really need two API calls -- one which says "read this block as fast as possible and I'll verify it", and one which says "the block you returned looks bad, try to reconstruct it and give me all available candidates".

<comment><USER10> > Do checksumming at the block level.

This doesn't handle phantom or misdirected writes. You can't really do that unless you keep the checksums elsewhere, and specifically in the place where you intend to reference the data.

> Let the filesystem handle checksumming, but extend the block device API to > provide feedback to the kernel if the checksum is invalid.

I think by the time you get this API right, it's going to seem like just as bad a layering violation.

ZFS has a coherent set of layers built into it. They're just not the ones that existed before it. But I don't think those previous layers were designed with many of the important failure modes of real hardware devices in mind. That's understandable given when they were designed, but we've come a long way since then in the understanding of those failure modes and our expectations of system integrity.

</comment></comment></comment></comment><comment><USER11> Btrfs people have been telling everyone that RAID5/6 code is incomplete for years now, so hopefully people weren't depending on it for important things. The wiki link is broken, here's the page [https://btrfs.wiki.kernel.org/index.php/RAID56](https://btrfs.wiki.kernel.org/index.php/RAID56) and you can see in the history that on the very first version of the page, in the first sentence, "the recovery and rebuild code isn't bulletproof or complete yet."

<comment><AUTHOR> This warning has only been added recently

[https://btrfs.wiki.kernel.org/index.php?title=RAID56&diff=30...](https://btrfs.wiki.kernel.org/index.php?title=RAID56&diff=30445&oldid=29769)

<comment><USER11> Oh, so that's ~7 months of people maybe using it for important things. That could be really bad then.

<comment><AUTHOR> Well that line about it stabilizing over the next couple of kernel releases was added in February 2015, so when I installed Ubuntu 16.04 on my NAS I thought things probably had calmed down sufficiently.

As I learned the hard way, I was wrong and that warning is very much justified.

[https://btrfs.wiki.kernel.org/index.php?title=RAID56&diff=29...](https://btrfs.wiki.kernel.org/index.php?title=RAID56&diff=29294&oldid=25681)

<comment><USER11> Well, I have a problem with a btrfs RAID1 (which isn't really RAID1 but whatever) where if I read certain files my kernel crashes. So that's fun too.

<comment><USER12> Better that than the unrecoverable dataloss I suffered (it trashed both disks... and also panicked the kernel). In fact, they were so screwed it panicked the system on every boot until I booted with a non-Btrfs-supporting kernel and blanked them with dd...

</comment></comment></comment></comment></comment></comment><comment><USER13> Fancy file systems are for people who think their life isn't exciting enough.

<comment><USER14> Depends how you're defining "fancy". I use ZFS because I don't want any drama, and it has delivered that (Linux md was more complicated to set up, to the extent that I lost data). But it's a mature, solid filesystem in a way that BTRFS isn't yet.

<comment><USER15> Downvoted because I frequently hear this sentiment from _both sides_ of the argument but rarely is anything more than an anecdote offered to support it. Feels more and more like a mud slinging competition instead of an assessment of merit.

<comment><USER12> Many people use both filesystems, and you'll see many claims of how both work really well and that the user hasn't had any problems. The problem with these people's experiences is not that they are untrue, but that they are primarily from people who haven't had hardware failure/glitches, and who have never had their systems run the failure-case codepaths. Everything's fine and dandy up until the point you lose all your data.

I've run Btrfs on many systems since just after it started to be usable, and written software with Btrfs-specific support which hammers it (and LVM) like nothing else creating and destroying tens of thousands of transient snapshots. I've also now run ZFS on several systems, admittedly over a smaller timeframe (3 years vs 7-8ish).

I've had Btrfs totally trash a RAID1 mirror from a transient SATA cable connector glitch. On this test system, I had half the disk using Btrfs, half using mdraid/LVM. The mdraid half recovered and resynced transparently as soon as I reseated the connector; no service interruption or dataloss. Btrfs ceased to function, and on reboot toasted both mirrors resulting in total unrecoverable dataloss and repeated kernel panics. That's been fixed a while, but right here we're seeing the same thing. The failure codepaths, which are of critical importance, are untested and buggy. And even non-failure codepaths are still bad. Take the snapshotting case above, I had to take the system offline and do a full manual rebalance every 18 hours. The time from fresh new filesystem to read-only unbalanced disaster was just 18 hours when thrashed continuously, at most using 10% of the total space. And lastly, the performance of some things such as fsync are truly abysmal, to the extent that we had to use "eatmydata" to completely disable it for apt/dpkg operations! When under heavy parallel workloads, it could take many tens of minutes or hours(!) to complete writes which ext4 would complete in a minute or so.

I've yet to experience any problems at all with ZFS. Now that might have been luck on my part, but it might also be down to better design and quality of implementation. It's certainly been battle tested in high end installations. That's not to say that Btrfs doesn't have some neat features; it does a few things ZFS doesn't, like rebalancing data over its devices while ZFS only does that on write. But Btrfs has let me down badly every time I've used it in anger, and those few neat features don't make up for its lack of robustness-- the primary purpose of the filesystem is to reliably store data, and it fails at that.

I don't like to see "mud slinging", since such fanboyism is unobjective and uninformed. I've reached my opinion based upon several years of practical intensive use of Btrfs for various things, the most demanding of which was repeated whole-archive rebuilds of the whole of Debian when I was maintaining the Debian build tools, and wrote btrfs snapshot support specifically for them, doing over 30 parallel builds on a single system using independent snapshots per build with over 20000 snapshots per run, creating and destroying several per second. The experiment was disastrous, and showed Btrfs to be unsuitable for such intensive workloads. When your filesystem is guaranteed be turned read-only at some unpredictable and unknown point in the future, you can't rely on it. Regular rebalancing mitigates but doesn't solve this, and has a terrible performance impact. Not dataloss per se (unless it makes you lose writes when it turns read-only), but it's a serious design or implementation flaw. I did all this testing and adding of Btrfs support to various tools because I had high hopes for its potential; unfortunately they exposed serious shortcomings, many of which exist to this day. Today I'm using ZFS, not because of any irrational prejudice against Btrfs, but because Btrfs has never managed to deliver a robust and well tested filesystem!

<comment><USER16> I'm not having nearly your experience but I just want to say I'm agreeing 100% to you conclusions based on my experience. We ran at Uni a Hadoop Cluster that had disks slowly dying (some bad sectors every few days, but otherwise fine) and lacked the money to replace them. We replaced ext4 with ZFS (no raid, just plain zpools with failmode=continue) and ZFS ran mostly fine and scrubs kept the metadata sane. Never had data loss (HDFS has it's own replication and checksumming, we just need sane metadata for Hadoop to run and intermediate MapReduce outputs where send to directories with zfs set copies=2) and we only replaced the botched disks that had longer scrub times or couldn't survive a scrub. I'm still surprised how ZFS managed to pull that off. Only bugs I've found where related to ZoL at this time but could be worked around.

btrfs switched to readonly as fast as ext4 (which is probably the correct thing to do) but was useless for this problem.

On new hardware with new enterprise disks we choose btrfs and we had a painful tour of crashes, data corruption, metadata corruption (undeletable files), deadlocks until kernel 4.4 where things got a little bit better. Here disks and server where enterprise class and fully working. Just btrfs bugs. Also no RAID. I'm not doing that anymore so I don't know if any new bugs appeared but the whole experience will keep me from ever using btrfs. This was ~2years ago and you could easily find lot's of slides Fujitsu or Suse that btrfs is stable and you can use it (around kernel 3.13-3.16).

It's probably fine for your notebook or even your backup HDD but don't think you can stress it without experiencing pain (be it corruption, hangups or dataloss) or just abysmal performance.

That beeing said ZFS on Linux is also a far cry from rock solid but I'm optimistic that they flesh out the problems and tackle them in a solid way.

As a Linux fanboy for years this gave me some solid appreciation for Solaris engineering.

</comment></comment></comment></comment></comment><comment><USER17> If anyone wants to help out with bcachefs, Reed-Solomon might be a fun place to jump in. And bcachefs so far has a better track record with people's data than btrfs :)

</comment><comment><USER18> Does this impact Synology's Hybrid RAID running on Btrfs? Btrfs is nowadays the default fs on Synology boxes.

<comment><USER19> No, there is no impact to Synology's Hybrid RAID running on btrfs. Synology uses mdadm/lvm for RAID.

</comment></comment><comment><USER20> Is it possible to install Linux on HAMMER?

<comment><USER6> You mean is it possible to port and run hammer on Linux?

Anything is possible but the work required would be a big challenge.</comment></comment>


Honesty is the new game changer - <AUTHOR> https://medium.com/@steviebuckley/honesty-is-the-new-game-changer-5342a32a40b9 <comment><USER2> My main disconnect with the hiring process these days is the near-ubiquitous use of code challenges that just keep getting more and more time-consuming.

A friend of mine asked me if I wanted a pointer to a job that wanted me to spend _six hours_ of my free time greenfielding a toy. I laughed and said no thanks. Code challenges are sucking all the liquidity out of the job market and are probably responsible for a good bit of the salary hikes I've been seeing lately.

One challenge asked me to do everything in Vanilla Javascript.

The other asked me to make a commit to their production codebase. Even if you offer to pay me, your offer is never going to be truly enough, and I'm not looking for side work.

Code challenges feel incredibly lazy to me. Your organization sucks at hiring, so let's throw all the obstacles we can at applicants and see who makes it through. I wanted the job that asked me to do the production commit bad enough that I made my best effort. I ran into a snag and raised an issue like I was asked to. No reply, and the door slammed shut in my face a few days later when the time frame elapsed.

I now refuse to do a more-than-trivial code challenge while I still have a job. It's just not worth it.

<comment><USER3> I partially disagree.

In hiring you need some way to assess whether or not applicants can actually produce useful work. You can't get that from a resume, you can't get that from education background, and you can't get that from talking to people.

Now, you could just ask applicants for a sample of their work (preferably open source) and evaluate them on that.

Coding challenges probably get done instead of that because it is easier for companies to evaluate a single coding challenge than to evaluate a broad range of unique work from different individuals.

As an engineer the employment process gives you an opportunity to evaluate companies in the same way that they have an opportunity to evaluate you. If a company has a sub-par hiring process then you cross them off the list and move on.

<comment><USER4> When looking for a software developer you're not just looking for someone who can implement a solution to a problem. Daily duties involve discussing issues and impediments, navigating organizational bureaucracy, managing code bases via source control repositories, troubleshooting environment issues, playing nice with other developer's code, updating ticketing systems, providing systems documentation, and writing maintainable code.

A coding challenge addresses none of that, it just asks the candidate to solve a problem. Often coding challenges are sent to candidates and there's no verification that they're even the one completing the work. I've spoken with many candidates who submitted coding samples or code challenges and were completely unable to discuss the code. Often times I wondered if their recruiter supplied the code and not the candidate because of their confusion surrounding the code.

I've had more luck accessing candidates by asking simple requests like "tell me about your daily duties at your last/present job" or "oh you've language x, what's your least favorite part of developing with it?" If the candidate can't discuss simple things with mild depth or command then they're probably not a good fit.

<comment><USER5> >I've had more luck accessing candidates by asking simple requests like "tell me about your daily duties at your last/present job" or "oh you've language x, what's your least favorite part of developing with it?" If the candidate can't discuss simple things with mild depth or command then they're probably not a good fit.

This is what I do. If an interviewer can be bullshitted doing it this way, they don't need to be interviewing. Non-technical managers really need to defer this sort of thing to technical people.

<comment><USER3> > If an interviewer can be bullshitted doing it this way, they don't need to > be interviewing

There is no magical BS detector. You can't judge a person's skill without seeing them work.

Everyone I have met who thinks they are smart enough to not get duped by smooth talkers gets duped all the time.

<comment><USER5> >There is no magical BS detector. You can't judge a person's skill without seeing them work.

You are conflating the issues. A tech top can detect BS, but no one can judge a person's skill without seeing them work _for several months._

Think of it this way. How long do you think it would take an MD to detect a fraud MD talking shop? Not long. How long does it take a military veteran to spot a fraud? Seconds. You have to be technical to spot the issues though. If something pops up that makes you think they are bullshitting, you have to drill in on it.

<comment><USER3> Those examples are not the same though.

You can have someone who has an extensive education in Computer Science and can talk about it all day long very articulately but has no implementation skills.

You might be able to catch someone with no comp-sci background by talking to them but not someone who has an advanced comp sci education.

The doctor example is relevant though.

Doctors don't get hired based on talking. They have to do residencies, get certified, and demonstrate ability before even applying for a job. Same reason. Someone could learn how to repeat everything in the text books without being able to do the procedures, which require _practice_ to master.

<comment><USER5> To use our Doctor simile, you wouldn't ask about anatomy (or only briefly), you ask about diagnosing and treating ailments; that's what you are paying the person to do.

For technical interviews, you don't ask them about Computer Science things (or only briefly), you ask them about real-world software development; that's what you are paying them to do. The stuff they really don't go over in CompSci classes.

What was the project you were most proud of and why?

What was the hardest project for you and why?

What are some of the ways you've connected to a database / datastore (if applicable)?

What were some database design decisions you've grappled over? What designs were you considering?

What's the most impressive code you've seen written by someone else and why?

Tell me about how you organized classes in a particular project and why you did it that way.

Describe a way you've organized your classes and wish you had done it another way, and why.

How do you make sure if your software has a problem, you know about it and can figure it out quickly?

What's the hardest software you've written and why?

How do you handle disagreements with a colleague? Can you site an example of a disagreement and each person's reasoning?

Notice almost all of those examples asked "why." That's the most important part. If someone isn't up to par, they will do one of two things:

1\. Answer Briefly. 2\. Babble on to the point it's obvious they don't know what they are talking about.

#1 of course can be because the person is shy or intimidated or an extreme introvert, but not likely. Because you're talking about a subject they should feel extremely comfortable about, even introverts will come out of their shell.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> I fail to see how yet another job board is going to solve what is essentially a human monkey-brain problem. Just how is this company going to get all players to be honest? I just had an interview at what I thought was a good company where I was--get this--accused of being _long-winded_. Yes, I was accused of _talking too much_ in an interview. How can a website solve this type of problem, among all the others?

You're going to discover that companies don't _want_ to be honest during the interview process. They'll lie to your face and tell you Everything is Awesome™. You're going to discover that applicants are dishonest as well. Everyone's lying to signal to the other that it should be a match.

<comment><USER7> Somewhat related - I've always wished that HN allowed comments on the Jobs posts (not who is hiring, which is a normal thread, but the YC backed company posts). I think there are a number of people in the community who have had interactions with these companies and could provide insight to those thinking about applying to them.

<comment><USER8> That's an unnecessary and pointless shitshow waiting to happen.

People with insider info and trust cannot share that info. It would involve cutting their own throat and burning bridges. Any nice things said about the company would get pissed on as "Well, of course, you would say that because this is HN/you must be a shill/whatever." Inevitably, someone would feel the need to say something terrible so as to be edgy/honest/whatever.

There is just absolutely no good that can come of that.

<comment><USER7> I disagree. Looking at the Who Is Hiring threads, most of the (minimal, sure) comments are either questions about the position or company, with some commenting poorly on the company.

<comment><USER8> That seems like a different use case to me.

</comment></comment></comment></comment></comment><comment><USER9> There is the old saying "Honesty is the key to success. If you can fake that, you are golden." :)

<comment><USER10> You put it in a nutshell perfectly.

</comment></comment><comment><USER11> Love the domain name!

And some of the screenshots reveal an approach to hiring which is sorely lacking. At this stage, every single LinkedIn email and cold outreach email from a recruiter gets marked as spam.

Not a single one seems to understand that hiring should be treating people as… well, people with lives and desires. Not just a resource to plug into a hole; not a fool to be tricked into changing jobs so the recruiter can hit their targets.

Any innovation in this space is welcome and I hope there are enough "decent" companies to make it a successful business.

fwiw, part of the advantage of going freelance was side-stepping all the bullshit hiring assumptions.

<comment><USER4> Recruiters are part of the trend of outsource everything. Most recruiters are just salesmen on par with the kid who knocks on your front door wanting to sell you magazines or AT&T DSL; they'll tell you any lie you want to hear to get the sale.

The sooner everyone starts refusing to deal with recruiters, the better off we'll all be.

<comment><USER12> Well, recruiters do produce /some/ value.

For employers, recruiters will do the dirty business the employers don't want to know about. For example, ignoring PII legislation to get a larger pool of contacts. Advertising highly paid nonexistent jobs to harvest CVs. Lying to candidates and pressuring them if they're wavering.

These convert into more hires.

For employees, recruiters will bypass bullshit job applicant websites that make you re-type your CV into their agonisingly long forms. They'll free you from emotionally draining work like customising your CV and writing a thoughtful cover letter - or even to really know what an employer does before applying to them. Often they'll also tell you interview questions in advance, and some will help you cheat on take-home tests.

This converts into less effort needed per job offer.

I'm not saying I like them - clearly, I see big disadvantages - but there are reasons they've lasted as long as they have, as an industry.

<comment><USER4> > These convert into more hires.

And more fires. This is why you see more and more Contract-To-Hire and Temp- To-Hire positions being posted.

Recruiters provide bad or inappropriate candidates to the employer. When we go through recruiters we spend a lot of wasted time interviewing candidates that should have never been sent to us.

</comment></comment></comment></comment><comment><USER13> > Some of the excuses that I often come up against are worth sharing though...

I'd like to add one. Salary range is low (pre-revenue startup) so the company needs to make up for it in non-salary ways. And the company wants a chance to talk to the candidate and sell the position without having the candidate not even bother to apply.

With that said, I don't agree with that. I used to, when I was younger, but now I understand the importance of padding your retirement account and saving while you are young.

I eventually made my opportunity costs back and then some but it was the long difficult road and most people aren't so lucky.

<comment><USER14> What exactly is non-salary advantage of typical pre-revenue startup?

<comment><USER13> Usually:

\- Significant equity

\- Flexible hours

\- Usually less bureaucracy and politics

\- More opportunity to make a critical impact as an individual contributor

\- Usually a higher job title (senior becomes principal, principal becomes director or VP)

Those last two have been a huge help in my career. Early in my career I got a huge amount of experience that would take much much longer at a larger company. And the job title thing adds a bit of a "fake it until you make it" thing. But it came at the price of a lot more stress and less work-life balance.

Whether it is worth it is up to you. I used to think it was but I'm not so sure anymore.

Edit: Added a note on job title.

<comment><USER15> _\- Flexible hours_

Only to the extent that you get do decide exactly which 14 hours you want to work each day (except when there is a 2am emergency of course)

<comment><USER13> Some (small) percentage of people would actually prefer to work 14 hours starting at 11 AM than 8 hours starting at 7:30 AM. But...

I learned something at my first few startups... if you are working 14 hours consistently:

1\. You will get burnt out quickly

2\. Probably only 6 of those hours are productive.

If you tell employees to work 14 hours they will record 14 hours in their time sheet but they are probably not actually "working" effectively.

Now as the CTO of a startup I consider it my personal mission to grow and improve the product by making sure the team workers more efficient not longer hours. I get the same 6 productive hours out of the team and much happier employees. In fact, it could be a coincidence, but this go around we ship features faster and have less bugs then when I was at a company that demanded 12+ hours.

With that said, I also grew a lot as a person and manager during that time. And learned what is effective and what is ineffective uses of time. If you aren't able to answer the question of what will have the largest impact and what is the best use of time than it is easy to end up with people working 80 hour weeks.

</comment></comment></comment></comment></comment><comment><AUTHOR> Hi folks. Author and chap behind [https://honest.work/](https://honest.work/) here. Happy to answer any questions and stuff.

<comment><USER16> Hello hello. I gathered through your series of subtle clues that you're based in the UK. Are you going to be focusing mostly on the British job market at launch?

<comment><AUTHOR> At launch, yes. We categorically intend to release a US focused version but we want to iron out a lot of the early stage kinks first.

</comment></comment></comment><comment><USER17> >I really hate the phrase "game changer". It's right up there in the TechBro lexicon alongside disrupt, growth hacking, hustle and burn the village

All those dumb Valley buzzwords were secretly the fault of bros all along?

Fantastic! I'd hate to have to do any actual introspection.</comment>


What does Dustin Moskovitz think of the Facebook movie? - <AUTHOR> http://www.quora.com/What-does-Dustin-Moskovitz-think-of-the-Facebook-movie <comment><USER2> The movie portrays Zuckerberg in quite a bad light. In real life, I think he's no worse than other people. In fact, he probably has more integrity than most.

There's a section in Sarah Lacy's book on Accel's investment in Facebook. Zuckerberg had almost finalised a deal with Don Graham of the Washington Post when Accel came in offering 2.5X as much. Lacy describes how this placed Zuckerberg in a moral dilemma, even though Accel was offering more money on better terms, he felt he had to honor his commitment to Graham. I don't think many people would give it a second thought, they'd take the money and run.

<comment><USER3> I think it's important to note that he still decided on Accel in the end

<comment><USER2> He did, but he went and discussed the situation with Graham first. Graham understood the moral bind that Zuckerberg was in and was impressed by how seriously he took his word. His response? "Mark, I release you from your moral dilemma."

</comment></comment></comment><comment><USER4> What is weird about the Facebook Movie trailer is that it seems to play out like 'Revenge of the Jocks'. Sorkin has all kinds of cutting one-liners in there directed at Zuckerberg, with the overarching frame an assault on the manners and character of the nouveau riche nerd.

For example:

[http://www.theaustralian.com.au/business/media/movie-wont- ex...](http://www.theaustralian.com.au/business/media/movie-wont-exactly-put- a-happy-face-on-zuckerberg/story-e6frg996-1225867479280)

      The film, set for release in October, starts on the night of February 4, 2004, when Zuckerberg, then 19, is dumped in a Harvard bar by his girlfriend, Erica.

      She tells him he will "go through life thinking that girls don't like you because you're a tech geek". The reality, she says, is rather more upsetting. "I want you to know, from the bottom of my heart, that that won't be true. It'll be because you're an asshole."

OUCH. That's particularly brutal because the implication is that there's no pot of gold at the end of the rainbow if you're a tech nerd. The standard "deal" offered by American society is that smart kids who are high school outcasts nevertheless end up on top after college, while the football captain who dunked them into garbage cans winds up pumping gas.

While Hollywood exaggerates this "deal" in movies like Just Friends (<http://www.imdb.com/title/tt0433400/>), it legitimately does reflect a real feature of American life. It's basically the concession that an otherwise anti-intellectual society makes to keep its roads running and internets-a- whirring so that it may continue worshipping Lebron James and Tom Brady on gameday.

But Sorkin's story is one in which huge VP bizdev jocks aren't just better looking, but also better people. Morality and popularity[1] during school years are no longer inversely correlated in this tale. The outsider is not just a social outcast, but the _bad guy_.

I'd wager this is a hard pill to swallow for many of the people on this forum. Zuckerberg built a legendary engineering operation and a multibillion dollar company only to find himself portrayed as an evil nerd who _deserves_ the harsh treatment he receives from women and society at large.

Screw that.

If I were Mark, I'd be sorely tempted to fund a movie biopic about Sorkin that puts him under the microscope and goes into his past as a crack cocaine addict (<http://en.wikipedia.org/wiki/Aaron_Sorkin#Personal_life>). Turnabout is only fair play.

[1] PG's essay on why they should inversely correlate is worthwhile reading; in short, high school popularity requires spending your time on zero sum activities:

<http://www.paulgraham.com/nerds.html>

<comment><USER5> > I'd wager this is a hard pill to swallow for many of the people on this > forum.

Not really. Didn't we go through this with Bill Gates already? Wealthiest man in the history of humanity, far more nerdly than Zuckerberg, and yet, roundly despised by much of the rest of society as an inept malicious nerd and even by his own demographic (this may be hard to remember these days since billions of dollars of philanthrophy has bought him a modicum of respect, but try to think back to the '90s).</comment></comment>


Bookshelf Porn - <AUTHOR> http://bookshelfporn.com/ <comment><USER2> Seeing this makes me a bit wistful.. it used to be if I walked into someone's home and didn't see shelves full of books, my impression of that person was forever changed.

Now, my home has no shelves full of books. All my (dead tree) books are in storage and I just have a kindle.

I still read as much as always, but I feel as if a certain degree of intellectualism is lost. No longer can someone look at my shelves and say "oh, I loved this book" and spark a nice literary discussion.

I should add, however, I am probably one of the few regular users of goodreads.com, though it is hardly a replacement (and I've only got one or two friends).

<comment><USER3> _Now, my home has no shelves full of books. All my (dead tree) books are in storage and I just have a kindle._

Which in its own way may lead some visitors to get an impression of your reading tastes too :-)

The majority of books I own either aren't on the Kindle or couldn't usefully be on the Kindle in its current form (most TASCHEN books, megasets like Modernist Cuisine, almost everything in large formats, atlases, Edward Tufte books, art and design annuals, anything with even the barest flare of graphic design..)

<comment><USER2> _Which in its own way may lead some visitors to get an impression of your reading tastes too :-)_

Oh I know, that is exactly what I meant. I fear the conclusions people might draw when the only reading material they see in my house are my girlfriend's Us Weeklys.

</comment></comment></comment><comment><USER4> I can't help but think every time I see a layout like this "oh boy another pinterest clone" and it's really too bad because I nearly instantly discard the idea no matter how cool it might be.

I realize the same cycle happened with digg, reddit, stack overflow, twitter, and facebook clones when they were the new hotness, but I really do wish designers and developers weren't so likely to take whatever the current fad is and apply it to their current idea.

</comment><comment><USER5> I'd love to know how this works: [http://bookshelfporn.com/post/26256181298/upstairs-in-the- th...](http://bookshelfporn.com/post/26256181298/upstairs-in-the-the-last- bookstore-los-angeles)

I presume this is either a) pressed together like stones in arches are (therefore rendering the books inaccessible), or b) held in by hidden shelving, wire, magnets or magic. :D

</comment><comment><USER6> Thanks for posting this! As a lover of books, these photos are great.

</comment><comment><USER7> Unfortunately this is not what I expected...

</comment><comment><USER8> See also: <http://www.lookshelves.com>

</comment><comment><USER9> How is this tumblr any different from just doing a google image search?

[https://www.google.com/search?q=bookshelf&hl=en&safe...](https://www.google.com/search?q=bookshelf&hl=en&safe=off&prmd=imvns&source=lnms&tbm=isch&sa=X&ei=mnI9UMa0LLH4igLq2IDQDQ&ved=0CAoQ_AUoAQ&biw=987&bih=706)

i don't see the point of it. I realize that there's a tumblr for everything, but i fail to see the value in this on hn.

<comment><USER10> The images are bigger on the tumblr :) ... and if this wasn't posted on Hacker News... I guess I wouldn't have googled "bookshelf" and I wouldn't have seen the images.. which is sad because I spent some pleasant and inspiring 10 minutes browsing the website and I've got some cool new ideas to apply to my bookcase...

Also... it's value on hn is that it also sparked a small discussion about having books/bookcases/bookshelves in your house or having a Kindle....... a google search would't have done that</comment></comment>


Lnav: The Log File Navigator - <AUTHOR> https://github.com/tstack/lnav <comment><AUTHOR> This is an impressive ncurses log viewer. It automatically and consistently highlights different fields, and can even group multi-line messages such as stack traces.

One neat feature I used the other day is the "go to next error" ability. Using warning levels (I think?) you can page through errors or warnings, skipping over routine output.</comment>


O-Ring Theory - <AUTHOR> https://en.wikipedia.org/wiki/O-ring_theory_of_economic_development <comment><USER2> In other words, poor countries are doomed to stay poor if their high-skilled labor is prone to emigrate.

So, perhaps not having all these visas for tech workers is the right thing to do, from a moral perspective?

On the other hand, so-called poor countries _could_ make themselves attractive with low taxes and low cost of living, attracting business and ultimately bringing back those brains.</comment>


Dinosaur-Killing Asteroid Could Have Caused 2 Years of Darkness - <AUTHOR> http://astrobiology.com/2017/08/dinosaur-killing-asteroid-could-have-caused-2-years-of-darkness.html <comment><USER2> > While the scientists think the new study gives a robust picture of how large > injections of soot into the atmosphere can affect the climate, they also > caution that the study has limitations.

> For example, the simulations were run in a model of modern-day Earth, not a > model representing what Earth looked like during the Cretaceous Period, when > the continents were in slightly different locations. The atmosphere 66 > million years ago also contained somewhat different concentrations of gases, > including higher levels of carbon dioxide.

> Additionally, the simulations did not try to account for volcanic eruptions > or sulfur released from the Earth's crust at the site of the asteroid > impact, which would have resulted in an increase in light-reflecting sulfate > aerosols in the atmosphere.

Additionally, the "world-class computer model" in which the simulations were run may or may not have been a Minecraft server.

<comment><USER3> How can they publish something so wrong?

Also: Get off my server

<comment><USER4> Before decrying how wrong it is, try estimating how much those effects matter.

</comment></comment></comment><comment><USER5> I think our present space technology research focus should be on these asteroids. Both how to deflect them (peacefully) and how to mine them for rare metals.

If I read this table correctly it seems that only eight days ago a fairly chunky rock passed within 60,000 miles of us:

[https://cneos.jpl.nasa.gov/ca/](https://cneos.jpl.nasa.gov/ca/)

<comment><USER6> Really cool link, thank you. I think you are referring to the 2017 QP1 asteroid that was closest to us on 14-Aug. the distance was about a sixth of the distance to the moon, or 38k miles. The speed was 24km/s and the diameter 37-83 meters. Just for fun I ran the Impact Earth calculator to see what would happed if such an asteroid hit Earth. Assuming the worst case scenario (iron meteorite, max diameter, perpendicular hit) the explosion would be equivalent to about 200 MT TNT, and the crater would have a 2 mile diameter and 2000 foot depth.

Edit: changed mph to km/s

[https://www.purdue.edu/impactearth/](https://www.purdue.edu/impactearth/)

<comment><USER7> > _The speed was 24mph_

No way. It was 24 km/second.

<comment><USER6> Good catch, thanks.

</comment></comment></comment></comment><comment><USER8> Graham Hancock dedicates at least 10% of his most recent book "Fingerprints of the Gods" to exploring the recent evidence for a cataclysmic meteor impact that catalyzed the precipitous drop in global temperatures we observe 12900 years ago at the "Younger Dryas Boundary." [https://en.wikipedia.org/wiki/Younger_Dryas#Impact_hypothesi...](https://en.wikipedia.org/wiki/Younger_Dryas#Impact_hypothesis)

He has some pretty out there theories but that section is pretty convincing.

<comment><USER9> My understanding is the duration does not really line up with an impact event also the starting periods don't seem to line up correctly. "In Western Europe and Greenland, the Younger Dryas is a well-defined synchronous cool period.[34] Cooling in the tropical North Atlantic may, however, have preceded it by a few hundred years; South America shows a less well-defined initiation but a sharp termination."

Which is what I find annoying about pet theories in books, you can make plenty of things seem convincing by leaving out other evidence.

</comment></comment><comment><USER10> Naively I would assume that if we really had experienced two years of darkness, avian species would not be here now.

<comment><USER11> I do have to wonder how plants could have rebounded from two years without being able to photosynthesize - you'd think lots of species wouldn't have viable seeds kicking around at that point. Which would make for some massive changes to the fossil record for plants. To say nothing of the animals that depend on them...

<comment><USER12> Lots of seeds remain viable for more than 2 years.

<comment><USER10> It would also have to remain viable in the sense that no animals starving in the darkness will try to eatthem, or fungus or bacteria running wild. I think you have to assume that are small burrowing ancestors ate pretty much every seed they could get their tiny paws on. We know that quite a few different organisms survived, and while some of them estivated or hibernated most of them presumably we had to live and eat in a roughly normal way.

<comment><USER13> Fungi and bacteria don't like the cold. There would be a lot more food to eat for survivors (chickens) as things that they competed with for food (T rex) died off.

</comment></comment></comment></comment></comment><comment><USER14> "A world without photosynthesis", "photosynthesis would still have been impossible for an entire year"

Clearly, photosynthesis did survive in some capacity. No? Life did not re- evolve photosynthesis in the last 66 million years.

<comment><USER15> Seeds can germinate after much longer than a year: [https://en.wikipedia.org/wiki/Oldest_viable_seed](https://en.wikipedia.org/wiki/Oldest_viable_seed)

<comment><USER11> "Some seeds can live for a long time before germination, while others can only survive for a short period after dispersal before they die."[1]

I wasn't suggesting all seeds would fail to germinate after two years, just that some would, and that change would be noticeable in the fossil record.

1: [https://en.wikipedia.org/wiki/Seed#Germination](https://en.wikipedia.org/wiki/Seed#Germination)

<comment><USER15> Old seeds had much higher germination rate according to the linked wikipedia article. In addition, cold storage increases the life of seeds. I doubt you would see much of anything in the fossil record.

</comment></comment></comment></comment><comment><USER16> Fascinating is the feedback loop recovering after the impact. So basically, all that is needed to bring the sood from the atmosphere down is a long enough cooling window, binding ice to the dirt.

</comment><comment><USER17> This may be a dumb question, but were all the dinosaurs reptilian and avian? Were there no mammalian dinosaurs pre-66M years ago? like rodents?

<comment><USER18> Disclaimer: this is proving hard from me to explain via text without the use of visualization.

Mammals did not evolve from dinosaurs. Mammals are synapsids, a branch of tetrapods. Dinosaurs come from a different branch of tetrapods entirely, called sauropsids. (There is a third branch of tetrapods, amphibia.)

Sauropsida is a really diverse branch containing both extinct and extant species. Look at the tree at the bottom of the page here [0]. A fork of interest is Sauria, which splits into lepidosauromorphs (including modern lizards & snakes) and archosauromorphs. Archosauromorphs have many branches as well (one of note being turtles), with one that traces far enough to get to archosaurs [1]. Modern crocodiles come from a branch of archosaurs. Dinosauria are another branch of archosaurs, and birds are descendants of dinosauria. They are the only extant species of dinosauria.

The confusion comes because many of these animals co-existed at the same time, some become extinct, others evolved and change and form new branches (but still belong to the parent group). The word 'Dinosaur' itself is also misapplied by the public - it's not all of the animals that lived 65+ million years ago. It's a very specific group of related animals and all of their descendants.

TLDR: Mammals didn't evolve from dinosaurs, but from a different branch further up the chain that dinosaurs also belong to. Those mammalian ancestors did co-exist with dinosaurs. Like mammals, modern reptiles didn't evolve from dinosaurs, but from ancestors that co-existed with dinosaurs. Reptiles and dinosaurs are more closely related than mammals and dinosaurs. Birds did evolve from dinosaurs, and are the only living dinosaurs today.

[0] [https://en.wikipedia.org/wiki/Sauropsida](https://en.wikipedia.org/wiki/Sauropsida) [1] [https://en.wikipedia.org/wiki/Archosaur](https://en.wikipedia.org/wiki/Archosaur)

<comment><USER17> Thank you for the thoughtful answer.

</comment></comment></comment><comment><USER19> I'd be curious how any animals survived.</comment>


"The next big thing that will dethrone Google..." - <AUTHOR>

Does anyone know where to find the article that said the next big thing that will dethrone Google will be started by Google employees?  I'm trying to put together a blog, and I can't find this for reference. <comment><USER2> There is one by Cringely...

[http://www.pbs.org/cringely/pulpit/2007/pulpit_20070524_0021...](http://www.pbs.org/cringely/pulpit/2007/pulpit_20070524_002134.html)

<comment><AUTHOR> Thank you</comment></comment>


Is Submitting Your Own Blog Articles Acceptable at HN? - <AUTHOR>

I'm not new to the blogging scene, but it's been just over a month since I've started taking it seriously as a medium for building contacts and getting to know great people. You can probably see from my profile that right now I have nothing interesting on my blog, but I plan to use it as a sort of scrapbook or diary where I can write down some of the really insightful things that I am told/read/discover. Is submitting your own blog posts over and over to HN considered "bad behavior"?
You see, I mostly use the web for programming related activities. I get all my news from websites like HN. I rarely discover great links/articles on my own, and I don't think HN readers will be even slightly interested in mailing list debates about memory management issues in Objective-C. But what I can do is submit links to my own blog. Sometimes there are issues I wish I could discuss with people, and I find my blog is a great way to discuss those issues and keep a record for later reference. Also, being very honest about it all, I sometimes like to see some traffic in my blog stats, and some comments on my posts. If none of my posts are of any value to anyone, I think the HN community will do a great job of not letting it reach the homepage. And, just so you'd know, I have no ads on my blog and the sudden onslaught of traffic when one of my posts reaches the homepage actually poses a risk of me being kicked off of my webhost's server (cheap hosting is not cheap).
So, HN, is shameless self promotion discouraged here or is it okay to post links to your own blog/website? <comment><USER2> This has been answered before: It's ok assuming the content is relevant.

</comment><comment><USER3> I asked the same question here : <http://news.ycombinator.com/item?id=445455></comment>