Treat it as finished - <AUTHOR> http://joel.is/post/4308658019/treat-it-as-finished <comment><USER2> It's finished when you have a disaster and recovery plan. If you don't have everything being backed up in some way, you're just running programs and waiting for them to die, or for accidental mistakes to be made and data to be lost.

At the beginning, this can be simple as a 3 or 4 line script that compresses all your files, dumps the database, and copies it offsite, which you run on a daily/hourly basis.

<comment><USER3> Seriously, if you're not running backups and testing them, you're not done. It's unbelievably cheap insurance.

We started doing informal "keep this in each of our houses" off-site backups on simple burned discs from our first week, and bi-weekly dumps to Iron Mountain a few months later.

Spend some time making sure the backups actually work. We found errors in our source repository that we had to write tools to fix, but since we budgeted some time, it wasn't a big deal.

</comment></comment><comment><USER4> Joel, great post

I think we are all a little afraid of saying "it's finished" because we have no comeback when someone says "it sucks". I guess its that fear of rejection, whereas if we say its unfinished we can always claim the "yup we know, and the next release will be better".

Surely this is why the fad of slapping a Beta sign all over the products started, just to let people know that "it has some bugs an we know about them" What we really wanted to say was "It has some bugs, which we have no clue about, and we want you to help us discover them" :-)

As for the blogs, I agree, way too many , and any review is a good review.

Iqbal

</comment><comment><USER5> It's about creating something simple but decent for V0.1 - then letting your customer's metrics guide you on how to improve it

</comment><comment><USER6> Great article Joel. It is hard to get into that way of thinking when you're being a perfectionist over things, but you're right. Getting validation early and often will drive your featureset and refinements going forward.

</comment><comment><USER7> Good post Joel. It's really hard to have that mindset, but I think you're right. Promote and gain traction, then refine.

</comment><comment><USER8> Great post. Don't just ship early, promote early.</comment>


Google Analytics + Chance Timing Helps Reveal In Domain Name Cat & Mouse Game - <AUTHOR> http://dodomainer.com/2011/06/26/featured/google-analytics-privacy/

<comment><AUTHOR> should be "...Helps Reveal Identity..."</comment>


Using real names online 'leads to discrimination and harrassment' - <AUTHOR> https://nakedsecurity.sophos.com/2017/01/05/using-real-names-online-leads-to-discrimination-and-harrassment/ <comment><USER2> Submission on referenced piece "The Real Name Fallacy" from 2 days ago (over 300 comments):

[https://news.ycombinator.com/item?id=13313199](https://news.ycombinator.com/item?id=13313199)

</comment><comment><USER3> "Hey John Smith here's one weird trick your friend Sarah Jones found and now she earns $1733.33 working from home, Dr Phil hates her."

</comment><comment><USER4> Real names is really about targeted advertising. But we knew this right?</comment>


Grateful Dead Lyricist Robert Hunter Dies - <AUTHOR> So sad about the death of Grateful Dead lyricist Robert Hunter https://rollingstone.com/music/music-news/robert-hunter-grateful-dead-dead-889788/ . In so many ways, his were the lyrics of my youth--the very best part of it. "Let there be songs to fill the air." <comment><USER2> Let there be songs to fill the air.</comment>


Simple mobile-friendly online calculator in Javascript - <AUTHOR> http://calc8.com Nothing special, just something I knocked up quickly. I took inspiration from the stock calculator that comes with the HTC Hero (running Android 2.1) . <comment><USER2> Anything interesting about it?

(Apart from the fact that it allows decimal numbers like 127.0.0.1 and uses annyoing alert() boxes for error handling.)

<comment><AUTHOR> Yeah, alert boxes aren't great I admit. I was aiming for the bare minimum of functionality but I could perhaps add nicer popups using script. It detects changes in browser window size and tries to redraw the button pad accordingly; and so should work well on mobile screens too though there are some width issues on iPod Touch.

</comment></comment><comment><USER3> Similar named <http://www.calc5.com/> calculator. But could do much more

</comment><comment><USER4> You think it could be as least as powerful as the standard Windows 7 calculator - which is actually pretty nice.

</comment><comment><USER5> On the iPhone 4 iOs 4.1 it raises the keyboArd each time you press a button

</comment><comment><USER6> cool. 1+1=3</comment>


Why Do Rich People Get Even More Rich? - <AUTHOR> http://bodegasale.tumblr.com/post/15016540129/why-do-rich-people-get-even-more-richer

<comment><USER2> I think you're absolutely correct to sum it up as "opportunity". When people complained that "Occupy" lacked a coherent message, I would say "the Tea Party connotations that through taxes and regulation the government curtails the liberty part of "life, liberty, and the pursuit of happiness", Occupy feels their right to an opportunity to pursue happiness has been taken away."

I'd actually guess (and bust a guess) that issues 1 and 3 are bigger deals than 2, though I think they all apply. For the third, I'm reminded of the British documentary series "7 Up", where 7 year olds were asked what they wanted to be, and what they wanted in life. Every seven years the documentary creators caught up with the people to see where they were. The three young white boys from well off families even predicted their lives largely. From their universities to one even serving in government. It was kind of creepy.

</comment><comment><USER3> Compound interest is a strange omission from such an article.

<comment><USER2> I would consider that a form of passive income. Does that seem unreasonable?</comment></comment>


EU court holds Estonian website responsible for offensive user comments - <AUTHOR> http://rt.com/news/eu-delfi-offensive-comments-075/

<comment><USER2> The European Court for Human Rights is _not_ an EU court! It belongs to the Council of Europe!

Journalists will never get that right I think.</comment>


Shipping a non-trivial app in Swift - <AUTHOR> https://www.lucidchart.com/techblog/2015/03/03/build-lucidchart-ipad-swift/

<comment><USER2> In the latest Debug 61 podcast (@1:27:45), Andy Matuschak from Khan Academy mentions that he and someone else wrote 25k lines of Swift.

[http://www.imore.com/debug-61-andy-matuschak-and-khan- academ...](http://www.imore.com/debug-61-andy-matuschak-and-khan-academy)

The build times sounded a little painful but some of us have been there before:

[http://xkcd.com/303/](http://xkcd.com/303/)

Now with Swiftt 1.2, it's probably an easier choice.

<comment><AUTHOR> With a lot of (cool!) features like type inference, it's not hard to see why compiling swift is slow.

To me, incremental compilation is a bare minimum for a "real" language; it's nice to know 1.2 will have that.</comment></comment>


China-based hackers stole Indian national security information - <AUTHOR> http://www.nytimes.com/aponline/2010/04/06/business/AP-AS-China-Hacking-Report.html

<comment><USER2> It seems like china has found a new way to gain intellectual property and classified informations without hard work. I hope it won't hurt them at the long run, and even worst they don't care about the rest.

<comment><USER3> > I hope it won't hurt them at the long run

Why?

<comment><USER2> I think the world needs china , and nobody likes doing business with peoples you can't trust. The idea of closing a deal with a chinese company and feeling like, you may have your intellectual property stolen is not really exciting.

<comment><USER3> > I think the world needs china , and nobody likes doing business with peoples > you can't trust. The idea of closing a deal with a chinese company and > feeling like, you may have your intellectual property stolen is not really > exciting.

I don't know how much the world "needs" a China that behaves that way, but I'm pretty sure that it would be better off with a China that didn't.

How do you propose getting to the latter without causing some pain to the former? Remember - they're the former because they think that it works for them.

</comment></comment></comment></comment><comment><USER4> I don't understand this 'China+hacker' hype. Millions of hacks happen on the Internet everyday, what makes Chinese hackers so interesting?

<comment><USER5> Because they are always politically motivated, and are direct threats to freedom and liberty?</comment></comment>


First Optical Rectenna Converts Light Directly to DC Current - <AUTHOR> http://www.news.gatech.edu/2015/09/28/first-optical-rectenna-%E2%80%93-combined-rectifier-and-antenna-%E2%80%93-converts-light-dc-current <comment><USER2> RF rectennas apparently already have efficiencies up to 90%. According to this article, improving the fabrication process could lead to optical efficiencies up to 40%. That's double the efficency of any photovoltaic cell available today.

<comment><USER3> Not all variants.

[https://en.wikipedia.org/wiki/Solar_cell_efficiency#/media/F...](https://en.wikipedia.org/wiki/Solar_cell_efficiency#/media/File:PVeff\(rev150806\).jpg)

</comment></comment><comment><USER4> I thought they already had photo voltaic cells. I guess this is just a more efficient version of a device like that. It made it sound like they had invented a photo voltaic cell which they didn't.

<comment><USER5> This is huge because it has potential (hah) to become an ultra high speed photodetector, which have numerous uses in the industry.

Conventional photodetectors are essentially transparent photodiodes, and you have to pass a constant current through them for them to work, which generates heat and is generally a bad side effect.

Antennas on the other hand are completely passive and therefore immune to heat related issues. Also, unlike photodiodes and semiconductor devices, antennas are completely linear. Nonlinearities cause distortion that limit usable bandwidth.

</comment></comment><comment><USER6> Lots of dupes of this story recently, [https://news.ycombinator.com/item?id=10305226](https://news.ycombinator.com/item?id=10305226) seems to be the oldest but there are a few more.

<comment><USER7> In addition to jcr's thorough response, I'll add that we don't officially consider reposts to be dupes until the story has had significant attention here. Otherwise the randomness of /newest is too brutal. I wrote about this here if anyone is interested: [https://news.ycombinator.com/item?id=9828818](https://news.ycombinator.com/item?id=9828818).</comment></comment>


Ask HN: How can I identify which problems I should solve in my spare time? - <AUTHOR>

More specifically, web projects.
I'm a front-end web developer who's looking to brush up on back-end development. Stuff like Rails, Node.js, Ember.js and so on.
I've been trying to work out what to build next, but I have writers block and it's very frustrating. I'm not expecting anybody to give me an idea, but I was wondering if there's anything I could or should be doing to muster up some ideas. <comment><USER2> <http://www.sixmonthmba.com/2009/02/999ideas.html>

</comment><comment><USER3> <http://paulgraham.com/startupideas.html>

</comment><comment><USER4> Are you not experiencing any minor issues to solve for yourself?

</comment><comment><USER5> Whichever problem you have that you would pay the most to solve.

If it isn't a problem that you actually have, then it probably isn't worth solving. If it isn't a problem you actually have, it's likely going to be much harder to solve (even if it exists).

If it's a _real_ problem that you have, it's likely that somebody else has the problem too. After you identify the problem, it should be easy enough to ask around and see who else has the same problem.</comment>


Wow, so how dumb was spring ejection SD cards in hindsight? - <AUTHOR> I've literally lost two sd cards while moving about in the wild with important code on them in the last 6 months.
I think we spend too much time discussing how were succeeding and not enough time discussing how we are failing.
For example, its still impossible to paste an image from google search onto twitter... <comment><USER2> I agree. Although I'd only use them when forced to eg in a camera otherwise I'd use Dropbox or similar or a USB stick (has its own problems though!). Infact thinking of it almost all sneakernet media sucks: USB sticks bend, floppy disks just fail, external HDDs are ok but clunky, sd as you mentioned sucks, DVD rw is slow and easily damaged.

As for image pasting yeah the one that gets me is slacks inability to consume anything but am uploaded local file image. For example if I use the snippet tool in Windows I can't paste that directly into slack. Probably this one is up to chrome to fix.

</comment><comment><USER3> > For example, its still impossible to paste an image from google search onto > twitter...

I can do this on Windows 10 using Firefox.

I agree that SD card slots can suck -- I had one that the pins got snarled and trapped a card.</comment>


Asus officially announces Eee Box details - <AUTHOR> http://news.cnet.com/8301-10784_3-9957071-7.html?part=rss&tag=feed&subj=NewsBlog <comment><USER2> I'm excited to see small low power pc's become commodity. I think there will be a lot of cool open source software coming out to help nerds turn them into various digital appliances.

I know I can already do this with existing hardware (and software) given enough effort - but I think prepackaged units spur existing technologies to be made more friendly. See what happened with OSS router firmware for an example of alpha geeks polishing and packaging software for the beta geeks to run on low power commodity hardware.

</comment><comment><USER3> I'm still holding out for the EeePhone.

</comment><comment><USER4> Hello, next home server! :-D</comment>


BlueScreen - a screen saver for Windows - <AUTHOR> http://technet.microsoft.com/en-us/sysinternals/bb897558.aspx

<comment><USER2> I wouldn't recommend this in a corporate environment. I used to have this screensaver at work, until a helpful member of IT took it upon himself to _replace my PC's hard disk while I was away from my desk_ as he thought my machine was repeatedly crashing.

I changed screensaver after that...</comment>


Blue Green Deployment - <AUTHOR> http://martinfowler.com/bliki/BlueGreenDeployment.html

<comment><USER2> Double the hardware can be an expensive proposition for a startup, unless you're paying by the hour. I prefer red pill / blue pill deployment. Build an environment where you incrementally deploy and roll back new code. During a rollout some servers will run old code (blue), some servers will run new code (red). This is what IMVU and others have described in their continuous deployment processes, only with Matrix-friendly colors.

<comment><USER3> This seems like a problem unless your application is truly shared-nothing. Imagine what happens if the red code puts a piece of data in the shared cache, that is later retrieved by the blue code. The blue code gets confused, and crashes (or more likely, subtly corrups some other piece of data, which confuses the red code...)

Also, database schema issues, users wondering why a feature disappears when they reload the page, etc.

Either are definitely improvements over the usual dev / staging / production nightmare, though. (Sometimes I think people like that setup because their lives are boring and hacking on the production site at 5am on a Saturday adds much-needed excitement to their lives. I can't think of any technical reason for it, anyway.)

</comment></comment><comment><USER4> Any insight about gems, git workflows or any other interesting bit to implement it on Rails Apps?

</comment><comment><USER5> This is pretty much what EngineYard does.</comment>


A database built like an operating system: the architecture of FaunaDB - <AUTHOR> https://fauna.com/blog/a-database-built-like-an-operating-system <comment><USER2> There is an old engineering aphorism that proper database engines are complete operating systems implemented in user-space. This is true both literally and figuratively; most sophisticated database kernels treat the underlying operating system as a glorified device driver.

It is why they are both so complex to implement and so much fun to build.

</comment><comment><USER3> When I was first introduced to Oracle years ago, I was surprised at how much it acted like an operating system inside. Once connected to the shell you could find processes running, examine memory through data dictionary queries and so forth. Tuning it involved looking at disk I/O patterns, memory usage and making your loops tighter. In the SQL world that meant indexing & reviewing plans for joins carefully.

MySQL & Postgres carry some of those same attributes. Even though they are user processes and the OS controls the lower levels, you would typically deploy them on their own servers, and give them all the memory & disk I/O you could, because they were their own operating systems essentially.

It's not clear to me how building a database "like an operating system" is something new. What I did see new in there was a consistency checking tool. That is super important. Because in a distributed environment you're bound to have drift.

Also I'd like to see how they managed to get full ACID Compliance. Fast network interfaces are great but there is still some latency which means hiccups when multiple nodes try to update the same row.

<comment><USER4> I agree with you that it's not an original analogy, in fact I think you can safely say that most larger, performance-oriented systems end up looking more like an OS than not. There's a large overlap in the problems that need to be solved and the resources available to distributed systems, that I'd be more skeptical of an implementation that didn't lean heavily on the OS, or at least borrow the general patterns.

To answer your question about consistency, we have implemented a transaction resolution mechanism inspired by Calvin. The whitepaper contains a lot more detail, but the short answer is that we use a consistent, distributed transaction log to provide a global order of all read-write transactions, which are executed deterministically on the responsible data partitions. This allows FaunaDB to provide a guarantee of strict-serializability for read-write transactions. Read-only transactions are by default a bit more relaxed and provide a hard guarantee of serializability in order to avoid global coordination overhead.

</comment></comment><comment><USER5> Hey, they re-invented PICK!

[https://en.wikipedia.org/wiki/Pick_operating_system](https://en.wikipedia.org/wiki/Pick_operating_system)

(actually, I don't know much about this new system to be commenting on it - but whenever I see something with keywords like "database operating system" \- my mind instantly drifts to PICK, because that was the first system I used when I started my software development career)

<comment><USER6> I thought the same - started out using PICK as well. Wow, I thought I was one of the few on HN. Nice to know you!

I felt the same way when "NoSQL", document dbs, etc. came out. Ok, yeah, now we are distributed and massive scale with better tooling... but.. same/similar concepts.

<comment><USER7> Me too. What is this 'Cocoon'?

</comment></comment></comment><comment><USER8> Similarly I'd love to see an operating system that operated like a database.

Hate the arbitrary ascii-file nonsense. If the state of the operating system is in a proper ACID database it would be so nice. Tooling could make it just as easy as ascii files but under the hood it would be better.

<comment><USER9> Aren't filesystems databases? The "arbitrary ascii-file nonsense" is just like storing text fields in your database.

<comment><USER10> In a way, yes, definitely.

But one does not get transactions. I would love a filesystem with transactional semantics. Surprisingly often, applications use some form of relational database not because their data is such a good fit for that model, but because they want/need transactions. In theory, one could roll their own transaction layer, after all, that's what RDBMSs do. But I have a hunch that getting transactions to work correctly without totally killing performance is not exactly trivial.

<comment><USER11> NTFS definitely have transactions. I'm not sure about other file systems.

<comment><USER10> I had totally forgotten about that, you are right, NTFS does have a transaction API.

I think I remember the API being marked as deprecated, though. Or is my memory playing tricks on me again?

<comment><USER11> Yes, they deprecated it, claiming that developers didn't use it: [https://msdn.microsoft.com/en- us/library/windows/desktop/hh8...](https://msdn.microsoft.com/en- us/library/windows/desktop/hh802690\(v=vs.85\).aspx)

<comment><USER12> IMO, part of why developers didn't use it was the API design made using it too painful. They took most of the existing file APIs and added new *Transacted APIs alongside them, e.g. CreateFile begot CreateFileTransacted, etc. So to add filesystem transactions to your existing code, you had to change all the file API calls.

I think, if they had made transactions work with the existing file API–such as by storing the hTransaction in thread-local storage, with an API to get/set the transaction association of the current thread, and having the existing non-transactional APIs behave transactionally in that case–it might have saw more adoption by developers.

</comment></comment></comment></comment></comment></comment></comment><comment><AUTHOR> Hey everybody. Direct link to the PDF is: [https://fauna.com/pdf/FaunaDB- Technical-Whitepaper.pdf](https://fauna.com/pdf/FaunaDB-Technical- Whitepaper.pdf)

</comment><comment><USER13> Daytona from AT&T Labs built the database into the OS...I don't think it worked very well. The inverted model makes more sense. A process per query isn't very scalable.

<comment><USER14> Go on? Postgresql does process per session / query, and it scales ridiculously well, especially with the hilariously multi-core systems we have today.

Although it doesn't handle unbounded concurrency, there are techniques (like manual locking, or pgbouncer) to deal with this.

<comment><USER15> Not sure how to read that. Isn't simply 1 process per connection?

<comment><USER14> Yes. But you can't run more than one query per connection.

<comment><USER15> One _executing_ query per connection, meaning a connection maintains a serialized query processing model.

</comment></comment></comment></comment></comment><comment><USER16> Technical issues aside, as a db service platform provider, the statement in its Terms, "FI adheres to policies and procedures to prevent data and information loss, including a daily back-up regime, but does not make any guarantees that there will be no loss of data or information, or that FI policies and procedures will conform to industry best practices. FI expressly excludes and disclaims any liability for any loss of Licensee's data and information, no matter how such loss was caused." will make any serious user think twice or more times ... Would welcome further comments.

<comment><USER17> Sounds like standard legal boilerplate. AWS has similar wording in their terms. [https://aws.amazon.com/service-terms/](https://aws.amazon.com/service- terms/) Search for loss of data.

</comment></comment><comment><USER18> The "starting from thinking about this like its linux instead of thinking of it like its oracle" description is one I regularly use in Hadoop talks for less technical audiences.

The paper is interesting, it leaves me with more questions than answers.

The biggest question is: -is this a new beast from the ground up or more efficient packaging around existing distributed computing libraries?

\--For example when I hear a company say something like:

> based on log-structured merge trees (similar to Google Bigtable

I want to understand if they are just rebranding HBase for that part of the tool

<comment><AUTHOR> It is a new beast (good pun, by the way). There are no service dependencies like HBase or Zookeeper, and there is not really any packaging to speak of since it's just a JAR file.

Some open source is used internally, especially Netty, but our goal has always been to build a tightly coupled system for performance reasons. For example, the entire replication pipeline including the optimized Raft implementation is from scratch, as is the scheduler.

LSM trees perform well especially on SSDs, but we probably will migrate away from them eventually to something closer to LMDB in order to get a zero-copy read path.

</comment></comment><comment><USER19> "FaunaDB implements a process scheduler that dynamically allocates resources and enforces quality of service...Execution proceeds via cooperative multitasking."

Just like Windows 95! I don't think I've ever seen a database do this. Apparently it operates per-query or at least per application. Operating system seems right; this is a long paper.

<comment><USER20> Presto does this as well. Execution is done by "drivers" which move data pages between operators. It's a cooperative push model rather than a Volcano (iterator) style pull model. There are typically more drivers than worker threads. Drivers are assigned to threads using multilevel feedback queues (inspired by OS schedulers).

</comment></comment><comment><USER21> So, can we get this running on a microkernel? Or, Fauna-on-Xen, anybody? :]

</comment><comment><USER22> How does FaunaDB plan to support or enable many OLAP use cases? As far as I understand it, there are no interactive query sessions due to Calvin, but can I easily e.g. ETL things into a data warehouse?

<comment><USER4> A better way to think about how Calvin fits into our stack is as a building block similar to two-phase-commit, on which we've built the higher level transaction semantics. And while we do not have any plans on supporting session _transactions_ , this is more of a design constraint to support the best possible OLTP experience rather than a hard technical limitation.

Since our focus has been getting OLTP right, ETLs are currently handled by chunking record imports across multiple transactions. (We have internal tooling for this that we've used to help customers move their data into FaunaDB; we are planning on open-sourcing it soon.)

However, first-class support for OLAP use-cases is on our roadmap and something we are excited about providing, so expect more from us about this in the coming months.

<comment><USER22> Thanks!

</comment></comment></comment><comment><USER23> Wasn't the Informix Database effectively an operating system? I'm sure it wasn't the only one. This paper makes it sound like they're the only one.

</comment><comment><USER24> Detailed consistency model explanation starts on page 15.

</comment><comment><USER25> Performance numbers anyone?

<comment><AUTHOR> Write benchmarks, although the latest numbers are better still: [https://fauna.com/blog/distributed-acid-transaction- performa...](https://fauna.com/blog/distributed-acid-transaction-performance)

Read benchmarks are coming soon.

<comment><USER25> Great, thank you. Apologies for not going fishing for it myself. :+1:

</comment></comment></comment><comment><USER26> Some older operating systems used a database-filesystem or database like filesystem as storage.

I heard older IBM systems had something like this. BeOS had BeFS with inbuilt metadata index. NTFS + WinFS in user land vision was a bit like this. Office server aka MS Sharepoint (which implemented WinFS vision for intranet) stores all files in the database.

<comment><USER12> > I heard older IBM systems had something like this.

Most (non-Unix) mainframe and minicomputer operating systems – not just IBM's – have support for record-oriented files and indexed files in the filesystem. The functionality is roughly equivalent to that of non-relational "flat file" databases or key value stores. Examples include ISAM and VSAM under z/OS (aka MVS), and RMS under OpenVMS. This is in contrast to Unix and Windows, where the filesystem itself only supports files as unstructured bytestreams, and any record structure or index structure is imposed by higher levels such as applications or shared libraries.

You might also be thinking of the IBM minicomputer operating system OS/400 (nowadays called "IBM i"), which embeds a relational database (a variant of DB2) into the operating system. (I have often wondered how deep the integration actually is–I believe it is something deeper than just bundling a relational database with an OS, like how many Linux distributions include MySQL or Postgres, but I don't really know.)</comment></comment>


Google Tech Talk: The Secret History of Silicon Valley - <AUTHOR> http://video.google.com/videoplay?docid=-8911340403810065276 <comment><USER2> This is a repost. See:

<http://news.ycombinator.com/item?id=101601>

<http://news.ycombinator.com/item?id=101527>

A search feature on News.YC would be nice.</comment>


Best college majors for a career - <AUTHOR> http://graphicsweb.wsj.com/documents/NILF1111/

<comment><AUTHOR> Interesting fact related to the recent discussion of Google salaries[1]: The fourth earnings quartile for CS/Math majors begin at $151,000

1: <http://news.ycombinator.com/item?id=3202661>

</comment><comment><USER2> Very sad to see Mathematics and Computer Engineering near the top in median income, while Mathematics and Teacher Education sits near the bottom.</comment>


Why Scrum Should Basically Just Die in a Fire (2014) - <AUTHOR> http://gilesbowkett.blogspot.com/2014/09/why-scrum-should-basically-just-die-in.html <comment><USER2> Thank you for writing this. There is nothing in the world that has sucked more joy out of my life and made me question my continued existence as a software developer than complying with the mindless tedium of Scrum. From the ditto- heads harping on about velocity calculations to mindless unit test coverage fanatics to the needy psychotics who demand the daily stand-up, it has succeeded in vacuuming the joy out of the art of software development from my perspective. In contrast, the shops where software development was measured in terms of effective working software and high-quality releases were joyful places to be and made me realize that it is not software development that is broken but the processes that are intended to "fix" it.

</comment><comment><USER3> [http://programming-motherfucker.com](http://programming-motherfucker.com)

</comment><comment><USER4> Great, another anti-Scrum rant!

As I've said before, defense of agile and Scrum can easily get into "no true Scotsman" territory, with tedious back-and-forths about whether you are really following the philosophy and if your processes were just more pure you would see the light. Fortunately, this essay neatly cuts off that debate with "I have literally never seen Planning Poker performed in a way which fails to undermine this goal." This is an argument that the ideals of Scrum are hard to achieve, or that managing people to get them to buy into a new process is difficult; it says nothing about the success (or lack thereof) that would result if you were able to follow the methodology.

</comment><comment><USER5> Translation: I'm a rock star engineer, and I don't need anyone telling me how to communicate or estimate my work. If you hire nothing but rock star engineers, you can just trust them to build great software. I'm going to write a future blog post about how to hire nothing but rock star engineers.

For anyone who wants to try that strategy, good luck. I think you'll find that if you actually do scrum (rather than all the aberrations the author describes), you'll find you can do pretty well even without a team of nothing but rock stars.

<comment><USER5> Ok, he said a bit more than that. But most of it was directed at the same straw men you find in the typical anti-agile rant ("Scrum makes you meet in parking lots! And only the managers get to sit down! And nobody bothers to see if the products work, they only look at velocity!"), none of which has anything to do with scrum and everything to do with incompetent management, which regrettably, along with global warming, scrum can't fix. The rant opens with an observation that group decision-making is heavily influenced by social hierarchy in the workplace, another observation that, while accurate, is always true as long as 1) you're working in a team and 2) that team is made up of human beings.

The one useful observation it makes is that you don't really need F2F meetings, something that has not escaped the notice of the thousands of scrum teams around the world that include remote members. If anyone wasn't already aware of it, at least they got something. The rest pretty much sums up to the comment I made above.</comment></comment>


Opera Browser adds integrated adblocker - <AUTHOR> http://techcrunch.com/2016/03/10/opera-adds-a-built-in-ad-blocker-to-its-desktop-browser/ <comment><USER2> I use Opera daily. I really like its "Opera Turbo" feature. It compresses images for lower bandwith.

Also uBlock Origin for ads and blocking images larger than 200KB.

Saves a lot of mobile data for me.

<comment><USER3> Some people might think that Opera Turbo reduces privacy, since everything goes through a centralized proxy.

</comment></comment><comment><USER4> if ((!!window.opr && !!opr.addons) || !!window.opera || navigator.userAgent.indexOf(' OPR/') >= 0) { throw alert('your browser is not supported'); }

<comment><USER5> I stopped using at least 4 websites which I followed for years just because they tried to enforce on me turning off PrivacyBadger and uBlock. It will hurt only you in long term. It's more important for me to use my browser then you website that doesn't respect my choices.

<comment><USER6> > It's more important for me to use my browser then you website that doesn't > respect my choices.

Likewise I have the right to reject you if you can't comply with my terms.

<comment><USER7> What terms? They send you a GET and you responded with 200 OK. What they do with the response once it reaches their property is up to them, you have no claim to how they view it.

<comment><USER8> The terms that are presented and (quasi-)enforced by the javascript returned with that 200 OK.

"This site is not supported on browsers that block ads. Please switch to a different browser or disable your adblocker to proceed"

Those are terms. You can choose to ignore or circumvent them but they're still terms.

<comment><USER7> FWIW I have no problem with sites doing this. You can participate in ad- blocking arms race if you so desire. I have no obligation to run your JavaScript just because you sent it to me. You can try to make it a requirement and I'll probably just use software to circumvent it.

But underlying point is that you have no moral or ethical claim to how people view your content, just as you have no moral or ethical claim that a person cannot rip ads out of a magazine. If you want them to pay for your content, the Open Internet is not the place you should be publishing.

</comment></comment></comment></comment></comment></comment><comment><USER9> Opera is being bought up by a Chinese consortium. Will they be adding a new big firewall too? :-)

<comment><USER10> At least they were not bought by an US consortium. A firewall beats pervasive surveillance any time.

<comment><USER9> Well the surveillance can't be any better in China. Mabe less covert but still there.

</comment></comment></comment><comment><USER11> We are in the beginning of the end of the era of online advertising.

<comment><USER12> No, not a chance.

We are in an arms race between the people and the corporations.

As long as you can see content on a website, there will be corporations trying to monetize that fact. And as long as corporations have to operate within the borders of an open system there will be people trying to circumvent their methods.

<comment><USER13> An arms race like that will end with the richest corporations lobbying for an end to an open system. I'm sure we'll soon start seeing legislation attempting to criminalize ad-blocking.

<comment><USER14> [http://boingboing.net/2016/03/02/uk-minister-compares- adbloc...](http://boingboing.net/2016/03/02/uk-minister-compares- adblockin.html)

The UK is already moving in that direction.

This is yet another battlefield in the War On General Purpose Computing; you either have the right to run whatever software you want on your computer, or someone else is the _de facto_ owner of your computer through their ability to control what software is allowed to run. These battles need to be fought now, because it's hard to fight over property rights once someone else has already taken possession.

</comment></comment></comment></comment><comment><USER15> I understand that to some ads are an important revenue, but this idea that interesting content will stop existing if we greatly reduce advertising seems false.

Just think, how many time do you spend sharing content in social networks for free? Most of the time it's uninteresting content it's true, but some people are actually interested in sharing highly interesting content made by themselves with the world for free, both in social networks and out of them.

May it be due to personal promotion or just out of some societal need.

True that the content provider (unlike the creator) will probably always want to be paid to provide it, but in that case, the monetization comes from other products and not from advertising. Think about GitHub personal pages for instance.

</comment><comment><USER16> I used Opera (v5 to 7) for a long time and it always had an internal URL filter I used to block ads. I guess they just reimplemented it with a different focus.

</comment><comment><USER17> Opera has long been known for being ahead of the pack. Let's hope at least Mozilla and Microsoft follow suit...

<comment><USER18> Firefox already has tracking protection, which also blocks ads (because they almost universally track you). It's Opera that is following suit, probably as a reaction to the publicity Brave is getting.

<comment><USER19> Opera actually had an integrated blocker years ago in their previous engine. However, they scrapped most of the functionalities when they switched to Blink in 2013.

</comment></comment></comment><comment><USER20> Logically, the new owners of Opera want to infect you with their own spyware. And then blame it on the ad blocker.

</comment><comment><USER21> As publisher, fuck you. As consumer, that was the original reason i used Opera years ago, the natively implemented blocklist was just amazingly fast, especially compared to the extension on the other platforms.

Its a little sad that they changed so much that they have to reinvent the wheel now.

<comment><USER22> As a publisher, you should say fuck you to the ads networks that pay you to track your readers, not to the readers trying to protect themselves.

<comment><USER21> I dont. Its the decision of my readers if they want that. I dont care about adsense tracking at all, its not harming me, i am more afraid of other companies tracking which are not as secure and somehow privacy respecting as google.

<comment><USER14> > its not harming me

You're selling out your readers, who _are_ harmed by that. The cross-site aggregation that Google does is _far_ more harmful (see: "pattern of life" analysis) than a smaller tracker with bad security. This makes you just as responsible.

<comment><USER21> Its how the internet works, i am not happy with it ether and tell anyone to use a AdBlocker plus something like PrivacyBatcher. Hence i am in the progress to block all this directly on my Firewall.

My point is its the users responsibility to protect himself, if he goes on the Internet without any idea whats going on its just naive and my ads are the smallest of his problems.

<comment><USER21> I dont think Google is particularly bad and that its a bad thing they are analyzing their traffic to deliver custom tailored ads. Yeah i actually prefer them by far over the classic ad approach, i rather see things i am probably interested in.

I dont want to be tracked personally. That does not mean that other people who also use Windows, iPhones, Dropbox and whatever even care. Why would i tell them obivous things they dont even care about?

Not sure what gives you the idea that i would depend parts of my site to external services, i dont even use S3/AWS even thought thats industry standard. My sites work perfectly without ads, i even include ads in a way that they get invisible when blocked, so there are no UI (or any other) issues for people who prefer to block them.

I am not going to play the internet hero and miss out on my awesome extra income just because some people have radical thoughts about Google. It may makes you happy that i only have ads on sites which are worth it, but all the others have Adsense, so i assume you dont care.

Edit:// I also use Cloudflare if that helps ;)

</comment></comment></comment></comment></comment></comment><comment><USER23> Why would I want to deprive sites of their ad revenue?

If you use ad blocking, just be sure that you and no-one you are close to depends on advertising - directly or indirectly - to get their bills paid.

So damn childish to have such a sense of self righteous indignation that people turn off ads but expect the Internet to be free.

<comment><USER10> To protect yourself from tracking and malware. Being able to inform and educate yourself without several third-parties profiling you should be a cornerstone of our society.</comment></comment>


Senators push bill requiring warrant for U.S. data under spy law - <AUTHOR> http://news.trust.org/item/20171024100735-n5f69 <comment><USER2> Politics has me so jaded these days. Any time I hear of a "good" bill, I'm wondering whether it's secretly a bad bill, or, they're just grand standing to get some votes with no intention of ever passing it.

(Because if it was ever not those two cases, you'd think our government situation would be getting better over time.)</comment>


Discussion on keyboard input - <AUTHOR> https://www.reddit.com/r/linux/comments/5u0jds/novel_way_of_entering_text/ <comment><USER2> Wait, you linked to the reddit comments of a post that linked to the comments of a previous submission here?

<comment><AUTHOR> woops I got sidetracked and did it by misstake! Sorry!</comment></comment>


Bup – towards the perfect backup - <AUTHOR> http://wrouesnel.github.io/articles/bup%20-%20towards%20the%20perfect%20backup/ <comment><USER3> A shoutout for attic [https://attic-backup.org/](https://attic-backup.org/)

Attic is one of the new-generation hash-backup tools (like obnam, zbackup, Vembu Hive etc). It provides encrypted incremental-forever (unlike duplicity, duplicati, rsnapshot, rdiff-backup, Ahsay etc) with no server-side processing and a convenient CLI interface, and it _does_ let you prune old backups.

All other common tools seem to fail on one of the following points

\- Incremental _forever_ (bandwidth is expensive in a lot of countries)

\- Untrusted remote storage (so i can hook it up to a dodgy lowendbox VPS)

\- Optional: No server-side processing needed (so i can hook it up to S3 or Dropbox)

If your backup model is based on the old' original + diff(original, v1) + diff(v1, v2).. then you're going to have a slow time restoring. rdiff-backup gets this right by reversing the incremental chain. However, as soon as you need to consolidate incremental images, you lose the possibility of encrypting the data (since encrypt(diff()) is useless from a diff perspective).

But with a hash-based backup system? All restore points take constant time to restore.

Duplicity, Duplicati 1.x, and Ahsay 5 don't support incremental-forever. Ahsay 6 supports incremental-forever at the expense of requiring trust in the server (server-side decrypt to consolidate images). Duplicati 2 attempted to move to a hash-based system but they chose to use fixed block offsets rather than checksum-based offsets, so the incremental detection is inefficient after an insert point.

IMO Attic gets everything right. There's patches for windows support on their github. I wrote a munin plugin for it.

Disclaimer: I work in the SMB backup industry.

<comment><USER4> Sorry, but "Untrusted remote storage" and "No server-side processing" are exactly the opposite of what I need.

If the original box is ever compromised, I don't want the attacker to gain any access to the backup. If you use a dumb storage like S3 as your backup server, you need to store your keys on the original box, and anyone who gains control of the original box can destroy your S3 bucket as well. Ditto for any SSH- based backup scheme that requires keys to be stored on the original box. A compromised box could also lie about checksums, silently corrupting your backups.

Backups should be _pulled_ from the backup box, not _pushed_ from the original box. Pushing backups is only acceptable for consumer devices, and even then, only because we don't have a reliable way to pull data from them (due to frequently changing IP addresses, NAT, etc).

The backup box needs to be even more trustworthy than the original box, not the other way around. I'm willing to live with a significant amount of overhead, both in storage and in bandwidth, in order not to violate this principle.

The backup box, of course, could push encrypted data to untrusted storage, such as S3. But only after it has pulled from the original box. In both cases, the connection is initiated from the backup box, not the other way around. The backup box never accepts any incoming connection.

Does Attic support this kind of use case? The documentation doesn't seem to have anything to say about backing up remote files to local repositories. I don't see any reason why it won't be supported (since rsync does), but "nominally supported" is different from "optimized for that use case", and I suspect that many of the latest generation of backup tools are optimized for the opposite use case.

<comment><USER3> By pulling backups, you're giving the backup box full control over your computer, meaning that yes it must be more trustworthy. Push backups can indeed allow the initiator to wreck the remote state.

But in the modern commercial market, who are you going to trust as your backup box provider? A USA company subject to NSLs? Run your own in a rack somewhere? Having an untrusted server greatly decreases cost and increases the chance that you can actually produce a successful backup infrastructure at all.

It is possible to do it safely.

Since you say you're "willing to live with a significant amount of overhead", i would suggest a two-tier push/pull configuration. Desktop pushes to site A; then site B pulls from site A. This also increases redundancy and spreads out the attack surface.

Append-only is another good solution - i don't believe attic formally supports this today but it should be as simple as patching `attic serve` to ignore delete requests. Good first patch.

(Also if you really trust your backup server, then you don't need encryption anyway and can just run rdiff-backup over ssh.)

<comment><USER4> > _By pulling backups, you 're giving the backup box full control over your > computer_

Not really. On my production boxes, I usually set up a "backup" account and give it read-only access to paths that need to be backed up. Nothing fancy, just standard POSIX filesystem permissions. The backup box uses this account to ssh in, so it can only read what it needs to read, and it can never write anything to the production box. I wouldn't call that "full control".

> _Desktop pushes to site A; then site B pulls from site A._

What you described is similar to my own two-tier configuration, except I pull first and then push to untrusted storage like S3 (using encryption, of course). The first step uses rsync over ssh. The second step is just tar/gzip/gpg at the moment, but if I want deduplication I can easily switch to something like tarsnap.

<comment><USER3> I guess it depends on your security model. With one single pull backup,

\- if your backup box is on another network then it can be coerced into malicious reads (leaking private information, trade secrets, your competitive advantage etc).

\- if it's on the same network then it's subject to your same failure patterns.

Push backup has some disadvantages, but there's a lot of peace-of-mind in never (intentionally) granting additional users access to the unencrypted data.

Two-tier is one approach. There's another comment in this thread about snapshotting filesystems (ZFS, or i suppose LVM snapshots might be easier) which would be another method of addressing concerns about the client tampering with the backed up data.

</comment></comment></comment></comment></comment><comment><USER5> I've long been a huge fan up bup, and have even contributed some code. I might be by far their single biggest user, since I host 96748 bup repositories at [https://cloud.sagemath.com](https://cloud.sagemath.com), where the snapshots for all user projects are made using bup (and mounted using bup-fuse).

Elsewhere in this discussion people not some shortcomings of bup, namely not having its own encryption and not having the ability to delete old backups. For my applications, lack of encryption isn't an issue, since I make the backups locally on a full-disk encrypted device and transmit them for longterm storage (to another full disk encrypted device) only with ssh. The lack of being able to easily delete old backups is also not an issue since (1) I don't want to delete them (I want a complete history), and (2) the approach to deduplication and compression in bup makes it extremely efficient space wise, and it doesn't get (noticeably) slower as the number of commits gets large; this is in contrast to ZFS, where performance can degrade dramatically if you make a large number of snapshots, or other much less space efficient approaches where you _have_ to regularly delete backups or you run out of space.

In this discussion people also discuss ZFS and deduplication. With SageMathCloud, the filesystem all user projects use is a de-duplicated ZFS-on- Linux filesystem (most on an SSD), with lz4 compression and rolling snapshots (using zfssnap). This configuration works well in practice, since projects have limited quota so there's only a few hundred gigabytes of data (so far less than even 1TB), but the machines have quite a lot of RAM (50+GB) since they are configured for lots of mathematics computation, running IPython notebooks, etc.

<comment><USER6> I've also been using bup to replace (in some areas) my use of rdiff-backup.

It's a great tool, but since you contributed already, I cannot understate the importance of pruning old archives. For online/disk-based backup solutions, space is always going to run out eventually.

I'm using bup where I already know the backup size will grow in a way that I can manage for the next 1-2 years.

For "classical" backup scenarios though, where binaries and many changes are involved and the backup grows by roughly 10-20% a week due to changes alone, I _have_ to resort to tools where I can prune old archives because I would either have to reduce the number of increments (which I don't want to do) or increase the backup space by a factor of 50x (which I practically cannot do either).

</comment></comment><comment><USER7> I wrote a very similar tool before I knew about bup - <USER2> ([https://github.com/basak/ddar](https://github.com/basak/ddar) \- with more documentation at [http://web.archive.org/web/20131209161307/http://www.synctus...](http://web.archive.org/web/20131209161307/http://www.synctus.com/ddar/)).

Others have complained here that bup doesn't support deleting old backups. ddar doesn't have such an issue. Deleting snapshots work just fine (all other snapshots remain).

I think the underlying difference is that ddar uses sqlite to keep track of the chunks, whereas bup is tied to git's pack format, which isn't really geared towards large backups. git's pack files are expected to be rewritten, which works fine for code repositories but not for terabytes of data.

<comment><USER8> It's true that git's pack files are made for being rewritten, but bup doesn't do that. Every new run will create a new pack along with its .idx (which means that some packs may be quasi-empty) and the size of packs is capped at 1GB (Giga, not Gibi).

The real struggle of bup is how to know whether a hash is already stored, and how to know it _screaming fast_. It could be interesting to compare bup style and standard sqlite style, as you do in ddar.

Also, it seems ddar stores each object in its own file, like git loose objects. SQLite has a page [0] that compares this and storing blobs in SQLite, and I don't know what's the median size of your objects but if it's < 20k it seems to be better to just store them as blobs.

[0] [https://www.sqlite.org/intern-v-extern- blob.html](https://www.sqlite.org/intern-v-extern-blob.html)

<comment><USER7> > I don't know what's the median size of your objects but if it's < 20k...

I can't remember the exact number off the top of my head, but I designed the average size of each object to be much bigger - more like 64-256M than kilobytes. IMHO this works far better for backups. So I just use the filesystem to store the blobs, which I think works better.

<comment><USER8> Doesn't that alleviate the benefit of deduplication, if you're working on multi megabytes objects ? You'll end up copying a lot, unless I'm missing something obvious...

<comment><USER7> Most backups have very large areas of duplication. If a small file has changed, chances are that the small files around it have changed also. So de- duplicating with a larger chunk size seems to work fine in practice.

</comment></comment></comment></comment></comment><comment><USER9> Is there anything out there that does continuous incremental backups to a remote location (like obnam, attic, ...) but allows "append only" access. That is, you are only allowed to add to the backup, and the network protocol inherently does not allow past history to be deleted or modified? Pruning old backups might be allowed, but only using credentials that are reserved for special use.

Obnam, attic and similar use a normal read/write disk area, without any server side processing, so presumably an errant/malicious user is free to delete the entire backup?

<comment><USER10> Tarsnap.

You can create a key that has write only access for your automated backups and a different key that has full access for administrative purposes.

That is what we do anyway.

</comment></comment><comment><USER11> Haven't seen this mentioned - but, since bup de-duplicates chunks (and thus may take very little space - e.g., when you backup a 40GB virtual machine, each snapshots takes little more than the actual changes inside the virtual machine), every byte of the backup is actually very important and fragile, as it may be referenced from thousands of files and of snapshots. This is of course true for all dedupping and incremental backups.

However, bup goes one step farther and has builtin support for "par2" which adds error correction - in a way, it efficiently re-duplicates chunks so that whichever one (or two, or however many you decide) break, you can still recover the complete backup.

<comment><USER12> I saw "par2" and got all excited to see if lelutin re-implemented all the Galois field goodness from scratch, looked at the source and - no, bup merely spawns the par2 binary. Damn :)

</comment></comment><comment><USER13> I was wondering if someone's done a side-by-side comparison of the various newer open-source backup tools? Specifically, I'm looking for performance, compression, encryption, type of deduplication (file-level vs. block-level, and dedup between generations only vs. dedup across all files). Also, the specifics of the implementation, since some of the tools don't really explain that too well, along with any unique features.

The reason I ask, is I had a difficult time finding a backup tool that suited my own needs, so I wrote and open-sourced my own ([http://www.snebu.com](http://www.snebu.com)), and now that some people are starting to use it in production I'd like to get a deeper peer review to ensure quality and feature completeness. (I actually didn't think I'd be this nervous about people using any of my code, but backups are kind of critical so it I'd like to ensure it is done as correct as possible).

<comment><USER14> There's [http://burp.grke.org/burp2/08results1.html](http://burp.grke.org/burp2/08results1.html)

</comment></comment><comment><USER15> Like any good hacker I got tired of other solutions that didn't quite match my needs and made my own dropbox-like backup/sync using only rsync, ssh and encfs.

[https://github.com/avdd/rsyncsync](https://github.com/avdd/rsyncsync)

Not polished, but it's working for me.

        - only runs on machines I control
        - server requirement is only rsync, ssh and coreutils
        - basic conflict detection
        - encfs --reverse to encrypt locally, store remotely
        - history is rsnapshot-style hard links
        - inspect history using sshfs
        - can purge old history

shell aliases showing how I use it are in my config repository

encfs isn't ideal but it's the only thing that does the job. Ideally I'd use something that didn't leak so much, but it doesn't exist.

<comment><USER16> How does that compare to this:

[https://raymii.org/s/articles/Set_up_your_own_truly_secure_e...](https://raymii.org/s/articles/Set_up_your_own_truly_secure_encrypted_shared_storage_aka_Dropbox_clone.html)

<comment><USER15> Same principle just different mechanics and assumptions.

I can't work (very effectively) in two places at once, so I don't need robust merging, just CYA synchronisation. Using only rsync features I can do a full 2-way rsync merge and catch potential conflicts, erring on the conservative so I have reasonable confidence I don't lose any work.

Minimal workstation dependencies: only bash, encfs, rsync, ssh, coreutils/findutils and optionally atd for automation. encfs is optional, too.

Instead of dvc-autosync and XMPP I just use periodic execution. I partition my stuff into smaller/high-frequency vs larger/lower-frequency to keep this efficient. These are triggered from bash (PROMPT_COMMAND, in the background) and recursive at (atd).

The local data is unencrypted on disk from this tool's POV. I use encfs --reverse and rsync the result. To browse the history, I combine sshfs with encfs in forward mode.

Linux only because that's what I use, but it should be possible to support OSX.

All in all I'm pleased I'm able to use such minimal tooling for such a successful result.

</comment></comment></comment><comment><USER17> I tried some backup software (of the rdiff variety, not the amanda variety) last year when I set up a small backup server for friends and family.

Obnam and bup seemed to work mostly the way I wanted to but obnam was by far the most mature tool, so this is what I chose in the end.

On the plus side, it provides both push and pull modes. Encryption and expiration works. The minus points are no Windows support, and some horror stories about performance. Apparently it can slow to a crawl with many files. I haven't run into that problem despite hundreds of gig in the backup set, but most are large files.

On the whole it's been very stable and unobtrusive during the time I've used it, but I haven't used it in anger yet. So a careful recommendation for obnam from me.

</comment><comment><USER18> Does anyone use zpaq[1]? It has compression, deduplication, incremental backup, encryption, backup versioning (unlike bup, with the ability to delete old ones), and its written un C++. But im not sure about performance over network and how its compare with bup or rsync.

[1] [http://mattmahoney.net/dc/zpaq.html](http://mattmahoney.net/dc/zpaq.html)

</comment><comment><USER19> Deleting old backups and the lack of encryption is what stopped me from using bup.

<comment><USER20> some ppl have already started working on this and there's been activity on the mailing list lately about this topic.

however it's a dangerous feature to add in (backup tools should _never_ screw up their storage, and this feature goes and removes things) so a lot of care is needed.

the boring answer is: it's coming, and we need a lot of help for vetting patches and testing them out.

<comment><USER21> I think naming backups and archiving the old files should be separate applications - that could be generic to whatever backup tool you decide to use.

</comment></comment></comment><comment><USER22> Adding a plug for git-annex. [https://git-annex.branchable.com/](https://git- annex.branchable.com/)

git annex is for more than just backups. In particular, it lets you store files on multiple machines and retrieve them at will. This lets you do backups to e.g. S3, but it also lets you e.g. store your mp3 collection on your NAS and then easily copy some files to your laptop before leaving on a trip. Any changes you make while you're offline can be sync'ed back up when you come back online.

You can prune old files in git-annex [1], and it also supports encryption. git-annex deduplicates identical files, but unlike Attic &co, it does not have special handling of incremental changes to files; if you change a file, you have to re-upload it to the remote server.

git-annex is actively developed, and I've found the developer to be really friendly and helpful.

[1] You can prune the old files, but because the metadata history -- basically, the filename to hash mapping -- is stored in git, you can't prune that. In practice you'd need to have a pretty big repository with a high rate of change for this to matter.

 _Edited for formatting._

<comment><USER23> [http://git-annex.branchable.com/not/](http://git-annex.branchable.com/not/)

"git-annex is not a backup system."

</comment></comment><comment><USER24> Is there an easy way to have the backups encrypted at rest? That's a nice feature of Duplicity. I don't have to worry about someone hacking my backup server or borrowing my USB drive having access to my data.

<comment><USER20> currently bup doesn't implement encryption (since it's a pretty hard feature to get right and we do want to finish coding other key features -- like old backup removal -- before we get to that)

some ppl have reported using an encrypted storage backend like ecryptfs to store their bup repositories in. that option shouldn't be too hard to put together.

<comment><USER24> Makes sense, I'd probably prioritize pruning old backups first too. bup looks quite cool regardless, I'll be keeping an eye on it.

I like & use Duplicity, but it's a little finicky to set up right and some of the more obscure features are counterintuitive or even broken.

</comment></comment></comment><comment><USER25> This seems like a fantastic tool, and I would love to try this out. And, it's free!

My personal obstacle in using a tool like bup is the back-up space. I could definitely use this for on-site/external storage devices, but I also like to keep online/cloud copies. I currently use CrashPlan for that which affords me unlimited space. If CrashPlan would let me use their cloud with bup, wow, I would switch in a heartbeat. Perhaps cloud backup tools could learn some tricks from bup.

<comment><USER26> You might want to have a look at Tarsnap: [https://www.tarsnap.com/efficiency.html](https://www.tarsnap.com/efficiency.html)

<comment><USER25> Aware of Tarsnap and looks very attractive. I'd rather pay a flat-rate out because I store a lot of video (Musician) and other large files. I have about 750GB pre-deduplication stored. That's a lot more $ when I go to Tarsnap.

</comment></comment></comment><comment><USER27> If you want a fantastic graphical frontend for bup, there is kup, which is a kde app: [http://kde- apps.org/content/show.php/Kup+Backup+System?conte...](http://kde- apps.org/content/show.php/Kup+Backup+System?content=147465)

It is really easy to set up what folders to backup and where, and I use it whenever a backup is simply take all files from X, do the rolling backups at Y, and done.

</comment><comment><USER28> If you're considering using it, keep in mind the limitations: [https://github.com/bup/bup/blob/master/README.md#things- that...](https://github.com/bup/bup/blob/master/README.md#things-that-are- stupid-for-now-but-which-well-fix-later)

The one most likely to be a showstopper seems to be: "bup currently has no way to prune old backups."

<comment><USER29> I can see how this would be theoretically possible in the same way I could see using `git filter-branch` to remove one or more commits from a code repository. But as it requires walking back up the tree to recalculate all of the commit hashes based on the new state of your files, I suspect it would be an extremely slow/expensive operation in bup's case. Someone who knows more about bup's internals can correct me if I'm wrong.

<comment><USER30> Isn't this exactly what Obnam does?

<comment><USER29> I suspect that Obnam doesn't cause all of the de-duplicated chunks to get so "entagled" (as bup puts it, in their readme).

There are existing discussions about the way bup pruning would have to work:

[https://groups.google.com/forum/#!searchin/bup- list/prune/bu...](https://groups.google.com/forum/#!searchin/bup- list/prune/bup-list/86j9ovQ-TaI/NOZPfOAu8WkJ)

Sounds like they are actually making progress on it.

</comment></comment></comment></comment><comment><USER31> I've been using duply [http://duply.net/](http://duply.net/) for a while. It is a simple frontend for duplicity [http://duplicity.nongnu.org/](http://duplicity.nongnu.org/). I find it very easy to setup. It also provides encrypted backups trough GPG.

</comment><comment><USER14> There's also Burp which is worth a look [http://burp.grke.org/index.html](http://burp.grke.org/index.html)

Looking at [http://burp.grke.org/burp2/08results1.html](http://burp.grke.org/burp2/08results1.html) it seems it can outperform Bup in some situations.

</comment><comment><USER32> > That is a dataset which is already deduplicated via copy-on-write semantics > (it was not using ZFS deduplication because you should basically never use > ZFS deduplication).

Can someone more experienced with ZFS say why?

<comment><USER33> "Basically never" is an overstatement, but it is true to the point of "Never unless you already know why I said 'basically never'"

It boils down to the fact that ZFS maintains a mapping from hashes to LBNs. This allows write-time deduplication (as opposed to a scrubber that runs periodically and retroactively deduplicates already written blocks). This is somewhat memory intensive though. For smaller ZFS pools you can get away with just having lots of RAM (and with or without dedupe ZFS performs better the more RAM you have). For larger ones, you can add a SSD to act as additional disk cache.

Here's a quick description of that setup:

[https://blogs.oracle.com/brendan/entry/test](https://blogs.oracle.com/brendan/entry/test)

Note in this example that they were already showing 128GB of RAM for a 17TB pool; the L2ARC was to augment that. In general, ZFS was designed with a much higher RAM/Disk ratio than a workstation typically has.

<comment><USER34> ZFS is also very far away from the state of the art in online dedup. For instance, [http://users.soe.ucsc.edu/~avani/wildani- icde13dedup.pdf](http://users.soe.ucsc.edu/~avani/wildani-icde13dedup.pdf) has a theoretical dedup regime that needs only 1% of the RAM for 90% of the benefit.

</comment></comment></comment><comment><USER35> This looks very interesting as a replacement for rdiff-backup. Hopefully the missing parts aren't too far away (expire old backups, restore from remote).

</comment><comment><USER36> Can this new tool do incremental realtime disk image backup like Acronis True Image?

<comment><USER20> do you mean something that would automatically update the backup when a file is changed on disk?

bup currently doesn't do that. but there's been some talk of using inotify or another such method of knowing exactly which files are modified when they are so that bup could instantly work on those.

in theory it should be feasible, it's not implemented yet however

<comment><USER5> I had my Ph.D. student (Andrew Ohana) spend a while last summer implementing exactly this using python-inotify, since I wanted it to greatly improve the efficiency of [https://cloud.sagemath.com](https://cloud.sagemath.com), which makes very frequent snapshots. It's pretty solid and is on github: [https://github.com/ohanar/bup/tree/bup- watch](https://github.com/ohanar/bup/tree/bup-watch) He's been busy with his actual math thesis work and teaching, so hasn't got this upstreamed into bup. It also depends on changes he made to bup to store the index using sqlite instead of some custom format.

</comment></comment></comment><comment><USER37> Given that old backups cannot be remove, isn't backuppc a better solution?

<comment><USER20> backuppc is a mature technology and can very well be trusted for collecting backups.

there are some areas in which bup is trying to innovate in order to change how backups are considered (storage size being one of those)</comment></comment>


Posterous adds Google Maps support - <AUTHOR> http://blog.posterous.com/posterous-works-awesome-with-google-maps

<comment><USER3> Have you guys considered oEmbed (<http://www.oembed.com/>) support? It would be perfect for things like Posterous (it's also what we use in 280 Slides)

There's a handy 3rd party oEmbed provider for popular services that don't natively support it: <http://oohembed.com/>

<comment><USER4> We actually already support oEmbed -- the big problem with it is discovery. We have to hard code lists of sites that support oembed -- which really defeats the purpose.

<comment><USER5> oEmbed does have a discovery mechanism, though it's its barely implemented by anyone. 280 Slides implements the discovery consumer portion of the spec.

</comment></comment></comment><comment><USER6> Posterous continues to rock. I'm curious if they have some master list of every magic integration they plan to make, or if they're just making them up on a whim as they go along.

<comment><USER7> I would imagine they have some pretty good stats on what people are posting and are making some inferences based on what they think would be most useful to the most people. That's what I would probably be doing if I were in their shoes (but since I'm not I'd love to hear how their actually doing it).

</comment></comment><comment><USER8> I just love Posterous, it's a great product and the pace at which they add new features is great.

One feature I just realized I would love to have is to "integrate" Dropbox Photo Uploads in a post in the same way I can add Photos to an email (they create a gallery for me). This would avoid having to upload the pictures twice which would be especially useful on the road where the Internet is often slow.

Also a very nice feature would be the ability to edit the galleries, especially the possibility to add and remove pictures.

</comment><comment> brown9-2 Just curious - I've seen Posterous mentioned a few other times on HN - <USER2> anyone using Posterous, what are you mostly using it for? As a true blog, with posts, or as a quick way to dump files to a public place?

Seems like a neat tool, and a very polished presentation, just curious how people are using it in the real world.

<comment><USER8> I'm using it as a blog. I like the ease with which I can use it and how I can write my posts offline (GMail with Gears) and then upload them as soon as I have an Internet connection.

</comment></comment><comment><USER9> I am amazed the rate at which they are continuing its development. But I wonder if all such niche features would only cater to a select segment of customers.

Does the common user really care if there is Google maps support or not?

<comment><USER10> Capture the "niche users" and the "common user" will follow...

Niche users are usually the early adopters, the experts, and the evangelists to everyone else.

<comment><USER11> Posturous can attract the Tumblr crowd only when CSS theming is available.

</comment></comment></comment><comment><USER12> has Posterous done themes yet? Maps are cool and all, but you know.

</comment><comment><USER13> Still waiting for OpenID support from posterous.</comment>


Subway has a new logo for the first time in 15 years - <AUTHOR> http://www.businessinsider.com/subway-has-a-new-logo-2016-8 <comment><USER2> Brand it all you want. Food companies ultimately boil down to the food they serve. Subway was able to capitalize on the early days of the revolt against Fried Fast Food, but floundered badly. Most of the sandwiches are just as unhealthy as class Fast Food, all of the restaurants have the weird yeasty smell, and you almost always walk out with a soggy mess.</comment>


Ask HN: Composing programs from global function registry – is it viable? - <AUTHOR> Here's a rough idea:
- have a globally available (s3, cdn and whatnot) registry of functions
- each function is a tuple (name, dependencies, arity/spec/something about types, source, compiled bytecode)
- functions are referenced loosely by name and strictly by sha of that tuple directly in the code
Why would anyone do that?
- you no longer have to deal with dependencies - they are inferred from functions you reference/require in  your code
- your build process becomes simpler and faster - dependencies are all pulled at boot time (or constructed locally at deployment site) from the registry
- your deployment becomes much more granular - down to single function that can be swapped during runtime if the language is right (erlang, clojure, anything that's functional and dynamic enough?), can't get any  closer to zero downtime deployments without hassle
- when bug is found in one of the functions or a change is needed for any other reason, a new function gets published in the registry and it has new sha - amazing for discovering vulnerabilities, auditing and having reproducible builds
now tell me that this has been around since 50s and why it can't work. <comment><USER2> Could that be implemented as a subset with a little bit of metadata of e.g. go packages?

They can be limited to a single function, there is a quasi naming standard via git URIs, and mod.sum contains hashes, albeit locally, but there is a beta for a public database [https://sum.golang.org](https://sum.golang.org).

<comment><AUTHOR> Yeah, the idea is to reduce distribution unit to single function, you could technically do that with any language, some are just better fit I guess.

<comment><USER2> Ah, ok. I guess with the solution I had in mind you could not simply replace a function without recompiling. So an interpreted language might be better suited.</comment></comment></comment>


Hilarious Software Fails - <AUTHOR> https://www.boredpanda.com/funny-computer-errors-software-glitches/ <comment><AUTHOR> My personal favorite is this gem: "Task failed successfully".</comment>


Ask HN: Best HN threads in '08? - <AUTHOR> There was a ton of hacker news in 08. What were your favorite threads? <comment><USER2> Who's Hiring? <http://news.ycombinator.com/item?id=375410>

Startup Ideas We'd Like to Fund <http://news.ycombinator.com/item?id=250704>

How I Turned Down $300,000 from Microsoft to go Full-Time on GitHub <http://news.ycombinator.com/item?id=338286>

I'm out, baby <http://news.ycombinator.com/item?id=300960>

Start a side project, says GitHub founder <http://news.ycombinator.com/item?id=282158>

Balsamiq hits $100,000 in revenue <http://news.ycombinator.com/item?id=364254>

</comment><comment><USER3> Sell Your Web App: Lessons I Learned From Selling DropSend <http://news.ycombinator.com/item?id=347381>

In the Basement of the Ivory Tower <http://news.ycombinator.com/item?id=246606>

How to go down with style <http://news.ycombinator.com/item?id=330689>

Hedge Fund Manager: Goodbye and Good Luck <http://news.ycombinator.com/item?id=335815>

Cities and Ambition <http://news.ycombinator.com/item?id=201373>

Looking Back on Selling Gravatar to Automattic <http://news.ycombinator.com/item?id=345531>

</comment><comment><USER4> Solve the halting problem, $300-$1000: <http://news.ycombinator.com/item?id=376238>

I don't want to work very hard: <http://news.ycombinator.com/item?id=394609>

</comment><comment><AUTHOR> Ask HN: What is the best way to promote your new fancy web application? (<http://news.ycombinator.com/item?id=341138>)

Ask YC: What would you tell your younger self? (<http://news.ycombinator.com/item?id=372573>)

</comment><comment><USER5> There are at least twice as many i could pick, but here are some of my favorites.

<http://news.ycombinator.com/item?id=219804> \- Poor people consume more conspicuously than rich people, wealth measured relative to peer group

<http://news.ycombinator.com/item?id=133282> \- Story about SR-71 Blackbird crash

<http://news.ycombinator.com/item?id=303666> \- The "Luke Arm" for amputees

<http://news.ycombinator.com/item?id=318827> \- How to bribe your way into expensive restaurants

<comment><USER6> All great stories, but unremarkable comment threads. What I want to know is, did anyone here actually try to bribe their way into a restaurant after reading that article?

<comment><USER7> Not a restaurant, but I've used the $20 Theory of the Universe for concerts, tickets, bars and other treats.

<http://www.esquire.com/features/ESQ0303-MAR_20DOLLARS>

</comment></comment></comment><comment><USER8> How Porsche fleeced hedge funds and roiled the world's financial markets ~ <http://news.ycombinator.com/item?id=350332>

</comment><comment><USER9> Evolution of Mona Lisa in JavaScript & Canvas <http://news.ycombinator.com/item?id=392036>

</comment><comment><USER10> "Ask HN : How do big websites roll out new versions?" <http://news.ycombinator.com/item?id=411282>

Lots of interesting replies to that one.

</comment><comment><USER11> Unfortunately my favourite one missed the '08 deadline: <http://news.ycombinator.com/item?id=417782>

</comment><comment><USER12> Thanks for sharing these links. I am commenting to visit this thread later. Is there another way to mark a thread to be visited later from my profile?

<comment><USER7> Upvote the thread, then click here:

<http://news.ycombinator.com/saved?id=mad44>

<comment><USER12> Thank you, that is helpful. I wish there was a button for it on the top bar at HN.

</comment></comment></comment><comment><USER4> Human Beautification Algorithm: <http://news.ycombinator.com/item?id=328377></comment>


Ask HN: Are there alternatives to the recurse centre? - <AUTHOR> https://www.recurse.com/ looks great, as an educational retreat. But I'd be more interested in going somewhere other than new york perhaps somewhere closer to nature - tall order but is anyone aware of similar institutes? <comment><USER2> It's not quite the same thing as Recurse, but some of the previous instances of Iverson College look absolutely idyllic.

[http://iversoncollege.com/](http://iversoncollege.com/)

</comment><comment><USER3> I'm a developer in Mexico and this would be a perfect place to do something like that, but I don't have the resources to make it happen and there isn't a strong VC atmosphere here.</comment>


Reddit, with wigs and ink - <AUTHOR> https://aeon.co/essays/how-personal-letters-built-the-possibility-of-a-modern-public <comment><USER2> > Habermas pointed to the early 18th century as a crucial moment of change. At > that time, newspapers and periodicals exploded in both number and influence. > 'In _The Tatler, The Spectator_ and _The Guardian_ the public held a mirror > up to itself,' Habermas noted of the impact of a trio of periodicals by the > journalists Joseph Addison and Richard Steele. The new publications allowed > readers to shed their personal identities as rich or poor, male or female. > Instead, in print they could enter into conversation as anonymous equals > rationally engaging with the topics of the day.

Just to note that the present-day publications with the same names are _not_ continuations of the periodicals referred to above.

" _The Tatler_ was a British literary and society journal begun by Richard Steele in 1709 and published for two years." [1] " _The Spectator_ was a daily publication founded by Joseph Addison and Richard Steele in England, lasting from 1711 to 1712." [2] And " _The Guardian_ was a short-lived newspaper published in London from 12 March to 1 October 1713." [3]

[1] [https://en.wikipedia.org/wiki/Tatler_(1709_journal)](https://en.wikipedia.org/wiki/Tatler_\(1709_journal\))

[2] [https://en.wikipedia.org/wiki/The_Spectator_(1711)](https://en.wikipedia.org/wiki/The_Spectator_\(1711\))

[3] [https://en.wikipedia.org/wiki/The_Guardian_(1713)](https://en.wikipedia.org/wiki/The_Guardian_\(1713\))</comment>


Are you going to Startup Riot 2011?  - <AUTHOR> http://startupriot.com/

<comment><AUTHOR> Andrew Warner is giving the keynote.

If you are going, where are you staying?

<comment><USER2> There are definitely a bunch of options near the venue including the Westin which has a rotating restaurant (warning: expensive!) at the top so you can see the entire city in about an hour or a Holiday Inn just a block over. Keep in mind that the after-party runs to 7pm so depending on how much you drink, you'll want to choose your place to stay based on that.

Not sure if there are any places on airbnb.com that are useful. Since I live in Atlanta I haven't had to try. I have stayed at the Twelve Hotel at Atlantic Station (which is a bit far but there is a Twelve Hotel at Centennial Park which is much closer) and those rooms are really nice. You can get a 2-bedroom suite with a full kitchen and living room and then potentially have a lot more people in there especially since you'd have 2 full bathrooms.</comment></comment>


Sudoku Solver in python - <AUTHOR> http://www.codingismycraft.com/2013/05/11/sudoku-solver-in-python/

<comment><USER2> The code is missing one of the "groups". There should be 27 (one for each of the 9 rows, columns and 3x3 squares), but there are only 26. The missing one is the column [8 ... 80].

Probably the most famous Sudoku program is Peter Norvig's, also written in Python:

<http://www.norvig.com/sudoku.html>

<comment><AUTHOR> Good catch... I've changed already.. thx

</comment></comment><comment><USER3> In my opinion the most impressive sudoku solver in Python is the one using Z3: <http://rise4fun.com/Z3Py/tutorialcontent/guide#h210>

Less than 30 lines of code just specifying what is a Sudoku game is!

</comment><comment><USER4> A few years ago I wrote a sudoku solver using the Knuth Dancing Limks algorithm. It's kind of a fun project, and I included an option to output the board as it recurses.

<http://cavernum.net/dlsudoku/></comment>


What Caused the Dinosaur Extinction? - <AUTHOR> https://www.theatlantic.com/magazine/archive/2018/09/dinosaur-extinction-debate/565769/?single_page=true <comment><USER2> The evidence for a massive impact precisely at the K/T boundary is ironclad. Any alternate theory in which the impact was not causal has to explain how this incredible coincidence (the largest impact in the past 100 million years occurring in a layer deposited in just thousands of years, right at the boundary) occurred.

<comment><USER3> Is it possible for a meteor to cause a chain of volcanoes to blow? like all the pressure under the crust gets rattled around?

<comment><USER4> Maybe.

On Mercury, it seems the impact that created the Caloris basin [1] may also have created a seriously busted-up region at its antipode (opposite on the planet from the impact) [2]. This may have happened due to shock waves propagating through Mercury's interior, focused by reflection at the planet's surface onto an area opposite the impact. There are alternate theories.

A few years ago, I looked into whether the Deccan Traps eruptions might have been either triggered or enhanced by similar antipodal seismic wave energy from the Chicxulub impact. Based on Earth's continental positions about 65M years ago, it seemed possible, maybe plausible. There were some papers discussing the possibility, but I don't have the data at hand.

[1] [https://en.wikipedia.org/wiki/Caloris_Planitia](https://en.wikipedia.org/wiki/Caloris_Planitia) [2] [https://en.wikipedia.org/wiki/Caloris_Planitia#Antipodal_cha...](https://en.wikipedia.org/wiki/Caloris_Planitia#Antipodal_chaotic_terrain_and_global_effects)

<comment><USER5> > A few years ago, I looked into whether the Deccan Traps eruptions might have > been either triggered or enhanced by similar antipodal seismic wave energy > from the Chicxulub impact. Based on Earth's continental positions about 65M > years ago, it seemed possible, maybe plausible. There were some papers > discussing the possibility, but I don't have the data at hand.

Could it be worked backwards, there a re a number of flood basalts, some as recently as 10mya, is there any evidence of impacts for any others? Or is there some reason the Deccan traps stand out compared to the others?

<comment><USER4> Good idea. There may be enough impact and vulcanism events with reasonably good dating to search for the former as a cause of the latter.

</comment></comment></comment></comment></comment><comment><USER6> This is about the supervolcano hypothesis.

Note that this and the Alvarez asteroid hypothesis aren't mutually-exclusive; a significant impact event could in fact trigger vulcanism on a massive scale.

There's plenty of evidence that the K-T impactor was real and slammed into the Earth. It alone might've caused the massive die-offs, or it might've triggered vulcanism and/or venting of poisonous gas deposits from beneath the seabed.

<comment><USER7> According to the article, the vulcanism starts two hundred thousand years before the impact.

<comment><USER8> Sure, started, but...

[http://science.sciencemag.org/content/350/6256/76](http://science.sciencemag.org/content/350/6256/76)

> The existing Deccan Traps magmatic system underwent a state shift > approximately coincident with the Chicxulub impact and the terminal- > Cretaceous mass extinctions, after which ~70% of the Traps' total volume was > extruded in more massive and more episodic eruptions. Initiation of this new > regime occurred within ~50,000 years of the impact, which is consistent with > transient effects of impact-induced seismic energy.

Then there's the possibility of another impactor coincident in time to Chicxulub: [https://en.wikipedia.org/wiki/Shiva_crater](https://en.wikipedia.org/wiki/Shiva_crater)

</comment></comment></comment><comment><USER9> Excellent article that I thoroughly enjoyed. I had no idea there was any other compelling explanation for the dinosaurs' extinction other than an asteroid but it makes sense. It's definitely convincing enough that it shouldn't be dismissed outright and it sounds like about half of the scientific community is doing just that.

</comment><comment><USER10> Is the Radiolab account off? That the impact basically fried the surface of the entire planet at > 1000 degrees? Wasn't that also kind of the premise of Seveneves? That a bunch of particulate matter gets into the atmosphere and rains down, on reentry it burns like the space shuttle but with so much of it around the entire planet it basically bakes the entire earth

In the Radiolab episode they claimed the entire planet is covered in the baked layer. That you can dig anywhere on the planet and find it.

[https://www.wnycstudios.org/story/dinopocalypse/](https://www.wnycstudios.org/story/dinopocalypse/)

at about the 10 minute mark

or visual

[https://www.youtube.com/watch?v=ZYoqtBEzuiQ#t=15m](https://www.youtube.com/watch?v=ZYoqtBEzuiQ#t=15m)

Is that still a valid theory?

</comment><comment><USER11> Extremely well written and fascinating story on a competing theory for the mass extinction. still not convinced that the generally accepted Alavarez Asteroid hypothesis is wrong. Worth a read nonetheless.

<comment><USER12> That's the thing in pop science though. String theory is generally accepted -- so write about the iconoclasts who reject it. Asteroid accepted as the end of dinosaurs? Write about the iconoclasts there.

<comment><USER2> String theory is not generally accepted. It has not made a single successful prediction, or indeed any predictions at all. It's more correct to say most physicists view it with skepticism mixed with disdain.

</comment></comment></comment><comment><USER13> This is interesting less for the science and more for the conflicts between different camps of "rationalists" about what "science" means. Especially given that the "rockstar physicist shows up with a model, gets angry when everyone doesn't accept their conclusions" dynamic is hardly unique to paleontology.

</comment><comment><USER14> Start with a 3 species model. Imagine a shallow, closed sea, in which there is a shark, a plesiosaur, and a bird. We need to explain why the shark and the bird survive, but the plesiosaur goes extinct.

An asteroid hits somewhere, causing environmental change. For the rest of this comment, I'm going to refer to temperature as a short-hand way of referring to any environmental change.

The shallow, closed sea is undergoing temperature change, so the environment for bacteria and viruses is rapidly changing, therefore new species of bacteria and viruses are emerging at a high speed.

The shark is safe because the shark has general purpose macrophages which fights all pathogens with equal efficiency -- the shark does not care whether a pathogen is new or old, because the shark has a disease fighting system which does not rely on the principle of immunity. Sharks are 350 million years old -- they evolved long before immunity existed.

The plesiosaur and the bird are both in danger, because they have a system of disease fighting that does rely on immunity -- they are efficient at fighting off pathogens they've already been exposed to, but they are inefficient at fighting off pathogens that they have never been exposed to.

The birds see half the flock grow sick and die, so some instinctual fear of disease kicks in and the birds fly away from the shallow, closed sea.

The plesiosaur has no escape. Their disease fighting system is inefficient when dealing with new diseases, and they are in a bath of new diseases, from which they can not escape. They go extinct.

Immunity is the best model I've seen so far for explaining which species went extinct and which species survived.

<comment><USER15> What's with all the pretentious exposition? You could've shared your shower- thought in 140 chars.

<comment><USER16> Please don't be nasty here.

[https://news.ycombinator.com/newsguidelines.html](https://news.ycombinator.com/newsguidelines.html)

</comment></comment></comment><comment><USER17> Great article, but the ending made little sense:

 _We all chuckled at this prediction—mass extinction, by this point, having become something of a macabre inside joke. Just past the spoil, we reached the end of the road, which was lined with piles of white dirt too tall to see over. Clambering over them in search of outcrops, we were confronted by a strange view on the other side: an enormous field of coal, pockmarked with holes. The black earth had been dug at regular intervals to create thousands of pits, all the size and depth of shallow graves. Each one had its own mound of white earth beside it, as if waiting to be filled. No one could explain what they were._

This has little to do with the rest of the article, raises a whole slew of questions, and the phrase "no one could explain what they were" is uncharactersitically vague.

It feels like I missed the punchline to a joke. I can imagine the kinds of debates about the origins of the pits might resemble the debate about the impact theory, but this indirection seems like an odd way to end an otherwise clearly written article.

What am I missing?

<comment><USER18> The author seems to be talking about mining fossil fuels. And how they believe that fossil fuels will end humanity.

</comment></comment><comment><USER19> This articles tone comes across as incredibly snide, sarcastic, and condescending. This is how it describes the impact theory:

'Mystery solved'. 'Scientists cheered'. 'a heartwarming story about the integrity of the scientific method.'

It reads more like a snarky movie review than science journalism.

<comment><USER20> 'snide, sarcastic, and condescending' was not just the authors view, but the way involved scientists view theories of each other. So it's unfair to blame the article for that, which is an article about a scientist and not science in general.

</comment></comment><comment><USER21> From the article:

"In 1997, hoping to reconcile disagreement over the speed of extinction, scientists organized a blind test in which they distributed fossil samples from the same site to six researchers. The researchers came back exactly split."

Oh dear, that is bad.

</comment><comment><USER22> tl;dr: Stuff happened 65+ mya. Theories developed over past ~50 years. Human extinction due in 50 y to 0.5 my. Implication: there is no need to rush a conclusion. New evidence will continue to surface. We might be extinct before certainty arrives.

</comment><comment><USER23> I'm going to disagree that this was well written. I expected a factual article about dinosaur extinction and had to stop at the navel-gazing comment on a suicide attempt.</comment>


Ask HN: How would you rebuild Facebook? - <AUTHOR>

If you had to rebuild Facebook from scratch right now what would you do differently (from a software architecture perspective)? What technologies would you use, and how would you make it more scalable from ground up?<p>More generally: what concepts and technologies are key in making a highly scalable web application similar to Facebook? <comment><USER2> If Zuck would have tried to build what now is Facebook he would never have launched, so I would keep it simple, using the tech I already know like he did.

Highscalability.com have some articles about Facebook's architecture here: [http://highscalability.com/blog/category/facebook](http://highscalability.com/blog/category/facebook)

</comment><comment><USER3> I would go with something similar to the Diaspora Project: descentralized social network.

</comment><comment><USER4> I'm building [http://sublevel.net](http://sublevel.net) using Django. I'm making sure everything is as simple as possible from code to UI to API.</comment>


CanJS: an MIT-licensed, client-side, JavaScript framework - <AUTHOR> http://canjs.us/

<comment><USER2> Anybody using this? How does it compare to others (e.g. Backbone?)</comment>