How I Converted My Subscription Site from Paypal to Stripe in 2 Days - <AUTHOR> http://justinvincent.com/page/1790/how-i-converted-my-subscription-site-from-paypal-to-stripe-in-2-days

<comment><USER2> I implemented Stripe for BCC, too, as an experiment. (I was going to milk a blog post out of it, but all my A/B tests so far have been pushes versus using Paypal, so it wasn't important enough to write before Christmas and I've been swamped since.)

It is every bit as easy as Justin says. BCC's integration was in production in 3 hours, including the wholly-new purchasing flow that my site required to support it. No hitches since then at all.

P.S. I am not unhappy in any way about Paypal, I just experiment with things all the time.

<comment><USER3> have you seen an conversion differences in your A/B tests?

<comment><USER4> That's the push part; implying that the tests have been about equal.

<comment><USER2> Yep. Technically speaking, I got three null results: not enough evidence to reject the null hypothesis (that A was about equal to B).

Specifically, I tested

1) Paypal + Google Checkout (my previous arrangement) vs. credit card (Stripe) signups alone

2) PP + GC + CC vs. CC

3) PP + GC + CC with in-application upgrade messages pushing insta- gratification CC-based upgrades vs. plain PP + GC + CC with in-application upgrade messages dumping them to the purchasing page like I have done for the last 3 years

n.b. These are my results but they may not be your results. It is entirely possible my credit card form design (which I flagrantly stole from Twitter Bootstrap) is just bad, depressing CC signups, or the UX disimprovement crushed the favored-payment-method improvement, or something else entirely. Test test test.

<comment><USER5> Are you measuring, for this specific AB test, the number of conversions? Revenue? Profit? (Admittedly they should all trend in the same direction with the exception of profit, which might depend on how well each company's imposed costs work for your situation.)

<comment><USER2> Number of sales. BCC (almost) always sells for $29.95, and the difference between the three in processing costs is rounding error.

</comment></comment></comment></comment></comment></comment><comment><USER6> Stripe is easier to deal with and Stripe is also much faster than PayPal.

Stripe gives way more flexibility in billing.

However in order to use that flexibility I think it's better to dump the whole subscription model and don't even use IPN. I think all subscription management should be handled in-house.

Yes, main subscription use-cases are handled ok in Stripe plan/subscription model. But there are so many edge cases that dealing with external subscription model is way too hard (too hard in Stripe and even harder in PayPal).

For example, what if customer does not upgrade the plan, but downgrades? How would proration work in that case?

Or what if Stripe cannot charge credit card for a day or two because it's over the credit limit? When do I cancel premium membership? How do I reinstate premium membership if card started to work? What if user managed to resubscribe again using different credit card, but user's previous subscription started working after that and I got IPN about successful payment?

What if the card is permanently unchargable and how do I differentiate between permanent problems with credit card from temporary problems?

My solution to that is to always initiate payment from my web site (or from my backend service for recurrent payments). Do not delegate charge initiation to Stripe subscription.

<comment><USER2> This post is _exactly_ why I use Spreedly (for my subscription product), because writing code for all the edge cases would cost me a metric truckload more than the pittance I pay them every month.

<comment><USER6> The problem with Spreedly is that I still have to do a lot of stuff on my side. Spreadly would just give me some hooks, but then I have to handle these hooks myself.

For example, I need to put on hold premium benefits of temporary non-paying users. Spreedly (or Stripe for that matter) cannot do it for me.

I don't know if Spreedly can send notification email to my users (Stripe cannot at that moment), but even if such notification can be sent, there could be issues with email deliverability or with mismatch between my website name and spreedly "from" email field.

User expresses the intent to upgrade or downgrade on my web site. In order to communicate that intent to Spreedly/Stripe/PayPal I have to make a call that reflects that intent.

If the change is as simple as switch to another plan - then it may make sense. But if it's an edge case (e.g. discount), then I have to use more complex version of API. If I go through all that complexity anyway - why not just communicate with my own database and drop third-party dependency whenever possible?

Fortunately with Stripe it's quite possible: I just charge credit card whatever the right amount is at whatever the right time is. That's it. No researching Stripe/Spreedly/PayPal subscription model. No dealing with arbitrary restrictions of these models.

Another problem with Spreedly is that it does not support Google Checkout (most of my existing customers are from Google Checkout - my first payment processor). So I need to maintain all my Google Checkout customers myself anyway without any Spreedly help.

Bottom line: if you want to polish subscriptions UX for your customers - handling subscriptions in-house is the only viable long-term option. (Keep only subscriptions in-house. Payments processing should be outsourced).

Yes, it takes time. That's the cost of doing subscription business.

</comment></comment></comment><comment><USER7> Has Stripe fixed their webhooks so that they re-POST when the endpoint is down? They can't be compared to IPN until they implement it _reliably_... without that you _have_ to manually reconcile Stripe's data with your own database to be sure you haven't lost a post during brief downtime, network glitch, transient application or proxy error...

> Setting up a merchant account usually takes weeks and requires hours of > paperwork to be filled

That's a bit of an exaggeration... I've worked with 3 different merchant service providers underwritten by 3 different banks, and none of them took more than 2 days to approve an application, and the applications were like two pages long... 10 minutes to fill out. Names and socials of the principals of the business, estimated ticket size and monthly volume, description of your business, signature, date, send.

If you have the volume, that extra percentage point Stripe will shave off your gross sales is a lot to throw away.

They're a neat company with a neat API, but they're firmly in the "backup plan" category for me.

<comment><USER8> > Has Stripe fixed their webhooks so that they re-POST when the endpoint is > down? They can't be compared to IPN until they implement it reliably... > without that you have to manually reconcile Stripe's data with your own > database to be sure you haven't lost a post during brief downtime, network > glitch, transient application or proxy error...

We're working on this right now. It's part of a larger redesign of our webhook system that we started right before the break, and we should be able to launch it pretty soon. Our webhooks are the last somewhat legacy part of our product, so I'm pretty happy that it's getting reworked.

<comment><USER7> That's great. I don't mean to rag on Stripe, it's just one of the things that's held it back from being an easy PayPal replacement even in the US.

</comment></comment></comment><comment><USER9> Great post Justin!

One of the reasons we formed a US LLC (we're Canadians) when building Appifier was to work with Stripe. They're simply the best way we've found to handle online payments without the headache.

I can confirm that at the moment our experience (other than some foreign ownership headaches that were promptly resolved by their support staff) has been entirely positive. Integration was a breeze and the reporting is top notch.

Best of luck!

<comment><USER10> I'm also based in Canada (Mtl) and I has been waiting for months for their international support. What were the other reasons (if you wan to share)? I'm interested in the experience of making US companies from Canada and what are the time and money expenses this generates.

<comment><USER9> We're also in Montreal!

One big reason we went US was to avoid Bill 101 and all the language restrictions it entails. It was going to really slow us down. We also like the access to US capital that setting up in the States allows for.

An LLC is an amazing thing that we don't have in Canada. It provides limited liability while freeing you from the governance requirements that a corporation has. As a bootstrapped startup, it really sucks to have to deal with endless paperwork to keep your corporate veil in order.

Carl Mercier posted a really good blog entry from a local Quebec perspective, check it out:

[http://blog.carlmercier.com/2011/08/29/us-incorporation- for-...](http://blog.carlmercier.com/2011/08/29/us-incorporation-for-canadian- startups/)

If you need any more details about how we set it up, send us a shout!

<comment><USER11> >If you need any more details about how we set it up, send us a shout!

If you have the chance, could you do a blog/HN post about this? I'd love to understand how this is done as well.

</comment></comment></comment></comment><comment><USER12> I will also concur that Stripe is very easy to add to a site, test, and turn on billing. I just spent almost 8 hours building a stock photo site for my wife and I to use (<http://sleepybird.us/sedonaredrocks> \- written in Clojure :-) and I don't think that the Stripe part of it took more than 90 minutes total, largely because I found a Clojure library for Stripe and everything just worked.

</comment><comment><USER13> To clarify, you added stripe as a new payment option and you're running both in parallel now right? I didn't think there was a way to get your old PayPal subscribers moved over.

<comment><AUTHOR> Yes. I made stripe the main payment option and left paypal as a second option. There is no automated way (that I know of) to move customers from paypal to stripe.

</comment></comment><comment><USER14> Anyone know when we can expect this in the UK?

I've heard good things about Stripe so far but as much as I know it's only available in the USA at current

<comment><USER15> I hope they launch here soon. It's a really awesome service.

<comment><USER8> We're working on it! We want to make sure we get it right. But I do want to mention <https://stripe.com/global> for anyone not in the US that wants to be notified when Stripe gets to their country.

</comment></comment></comment><comment><USER16> Don't many users already have PayPal accounts, though? I'd happily spend twice the development time if it meant users were less likely to leave the checkout page because they already had an account and wouldn't have to type as much.

A little more dev work upfront can improve your conversion rates from then on, basically. Heck, I pay with PayPal instead of direct debit/credit cards when both are offered just because typing the PayPal password is easier than filling out all the card and address info.

<comment><USER6> > Don't many users already have PayPal accounts, though?

Nope.

I have 3 options: PayPal, Google Checkout, and Credit Card (Stripe, but my users don't know about Stripe).

Approximate proportion in new subscription payments is: 15% PayPal, 10% Google Checkout, 75% credit card.

Many existing users are switching from PayPal and Google Checkout to Credit Card (because now I have Stripe option).

</comment></comment><comment><USER17> "To get going the only new stuff I needed to add was a credit card form .."

One difference between PayPal and Stripe is that PayPal allows you to accept payments without hosting a credit card page. It would be great if Stripe could offer this kind of feature.

One alternative to PayPal, with hosted payment pages and IPN, would be WePay.

</comment><comment><USER18> AnyFu looks like a GREAT site. I've wanted access to subject area experts in the past for specific issues, but didn't want the delay involved with finding a long-term freelancer on oDesk or eLance. Plus, the quality is often sub-par. Best of luck to you. I signed up, and am excited to hear more.....

</comment><comment><USER19> Can you create subscriptions with Stripe on the fly? My wife runs a subscription business but pricing is based on the products the person chooses so my subscription rate is always variable. When I first looked at Stripe it seemed like I had to have subscription + prices created beforehand.

<comment><USER8> Yup, we have an API to create plans - <https://stripe.com/docs/api#plans>

</comment></comment><comment><USER20> Are people really choosing one or the other? Why not offer both? People DO like to shop with Paypal only.

<comment><AUTHOR> I guess I should have made it clearer that I converted the site to use stripe as the main payment processor but people can still pay with paypal if they want.

</comment></comment><comment><USER21> I wish stripe was in Canada.

<comment><USER22> Fill out this form:

<https://stripe.com/global>

</comment></comment><comment><USER23> Stripe is US only.

</comment><comment><USER24> Keep an eye on Dwolla.com if you want an alternative payment system without the 2.9% transaction fee.

<comment><USER7> Why do I have this scummy feeling about Dwolla? Was it something I read? I have this feeling that they're going to evaporate as soon as sites start taking Dwolla payments and there's a reason for them to be targeted by criminals... will 25 cents per transaction really cover the fraud-prevention excellence they need to develop?

<comment><USER25> I really think this is the case of their website design not giving a feeling of quality... Their design is OK... but I bet if they could give the site a softer look and have it convey a feeling of assurance/reliability - they'ed have a much better product... I realize design is a small part of the equation, but it goes a long way in gaining trust in the brand... Right now their site gives me the feeling of "stodgy corporate", slightly sketch... they need to fix that... stripe in comparison gives me the feeling of "cool", "reliably", "smart"... i'm just one dude though... who has been regrettably using authorize.net - so take it with a grain of salt...

<comment><USER7> There's nothing wrong with Authorize.net. Well documented, easy to implement API, automatic billing or charge-by-reference, reliable service, not overpriced... Why would you regret it?

<comment><USER25> As my service has evolved and newer payment options have emerged braintree, stripe... I'm just probably jealous envious of the coolness of those services... but you're right - authorize.net has not yet failed - good point.</comment></comment></comment></comment></comment>


Ooops. - <AUTHOR> https://github.com/MrMEEE/bumblebee/commit/a047be85247755cdbe0acce6#diff-1 <comment><USER3> I learned a rather unusual trick to keep myself safe from unintended glob matches. It is not "fool"-proof, but it will probably dilute an unmitigated disaster into an incomplete disaster: Keep a file named -i in the sensitive directories. When glob picks it up, which should be fairly early, it will be treated like a command line argument. Has saved me on occasions.

I also had a friend in school who used to, for whatever reasons, name his files using characters drawn from a pool of only two characters. One being "." the other "*". Please don't ask me why. He would then try to delete some particular file. You can well imagine what would happen next. This happened multiple times, till I went ahead and populated his directories with "-i" files. That worked great.

I usually keep rm aliased to 'rm -i', but once I did get burned. It was not because of hitting return early but because of having a space between a directory and the trailing "/".....while running rm as root. It was taking a bit longer than I had imagined, so I looked again at the prompt to see what had I typed..$#@!&~ :)

<comment><USER4> I hate that feeling you get in the pit of your stomach after you realize what's happening.

Same thing when you forget the WHERE on a DELETE FROM.

<comment><USER5> Over the years I've been steadily training myself to type "WHERE" earlier in the process, until I have finally settled on the obvious-in-hindsight solution: Always simply _start_ with the WHERE clause.

(Of course every effort not to be on the SQL shell of a production server in the first place should be taken, but sometimes you need a sledgehammer and nothing else will work.)

<comment><USER6> The habit I learned was: before running any DELETE or UPDATE statement, run a SELECT with the same WHERE. (e.g. if I meant to say DELETE FROM puppies WHERE cute = 0, first run SELECT FROM puppies WHERE cute = 0.)

I find I remember to do that because of the direct benefit (getting a sneak preview of what I'm going to delete), but it also means I end up thinking twice about the WHERE statement, so I'm much less likely to miss it out or get it dramatically wrong.

<comment><USER7> I'm a couple of levels more paranoid than that. First, I'll write the DELETE as a regular SELECT (to preview number of rows), then turn it into a SELECT INTO to save the soon-to-be-deleted rows into a table with a backup_date_ prefix (So old backups can be deleted occasionally). Next, before changing anything, I wrap the statement in a BEGIN TRAN, and ROLLBACK TRAN. After all that, I will finally modify the SELECT INTO into a DELETE statement, run it once while wrapped in the transaction (to verify that the number of rows modified hasn't changed), and then finally run without the transaction to delete the rows. Overkill?

<comment><USER4> Possibly overkill. But at least there won't be any tears.

</comment></comment></comment></comment></comment></comment><comment><USER8> Perhaps because I'm not a GitHub user, and because I've only ever peeked at HNers' GitHub accounts, but I was always under the impression that given the nature of the service, it would have an early-days-of-HN feel wrt to user behaviour.

It was a little disheartening to see the number of Reddit-esque comments that are simply a couple of words along the lines of "omfg" and a constant stream of meme abuse. I expected better from the programming community.

Sigh. Am I just becoming old, jaded and too elitist for my own good?

<comment><USER9> It's like reading Youtube comments--not for the faint of heart.

And the jab at reddit from the HN pedestal is probably misguided... reddit used to be more like HN, and HN is becoming more like the bad parts of reddit every day.

Every site tends toward Youtube level comments as time passes, and the people who don't like it eventually jump ship to a new site, and then the process repeats itself.

<comment><USER10> "And the jab at reddit from the HN pedestal is probably misguided... reddit used to be more like HN, and HN is becoming more like the bad parts of reddit every day."

But you just did the same thing...

<comment><USER9> Not really the same thing. I'm saying that HN shouldn't look down its nose at reddit, since it appears to be heading the same direction. I'm taking a jab at _both_ sites (and all sites, really).

</comment></comment></comment></comment><comment><USER11> The reason we all know to be careful with rm -r is because of that one time we weren't.

Me, it was the time I rm r'ed the MySQL data directory for my company's customer service management system. Oops. Thankfully we had a backup from the month prior, but I learned two things that day: a) be really careful with rm and b) take it on yourself to make sure IT is backing up stuff that you're messing with.

You gotta hedge against your own stupid.

What about you, how'd you learn the hard way?

<comment><USER12> Was asked to uninstall IBM/Rational ClearCase from our source code repository server. Apparently at the time, Clearcase's installer NFS mounted 127.0.0.1:/ to a subdirectory. Don't ask me what brain-dead system designer thought this was a good idea.

So, I did a simple /etc/init.d/clearcase stop (not sure that is the exact name) and:

# rm -rf /opt/clearcase

(hmm... that seems to be taking a little too long to run)

Panic - then ctrl-C - it was too late, /opt/clearcase/foo was NFS mounted 127.0.0.1:/ and it had already trashed /bin, /sbin, /etc, /var, and most of /usr.

Luckily I had good backups, but we did spend the rest of the day rebuilding the source repository while the developers couldn't check in any code.

<comment><USER13> One of my first tasks after being given root access at my first _n?x job at an ISP was "clear off all of the DNS stuff on $main_server since we have $dedicated_dns_server now".

So I merrily started mv-ing things to a scratch directory that we could wipe in 6 months if we didn't need anything from it.

Unfortunately, the zone file directory was NFS mounted _from* $dedicated_dns_server. With pass root set.

I think I took all of A through to K of client zones offline before we noticed.

I'm just very very glad I decided to do it as an mv rather than an rm, since it meant all I needed to do was copy things back.

Not that I really learned my lesson that time; it took a couple more semi- disasters before I got sufficiently paranoid to be reasonably safe as root.

<comment><USER14> Too late now, but remember to escape your asterices.

</comment></comment></comment></comment><comment><USER15> I've started quoting arguments in shell scripts even when it's not technically necessary to avoid problems with spaces. I can't count how many scripts I've written/encountered that didn't work with a path containing a space (apparently much more common with OS X users than Linux users)

This wouldn't delete the correct directory, but at least it won't delete "/usr" either:

        rm -rf "/usr /lib/nvidia-current/xorg/xorg"

There are lots of other pitfalls associated with not quoting things in shell scripts, like this common one:

        if [ $foo -eq "bar" ];

will cause an error if $foo isn't set or is an empty string, while this will work correctly:

        if [ "$foo" -eq "bar" ];

Bonus that your syntax highlighter can highlight the arguments. My rule is that flags aren't quoted, but paths and other parameters are.

<comment><USER16> I always use set -eu. Makes my scripts much safer and less unpredictable.

</comment></comment><comment><USER17> I don't remember the exact details, but a few years ago there was a Perl module that did something like:

       my $path = something_that_can_return_undef_on_failure;
       `rm -rf /$path`;

during the execution of its test suite. The author didn't catch it in testing because he never ran "make test" as root (who would?). But people on the Internet ran "make test" as root, with disastrous consequences.

<comment><USER18> I'm not sure if you're being sarcastic with the "who would?" portion of your comment, but doesn't cpan run tests, by default, whenever installing something? And, most people who aren't Perl developers with their own private Perl installation install CPAN modules as root, so they are available to all users. So, to answer the question seriously: Most people.

<comment><USER17> Uh, the cpan client runs "make install" as root, not "make test".

</comment></comment></comment><comment><USER19> Here is the original bug report. <https://github.com/MrMEEE/bumblebee/issues/123>

I've never heard of BumbleBee, but it must be great if a user can have their machine wiped out and still thank the library author for their work.

<comment><USER20> The other bug reports linked in the commit are kind of sad. From 122:

"I thinking that something is messed up with bashrc or sh because in few seconds I cannot execute any command in other terminal.

Am I doing something wrong? Hope You can help me."

<https://github.com/MrMEEE/bumblebee/issues/122>

</comment></comment><comment><USER21> reminds me of

<http://thedailywtf.com/Articles/Bourne-Into-Oblivion.aspx>

<comment><USER22> When you design something and the default case is to shoot yourself in the foot, you should stop and rethink things.

</comment></comment><comment><USER23> I use the safe-rm (<http://freshmeat.net/projects/safe-rm>) package to help me avoid doing this kind of thing.

<comment><USER24> Do you not find that a lack of wide installation limits its usefulness?

<comment><USER23> I just use it for my boxes. It doesn't stop me putting something silly in a make file (as in the example here), but it does stop me getting burnt by it.

<comment><USER24> I see; thank you.

</comment></comment></comment></comment><comment><USER25> Back in the late 90s I worked on a small Windows product... our CEO complained that when the uninstaller ran, it left the empty directory behind along with some small temp files created by the software (that the package hadn't put there during install). So the guy making the package added a command to remove the directory and its contents...

... and the first reporter to try software, for reasons I'll never totally understand, chose to install it in C:\\. Worked great until he went to uninstall it.

</comment><comment><USER26> I've done a similar thing a few years ago when I was first starting work on my guild hosting company's code.

At the time, the main thing hosted on that machine was my WoW guild's website, which I had been working on for close to a year, and was beginning work on converting the site over to a general purpose guild hosting site.

I was doing some work for a client, setting up a mirror of sorts for some kind of yearbook thing I had built for them. For that, I made a script that would mirror the online yearbook with wget, zip up the whole directory, then clear out the mirrored pages (all I cared to store and serve was the zip file).

All of my websites were stored in /www on the server, and the raw yearbook was located at /www/clientname/www. Inside the clientname directory, I had the mirror script which was something like this:

      wget -whatever_options http://whatever.address www
      zip yearbook.zip www
      rm -fr /www

Unfortunately, because of how frequently I type / before www to get to my web dev directory, I instinctively put "/www" in the script where I just wanted to do "www". I ran the script, checked to make sure and it looked good, and deployed it to a cronjob.

My heart sank when I tried loading my guild page a few minutes later (just to see what was going on on the forum, if anything), and it served up a bunch of 404s.

I went to /www/guildsite and saw it completely empty, and almost immediately figured out what had happened. At that point, I had to get my composure and figure out what I was going to do (I did not have backups or source control). I unmounted the directory, and went off to lunch with a friend, shaking with anxiety.

Upon return, I started writing out a perl script to scour the device byte for byte looking for PHP scripts, looking for instances of <? and then capturing the following 2000 lines or so, assuming that would be a sufficient buffer. When the script was done, I set it to run on the partition, and 45 minutes later I had a few hundred files to work with.

I had to manually go through every file (which were named numerically in the order they were found in the filesystem) and determine if it was the most recent (or recent enough) copy of the code, clear off any trailing bytes, and rename it to the filename it used to have. Luckily I could remember the name of almost every file in the system. It took about 8 hours to go through the few hundred files and recover them.

Needless to say, I learned my lesson after that, but the feeling of victory I got from recovering those files from the device was epic.

6 years later, I realize that that's a rather trivial thing to do, but at the time, I didn't know what I was going to do, and remembering that the file system doesn't clear all the bytes of the a file just it's reference gave me tons of hope.

<comment><USER27> _I've done a similar thing a few years ago_

Ha - I'll bet _everybody here_ has a story that starts like that. (Although your heroic save was heroic!)

My sad delete story was in 1982 - I had a fantastic graphical robot-vs.-robot game stored on tape on an HP portable computer (I was in high school). For some reason, the delete command on the HP had a variant that deleted from a given block forward on the tape, as I recall, and for some other reason, my fingers just decided to type that variant even though I had never done it before.

I still miss that game. It just may have grown in my memory, but I'm pretty sure that was the _coolest piece of software ever written anywhere at any time_.

<comment><USER28> Mine was plugging in the power connector backwards on a super important ide drive (it was the only backup for an already failed server) and watching smoke issue from its controller board.

Having to explain what I'd done to the boss was so scary, I slunk home with the drive and traced the power circuit with my oscilloscope until I found a capacitor that had blown short. I soldered on a through-hole replacement and it worked!

I pulled the data and felt like king of the whole frikkin world for the next week or so.

<comment><USER29> My gosh that's one of the best stories I've heard in a long time. The thing that worries me is that things like this are going to me rarer and rarer - there seems to be less interest in electronics, and less electronics that can be hack-fixed like that.

<comment> X-Istence I am a software engineer by degree, but also took many electrical engineering courses in uni, and I can honestly say that it is starting to make a comeback. Sparkfun, Adafruit, MAKE and many other places are starting to make it more accessible, cheaper and easy to learn. The Arduino has been a boon, providing people with a cheap but powerful microcontroller to get started.

While yes technology is getting smaller I have found that with many parts I can now easily find replacements online, I can get advice from other professionals, I can easily figure out how something works so that I can fix it. I've currently got a power supply sitting on my work bench that has a weird issue and I am slowly going through, making a net list and building a schematic with part numbers in an attempt to isolate the fault.

Maybe I am a rare breed, but seeing as how the interest at Maker Faire keeps going up, and interest in electronics also keep going up I will assume that eventually more and more people will get into experimenting in this field.

At least that is my hope.

<comment><USER30> arduino FTW! just getting into it and caused my first electrical fire. :-) good stories.

</comment></comment></comment></comment></comment></comment><comment><USER31> This terrifies me now of every single install script that refuses to run if not root.

<comment><USER32> With a bug like this, an install script without root permissions can still wipe out your $HOME when messing around in the dotfiles. Scary!

<comment><USER33> That is why I keep a second home directory, /home/experimental for trying things out.

</comment></comment></comment><comment><USER34> It's a pretty messy script anyway - rm'ing /usr/lib/nvidia-current/xorg/xorg doesn't strike me as the most delicate approach to the problem.

</comment><comment><USER35> This release fixes the install script; it no longer deletes the /usr directory in its entirety. But how on earth did this get through even basic testing? Absolutely shocking!

<comment><USER36> Seems non-trivial to test. First, its in a bash install script - i wouldn't know where to start. Second, if you tested the behavior, you might just test that it "deletes /urs/lib/nvidia-current/xorg/xorg" by running the install script and checking that the folder is gone. Guess what, the folder _is_ gone...test passes.

</comment></comment><comment><USER37> A couple months ago I had to recover some rm'd files by basically grepping 512-byte blocks on the file system for the file headers then writing out the next few KB to a file on a separate partition to manually go through..

My command sequence was more like this though, rather than a straight rm:

        find -name '*.java' | xargs grep --color 'something'
        # guh, get rid of these old copied .svn dirs polluting output
        find -name '.svn' | xargs rm -rf
        # now what was that..
        find -name '*.java' | xargs rm -rf

Forgot to edit the right side of the pipe back to the grep. Zealous use of the up-arrow burned me...

</comment><comment><USER38> While we're on the subject, has anyone successfully found/created a replacement for rm on OS X that moves files to the trash, but doesn't break the interface of rm for use in scripts?

<comment><USER39> rmtrash

<http://www.nightproductions.net/cli.htm>

I prefer 'brew install rmtrash'

<comment><USER38> There's lots of stuff like that out there, I'm talking about something I can actually _replace_ the rm binary with, a wrapper that precisely maintains the rm interface as described here: [http://developer.apple.com/library/mac/#documentation/Darwin...](http://developer.apple.com/library/mac/#documentation/Darwin/Reference/ManPages/man1/rm.1.html)

I like the idea of actually wrapping the unlink syscall, but I have no idea if/how darwin allows that.

<comment><USER40> You can alias trash-cli over rm, though it ignores -i, -r and -f and instead runs recursively, forced and non-interactively by default.

</comment></comment></comment></comment><comment><USER41> I only recently discovered molly-guard:

<http://packages.debian.org/sid/molly-guard>

which prevents you from running halt/shutdown etc. via SSH without first confirming the hostname.

I've done that before with pretty traumatic consequences, so it's now on my list of must-have's for any important remote box.

<http://www.catb.org/jargon/html/M/molly-guard.html>

</comment><comment><USER42> rm is like a knife where the handle has a sharp edge too...

<comment><USER43> A double edge knife/sword?

<comment><USER44> "Double-edged sword" has never worked for me, as a cliche. Do you often find yourself inadvertently smacking against the dull side of a single-edged sword, such that you stay away from double-edged swords for your own safety? rm is like a double- _ended_ knife, i.e.: [http://image.shutterstock.com/display_pic_with_logo/4253/425...](http://image.shutterstock.com/display_pic_with_logo/4253/4253,1131019295,1/stock- photo-double-bladed-throwing-knife-686738.jpg)

(Fun fact: many knife throwers grip the blade end anyway, rendering the cliche to an even simpler "rm is like a knife".)

<comment><USER45> I had a friend who actually knew proper sword technique, and he was not impressed by my machete with saw teeth on the other edge. With a single-edged sword you can use your forearm to support it when blocking, but with a double edged sword, not only can you not do that, but trying to block with the sword itself allows your opponent to overpower you and press your own sword against you.

<comment><USER46> For what it's worth, you can use your arm to support it if you're armored as well. If you're in chainmail/plate, the force is distributed well enough that there's no reason you can't do this, even with a double-edged weapon. That said, there are many swordfighting styles where armor hinders you significantly, and you frequently see single-edged weapons in these. It's actually really interesting to study the history of swords, their techniques, and the armor commonly used along side them -- used to have a really good book about this, but can't find it now.

</comment></comment></comment></comment></comment><comment><USER47> I actually ran that piece of code. Sure was glad it was only on a test partition. I did lose some trust in the developers after this, _but_ I tried bumblebee again later and am happy I did because it works great!

I forgive them.

</comment><comment><USER48> Never use rm's -f flag while operating as the root user. Never. Replace with -i until you are absolutely 100% certain the script you're writing works as expected. Always doubt yourself; be humble.

<comment><USER49> The -I flag is a bit more friendly; one may safely alias rm to rm -I and not even notice on the common case (deleting a single file).

<comment><USER33> I use a script that moves a file to a ~/.trash directory when I use rm. If I am sure I want to actually delete a file permanently I just use /bin/rm.

<comment><USER49> Sounds like a great way to fill up your disk to me.

<comment><USER50> You could use a cron job that removes the "deleted" files after some period, such as 30 days.

</comment></comment></comment></comment></comment><comment><USER51> I had a hard drive crash on me once, and I wasn't that worried about it, because I had setup network-based backup to the server in a different room. I remember thinking, "Restoring this backup will be sooo much easier if I can just connect it directly to the PC." Que me walking with the backup HDD towards the PC in question, when I drop it on the floor. When I plugged it in, it literally bounced inside of the chassis (platters fell off motor or something).

</comment><comment> cool-RR Is there a good reason why Linux doesn't have a recycle bin?

<comment><USER52> <sarcasm>because _real_ Operating systems don't need one?</sarcasm>

<comment> cool-RR That's what I thought. I'd be happy if anyone chimed in with a legitimate reason, but I won't be surprised if the lack of a recycling bin is just one more symptom of the Linux developer community's apathy towards the actual human beings who use their software.

<comment><USER53> I would speculate that it is a historical reason. It's no secret that the Unix environment was not designed for personal use in homes, but on mainframe time shared computers inside universities and businesses. Space was limited, and just moving files to another location to deal with later added unnecessary steps to a process that didn't have much of a benefit, at the time.

As space has become less valuable on computers, and they have become less of a specialized tool, it may be wise to add one, but most of the desktop environments already implement it already so there is no need to recreate the functionality at a lower level.

<comment> cool-RR _there is no need to recreate the functionality at a lower level._

I refer you to the original submission.

<comment><USER54> Care to provide any example of an OS that would prevent this? Windows and OSX both have this same fault.

<comment> cool-RR I don't have an example, I'm just thinking what's the best way for an OS to handle deletion.

</comment></comment></comment></comment></comment></comment></comment><comment><USER55> Looks like someone never read the Unix Hater's Handbook. Another fun thing is rm + shell expansion. A file named * or / can cause extremely unintended deletions.

<comment><USER56> Files can be named anything but / in Unix.

<comment><USER57> I'm pretty sure there's a file named /

</comment></comment></comment><comment><USER58> I keep /usr in a squashfs, mounted with aufs over top of it, for the 0.01% speedup I probably get (I grew up on gentoo, forgive me). Periodically, I need to rebuild the squashfs to reclaim space in the aufs writable directory.

I once had a script something like this:

        mksquashfs /usr /squashed/usr/usr_tmp.sfs -b 65536
        umount -l /usr
        umount -l /squashed/usr/ro
        rm /squashed/usr/usr.sfs
        mv /squashed/usr/usr_tmp.sfs /squashed/usr/usr.sfs
        rm -rf /squashed/usr/rw/*
        mount /squashed/usr/ro
        mount /usr

Guess what happens if you hit Ctrl-C during the mksquashfs? That's right, bash runs all the rest too, including the deletion of the old squashfs file. I was left without a /usr, and it was brutal. Managed to recover (it's incredible how resilient a linux system is, as long as /bin and /lib are intact), and immediately put "set -e" in almost every bash script on my machine (I also fixed the script to keep the system in a recoverable state at all times...).

</comment><comment><USER59> You get in trouble not at the moment this happens, you get in trouble much earlier than that - when you allow yourself into a situation where a single typo leads to ruin.

Myself, I use TimeMachine and bitbucket on my Mac, and every-15-minutes snapshots on all Amazon EC2 EBS volumes. Similar solutions can certainly be found for your platform of choice.

</comment><comment><USER60> 1000 points really? This is on par with initial tsunamii news.

</comment><comment><USER61> The designer of rm is the greater culprit here, not the author of that install script. A single mistyped character should not lead to such drastic consequences.

"Usability? What's that? I'm really smart, so I don't make mistakes. If lesser humans do - that's their problem". That seems to be the attitude of many Linux programs and rm is among the worst of them. No doubt I'll get downvoted for saying this, but I've rarely, if ever, heard of such things happening in Windows. (And people still manage to delete files in Windows without too much difficulty.)

</comment><comment><USER62> The two sure fire ways to live a long and happy life:

1\. find . -name " _pattern_ " <enter> <look carefully> <up arrow> | sudo xargs rm -f

2\. WHERE some_id = 36; <home> DELETE FROM table_name

</comment><comment><USER63> Unix could really do with a command that you can wrap around this type of call. Either a sanity check on the path part or a safe rm alternative that contains it. I would gladly give up full rm access to know that I can safely (or safer-ly) delete in scripts.

It could be something as simple as a file with paths on each line it - <USER2> one path or a path with a glob - and the script fails before destroying anything important.

Overriding it might involve adding a --override=/path/to/something but at least then it would be very explicit

<comment><USER63> Ah. It seems I have just described 'safe-rm'.

Great minds etc :)

<comment><USER64> Indeed. Those were exactly the design goals I had when I wrote safe-rm (shortly after deleting my /usr/lib!).

</comment></comment></comment><comment><USER65> just tried to explain this to a few of my windows using coworkers.

<comment><USER66> Just tell them it's the same as a Windows installer accidentally removing the entire Program Files directory.

<comment><USER67> I knew someone who, in the days of Windows 3.1, managed to accidentally invoke "format c:" from inside Microsoft Word - I was in the same room as them when they did it and heard the cries for help. What they couldn't do was explain to me what they had done to accompish such a feat.

<comment><USER68> As Windows was probably installed on C it probably didn't do any real damage, did it?

<comment><USER69> If I remember correctly, that was before Windows started protecting boot volume.

<comment><USER70> To be more precise, NT had file locking enforced by the OS unlike DOS.

</comment></comment></comment></comment></comment></comment><comment><USER71> This tip only applies to interactive shells: I often prefix potentially dangerous commands with '#' while I edit them. Tab-completion still works (with bash at least, so I assume also with certain other vastly superior shells).

        # rm -rf /path/to/^I

</comment><comment><USER72> Reminds me of some slackware install script back in 1999 or so, I think it was for the gnome documentation.

Anyway, it accidentally removed rm-rf'ed /, I only discovered it in time because it gave errors about removing nodes in /proc...

</comment><comment><USER73> (I'm not a Linux user, forgive my ignorance)

Is there no way to prevent bugs like this at the source, by modifying Linux, rather than hoping there isn't any extra white space in a command that might delete your usr directory?

</comment><comment><USER74> Once we got catastrophic backup

rsync -a --delete /home/project/ /mnt/backupDisk

Left unnecessary slash after "project" and rewrite all content on "backupDisk" by project files (instead of sync project folder on it)

</comment><comment><USER75> [http://www.google.com/codesearch#search/&q=%22rm%20-rf%2...](http://www.google.com/codesearch#search/&q=%22rm%20-rf%20/%20%22&type=cs)

</comment><comment><USER76> linux can still boot without a /usr. this was a commit; how many people ran it?

on the other hand, delete the boot.ini[1] and most windows systems cant even boot. now deploy that boot.ini deleting build on an MMO (eve online) and watch the fur fly. that, ladies and gentlemen, is how you earn a :golfclap:.

[1] <http://google.com/search?q=ccp+boot.ini>

</comment><comment><USER77> This would be a lot funnier if there was anything preventing me from doing exactly the same thing... ouch.

</comment><comment><USER78> That's pretty heinous, but deleting /usr shouldn't cause you to lose any irreplaceable data.

<comment><USER79> From the FreeBSD handbook ( <http://www.freebsd.org/doc/handbook/users- introduction.html>):

        The home directory is the full path to a directory on the system in
        which the user will start when logging on to the system. A common
        convention is to put all user home directories under /home/username
        or /usr/home/username. The user would store their personal
        files in their home directory, and any directories they may create
        in there.

All of my irreplaceable data is stored in `/usr'. `/home' is a symlink to `/usr/home', as created by the installer.

</comment></comment><comment><USER80> Oh man.

an RPM I had at one point had / as one of its trees. There's a reason I moved to debian.

</comment><comment><USER81> I wonder if this is what the founders of github expected from "social coding"?

</comment><comment><USER82> That poor guy. But I appreciate the reminder to double-check any rm command.

</comment><comment><USER83> That's quite a big bug. :/

</comment><comment><USER84> and it was 666 points a second ago. symbolic.

</comment><comment><USER85> glad it wasn't me..

</comment><comment><USER86> A decade ago, I worked on a DNA sequencer / aligner product. This produced easily 1GB+ raw data files, and they typically exploded by a factor of ten by the time you performed a bunch of cleaning, smoothing, filtering, etc on them. For several reasons, not least of which was a 4GB file size limit in fat32, this software had to use a directory as a pseudo file.

I was working on some file saving logic. A customer had a problem where they'd overlaid a new logical file on top of an old logical file. Where these actual files, this would just have overwritten the old file, but since these were directories, we got a mishmash of pieces of two different logical files overlaid in the same directory, and of course our software got confused as hell. So, I wrote code that, in case you saved a new file as an extent filename (really directory name), would perform the equivalent of

      rm -rf $dirname; mkdir $dirname;

You can see where this is going... Some grad student didn't understand this, and named a pseudo file as the root directory of a tree of research. Two years of research vanished into the ether, despite a dialog box that had red text in it. That sucked.

<comment><USER87> It is crazy not to have backups of research data.

</comment></comment><comment><USER88> <rm -rfi> -i interactive mode man is here to save the day - now you can put up to 50% more blame on the end-user!

</comment><comment><USER89> I hate languages with significant whitespace

<comment><USER90> The "significant whitespace" here is a space between command-line arguments. For that to qualify, you'd also have to include C as such a language.

      long jmp();
      longjmp();

<comment><USER89> c'mon everyone, I was being facetious :)

</comment></comment></comment><comment><USER91> Das ist nein gut.

</comment><comment><USER92> lol, f*ckups don't get any bigger than this

</comment><comment><USER93> OHHHHHHHHHHHH wow .... great link, passed around the office for a few laughs ... this is one of those heart sinking moments ....</comment>


Some AMD CPU's RDRAND might not return random data after a suspend/resume - <AUTHOR> https://github.com/systemd/systemd/issues/11810#issuecomment-489727505 <comment><USER2> RDRAND is not guaranteed to always succeed (and never was). You're supposed to retry on failure.

(Although linked in that thread systemd code has fallback anyway, so I'm not sure how it fails at all).

 _Edit_ : not to mention that it's better just to not use it, ever. Quite sane and sensible thing.

<comment><USER3> Sounds like a bit of code which was hard to trigger the negative test and therefore the fallback failed to work properly.

Not sure how the kernel devs generally go about testing the "this virtually never happens" code paths without adding debug switches to every unhappy path.

Certainly I doubt they are using DI/IoC to wrap an interface to RDRAND which allows unit testing the failure modes.

At least the result is a failure to generate a key, not a compromised key.

<comment><USER4> > Sounds like a bit of code which was hard to trigger the negative test and > therefore the fallback failed to work properly.

No; it turns out that's giving systemd too much credit (sadly). See [1].

The problem appears to be that RDRAND _was_ signalling success, but producing a nonrandom value. This is bad and a violation of the specification.

Can't speak to Linux kernel development, and in this particular case, that isn't the problem.

The linked bug involves systemd using the world's worst random number generator. A security engineer goes into more detail on this twitter thread[1]: [https://twitter.com/FiloSottile/status/1125840275346198529](https://twitter.com/FiloSottile/status/1125840275346198529) (or unrolled: [https://threadreaderapp.com/thread/1125840275346198529.html?...](https://threadreaderapp.com/thread/1125840275346198529.html?refreshed=yes) ).

> At least the result is a failure to generate a key, not a compromised key.

In fact, the result _is_ a compromised key -- the bug report is due to colliding "globally unique" identifies generated through a flawed random gathering process.

</comment></comment></comment><comment><USER5> Is there a good reason this doesn't just use urandom/getrandom? This doesn't look like it's in the "we're so early in boot I haven't restored the random seed yet" case, for example.

<comment><USER6> No. There's a bad reason though. Want to guess what? [https://github.com/systemd/systemd/blob/master/src/basic/ran...](https://github.com/systemd/systemd/blob/master/src/basic/random- util.c#L83)

Yes, the old "draining entropy" fallacy.

<comment><USER7> Why do you say draining entropy is a fallacy? It is certainly true that entropy recorded from I/O sources accumulates at a very limited rate.

<comment><USER8> Because after you gather ~256 bits of entropy from I/O sources (or rdrand^H^H^H rdseed [1]), using it to generate infinite output of /dev/urandom does not drain it. Adding more entropy only helps if the entropy pool inner state leaked, it is not needed to add more entropy because it drained through using it.

Proof:

1\. Take 256 bits of entropy.

2\. Use HKDF to generate 128 bit key and 64 bit nonce.

3\. Use key and nonce for AES-CTR with all possible 2^64 counter values to produce 2^68 bytes of output.

4\. goto 2.

If you can compute the inner state of the RNG (and thus predict future output) by just observing the output (not through side channels), using any amount of output, then all modern symmetric crypto is broken. If you can't, then using the entropy in an RNG does not drain the entropy pool.

And here is a CCC talk about it: [https://www.youtube.com/watch?v=OSfmtRc4VsE](https://www.youtube.com/watch?v=OSfmtRc4VsE)

1 - [https://software.intel.com/en-us/blogs/2012/11/17/the- differ...](https://software.intel.com/en-us/blogs/2012/11/17/the-difference- between-rdrand-and-rdseed)

EDIT: Thanks for 'dragontamer for pointing out difference between rdrand and rdseed.

<comment><USER7> So after reading this thread my understanding of the "draining is a fallacy" view is

(1) yes genuine non-programmatic entropy bits accumulate at a slow rate and can be drained

(2) in practise no one should care about (1) because once you have ~256 bits to initialise a CSPRNG you can use the output of that CSPRNG until the Earth is swallowed up by the sun; thats what the S of Cryptographically Secure Programmatic Random Number Generator promises.

(3) the linked systemd code is silly to even provide a function genuine_random_bytes that tries to get additional genuine non-programmatic entropy as every possible use case is covered by (2)

(4) The real fallacy is when people think that it might be possible to discover the seed of a CSRNG or predict its next outputs by examining a long enough run of its previous output. In practise such an attack should not be possible.

If I've got the summary right, the only point I would disagree on principle is (3). I can imagine that someone might rationally want "genuine" entropy independent of a kernel CSPRNG, for example for seeding their own CSPRNG.

<comment><USER8> re your (1), the bits cannot be drained. The only thing that can happen to them is that there is an attacker with root on your computer, and they see the CSPRNG inner state, and the attacker loses their access but they can still predict output of CSPRNG because it's deterministic, so you want to inject new entropy into it so the attacker will lose ability to predict CSPRNG output. djb says this is nonse. [https://blog.cr.yp.to/20140205-entropy.html](https://blog.cr.yp.to/20140205-entropy.html)

</comment></comment></comment></comment></comment></comment><comment><USER9> [https://news.ycombinator.com/item?id=6336505](https://news.ycombinator.com/item?id=6336505)

<comment><USER10> For the lazy, this is a link to a comment by Theodore Ts'o, kernel dev, who says:

> I am so glad I resisted pressure from engineers working at Intel to let > /dev/random in Linux rely blindly on the output of the RDRAND instructure. > Relying solely on an implementation sealed inside a chip and which is > impossible to audit is a BAD idea. Quoting from the article...

Theodore Ts'o is maintainer of the ext filesystems (particularly ext4), as well as, IIRC, /dev/random and other CSRNG related components of the kernel.

Thank you, Theodore Ts'o.

<comment><USER11> Theodore Ts'o is responsible for perpetuating the myth that entropy/randomness can run out, leading systemd (and other software) to do crazy things, such as trying to use RDRAND, to avoid "drain[ing] randomness from the kernel pool". The bugs and security vulnerabilities resulting from this myth probably neutralize the benefit that came from from his resistance to RDRAND in the kernel.

<comment><USER12> > the myth that entropy/randomness can run out

Can you expand on this, or link to some sources that expand on this idea that the assumption above is wrong? As a person who has not dealt with crypto really at all I had heard this explained several times before and assumed it was generally accepted.

<comment><USER13> 128-bits of random data is sufficient to securely generate a stream of 100s of terabytes of random data. It's not /that/ hard to find 128-bits of true entropy, even during boot phase. Here's one example:

        1. Seed with any fixed hardware IDs

        2. Mix-in the wall clock time

        3. Spin up a kernel thread and flip a bit on/off in a tight loop. Interrupt it every 100 nanoseconds and take the value of the bit at that time. Do this 256 times. Mix that in too.

        4. Mix-in 256-bits from RDRAND

        5. Mix-in timings from other interrupts as and when they happen.

        6. Repeat steps 4. and 5. ad infinitum.

By step 4 we have taken 26 microseconds and we have the kind of entropy I would be comfortable generating an RSA private key with.

Note that step 3 is effectively a measure of how precise the system clock and CPU are. Attacks have been demonstrated against step 3, but they require co- resident processes and don't apply during the boot-phase, if you've got a dedicated core at least. In theory if system clocks and CPU got super precise it could become too deterministic, but the point is the likelihood of /both/ that happening /and/ RDRAND being broken.

<comment><USER14> > 128-bits of random data is sufficient to securely generate a stream of 100s > of terabytes of random data.

What you are describing is /dev/urandom. Your argument is basically "urandom is good enough for anybody". If you want to use that, use it.

<comment><USER15> /dev/urandom is not always sufficiently seeded.

/dev/random makes sure that it's seeded, then pretends it can run out somehow.

getrandom() with default settings is the right behavior almost always, and it took ages to get implemented.

</comment></comment></comment></comment></comment></comment></comment><comment><USER16> Note that the corresponding kernel bug was reported already in 2014.

[https://bugzilla.kernel.org/show_bug.cgi?id=85911](https://bugzilla.kernel.org/show_bug.cgi?id=85911)

</comment><comment><USER17> Later in the thread:

> Given that RDRAND is allowed to fail, it seems to me that you should either > try it only once, or only a few times, before falling back to whatever code > is used when RDRAND is not implemented.

<comment><USER18> ... which is what the systemd code actually does. The problem is that there appears to be a possible AMD processor state, caused by suspend+resume, where the instruction _succeeds_ but the data returned are not in fact random.

<comment><USER17> There is no mention anywhere, in this thread nor the one from 2014, of it returning non-random data from the issue reporters, just assumptions from onlookers.

<comment><USER18> Wrong.

* [https://news.ycombinator.com/item?id=19851693](https://news.ycombinator.com/item?id=19851693)

</comment></comment></comment></comment><comment><USER19> Oh, this could be really bad. I wonder if any private keys are compromised this way - would certainly be nice to know what rdrand is returning if it isn't random data.

<comment><USER2> Catchy title. Nowhere it says it returns non-random data. It just fails.

<comment><USER20> Sure it does... read all the posts. Also read all the history.

Also read this from 2013 " I am so glad I resisted pressure from Intel engineers to let /dev/random rely only on the RDRAND instruction... Relying solely on the hardware random number generator which is using an implementation sealed inside a chip which is impossible to audit is a BAD idea. "

And this [https://www.theregister.co.uk/2013/09/10/torvalds_on_rrrand_...](https://www.theregister.co.uk/2013/09/10/torvalds_on_rrrand_nsa_gchq/)

So, uh, this isn't news, and isn't limited to AMD.

Sure it'd be nice to fix.

<comment><USER21> I fail to see how a dev talking about a wholly unrelated implementation from an entirely different vendor has anything to bear on this conversation unless you're just maliciously spreading FUD around the entire concept of random numbers.

Do you actually know if the instruction is faulting or delivering back non- random data ("sure it does...")? Is the non-random data 0's or something with a pattern like 0x9090? Does that match the Intel implementation's behavior exactly?

</comment></comment></comment></comment><comment><USER10> And this is why cryptography has increasingly reduced its reliance on randomness. Strong CSPRNG that only need a single seed to be secure, signing constructions that use deterministic hashes, deterministic derived keys, DAE- secure ciphers that fail-safe when IVs are re-used, etc.

Randomness is definitely something we took for granted for too long.

</comment><comment><USER22> Suspend/resume seems to be the cause of a whole host of bugs; I run into obnoxious suspend problems frequently on all platforms except maybe Windows. It's so common that I've pretty much stopped using suspend on most of my machines. It's a bit inconvenient to power down / power up things each day, but I'd rather deal with that than have intermittent wifi issues, display problems, etc.

<comment><USER23> I consulted once for a hospital that was experience seemingly random network outages. After a few questions with the staff there it seemed to happen whenever a person stepped away from their windows (Lenovo) workstations for too long. Say, after lunch or breaks. After a wireshark of the network I determined it was a network card driver that was causing a broadcast flood on the network from multiple points for stations with the same driver version. While not the fault of Windows (I dont think anyways, as an update of the driver fixed the issue) your comment did remind me of this experience.

<comment><USER22> Bet the vendor made sure the drivers "worked" for suspend/resume, but only from the user point of view.

I may have just been lucky with Windows on my particular hardware. A quick search turns up a bunch of different problems. I think suspend/resume may just be a particularly difficult thing to get right.

</comment></comment></comment><comment><USER24> Wow, so when would be the worst time to suspend your computer?

The only time I imagine this, is when generating a private key for a production environment.

<comment><USER25> My reading is that systemd uses the cryptographically secure rng to generate a unique id for a filename, and doesn't handle collisions properly.

 _sigh_

<comment><USER26> There shouldn't be collisions. I mean that really: if you see a collision it's so much more likely that your computer / program / source of randomness is faulty [as in this case] than that the two random numbers collided that it's not worth considering the collision case.

<comment><USER27> But there can be collisions, at least in theory. And as we can see, in practice too.

Not checking for collisions because you trust that an RNG returns unique values is a fallacy and should be avoided. Especially since it's so easy to deal with the problem.

<comment><USER28> It would be better to report the collision because most likely it means some error.

<comment><USER27> Yes, of course. As long as you do something. Just ignoring it because it's assumed it'll never happen is just bad design.</comment></comment></comment></comment></comment></comment>


Whites earn more than blacks – even on eBay - <AUTHOR> https://www.washingtonpost.com/news/wonk/wp/2015/12/11/whites-earn-more-than-blacks-even-on-ebay/ <comment><USER2> "In addition, the cards that were held by the African-American hand actually ended up being worth more, suggesting they should have sold for more than the other batch. That is, when the researchers added up how much they had originally paid for all of the cards sold by the black hand versus the white hand, the first total was larger."

That's an obvious logical fallacy - how much the seller _paid_ for the item may not be relevant to how much it is _worth_ much later.

For example, not so many years ago, Apple 1s were worthless e-waste.

</comment><comment><USER3> Socio-economic perceptions and signaling could still be at play here. Does someone not buy the cards because the person is black, or are they less likely to buy the card because they worry that it might not get delivered, or that it might be stolen.

I wonder if you'd see the same effect for a women's hand as a man's hand.

I also wonder how many fewer bidders you would need to show this effect. In small illiquid markets the loss of a couple of bidders could put more downward pressure on the top bid.

The part that was most convincing that the authors identified the correct effect though was that the price was also affected by the race of the person on the card.

Looks like there is an opportunity for racial arbitrage.</comment>


Steam Inventory Helper Now Requesting EVERY SINGLE HTTP Request - <AUTHOR> https://www.reddit.com/r/GlobalOffensive/comments/70xofs/warning_trusted_steam_inventory_helper_now/ <comment><USER2> For those like me who have no idea what this is about, here's the Chrome Web Store description.

Steam Inventory Helper: Provides many utilities for steam inventory and market (offers notification, quick selling, buying, offering trades, price check...)

</comment><comment><USER3> The title as submitted to HN is a bit misleading. Should be "[...] monitoring every HTTP request"

</comment><comment><USER4> At least they've stopped running "rm -rf /" on user machines ([https://github.com/valvesoftware/steam-for- linux/issues/3671](https://github.com/valvesoftware/steam-for- linux/issues/3671)). So it's a step in the right direction.

<comment><USER5> This isn't an official Valve extension.</comment></comment>


Python decorators: A neat way to modify functions - <AUTHOR> http://kember.net/series/python-on-the-job-what-did-i-miss/decorators-a-neat-way-to-modify-functions <comment><USER2> It is a crying shame that this article fails to realize that decorator.decorator is itself a decorator, and that failing to use it as such undermines the premise that decorators are "a neat way to modify functions". With this realization:

        @decorator.decorator
        def heading1(func, *args, **kwargs):
            return "<h1>" + func(*args, **kwargs) + "</h1>"

With this construction, you no longer need to have the two copies of the function: the degenerate wrapper can be removed and replaced with this decorator.

Even if the library didn't provide this, you can just look at this code and feel "no, wait, this is wrong: this should be implemented with a decorator" and write this:

        def better_decorator(function):
            def wrapped(function, *args, **kw):
                return decorator.decorator(function, *args, **kw)
            return decorator.decorator(wrapped, function)

This new decorator, @better_decorator, can now be used in place of @decorator.decorate in the above example, and again solves the problem of the degenerate wrapper by generalizing it into a single universal degenerate wrapper.

<comment><USER3> A crying shame indeed. Thanks for taking the time to point that out and provide such detailed criticism - much appreciated. I've updated the article and credited you.</comment></comment>


Falcon Heavy core booster lost at sea - <AUTHOR> https://techcrunch.com/2018/02/06/spacex-landed-two-of-its-three-falcon-heavy-first-stage-boosters/ <comment><USER2> I kind of figured that happened when they lost the video. Still a really amazing accomplishment--the video of the boosters landing was like something out of a sci-fi movie. It's the best space launch since the first Space Shuttle in 1981.

<comment><USER3> In a press conference an hour or so ago, I think Elon mentioned that, if he had to pick one core that would experience "rapid unscheduled dissassembly", it would have been the center one. I think he mentioned it was due to the boosters having the expensive titanium grid fins and also being reusable for future flights.

Regardless, it was absolutely wonderful to watch! I think the Youtube stream had 2,500,000 viewers at one point. I can't wait for more footage of the launch the coming days.

<comment><USER4> The unspoken point in Musk's comment is that SpaceX doesn't actually care if they get these particular rockets back. They want to prove reusability, of course, but no matter the outcome they never plan to re-fly these cores. They will focus on the not-yet-flown Block 5 cores, and standardize on that going forward.

In this flight, the side boosters and center core were Blocks 3 and 4. So SpaceX definitely wants to recover the boosters, to perfect reusability, but they never intend to re-fly any of these boosters, so better to get back the one with the expensive and reusable titanium grid fins.

<comment><USER5> Didn't they say these boosters had already flown before?

I'm also a bit surprised the grid find are so pricey, they seem pretty small? Not that I really know how much titanium costs to buy and machine.

<comment><USER6> They measure 4 by 5 feet, so we can estimate they are probably made from a solid chunk of titanium that was at least 20 cubic feet or almost 5,000 pounds. That would be nearly $150,000 in raw material and probably another 5x that in processing (guesstimating cost per cubic inch of titanium removal).

Either that or they were specially made castings, which are similarly expensive in low volume, if not more.

<comment><USER7> Wouldn't you be able to recover most of the removed titanium, amortizing the majority of that guesstimated materials cost? Processing likely remains expensive, however.

<comment><USER8> I'd think so. They also 3D print a lot, but metal powder does seem expensive.

<comment><USER9> Printed metal has huge issues with heat, and they just switched from aluminum gridfins to titanium because of the heat. I seriously doubt that they're usung additive techniques here.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER10> > But the core, middle booster, which attempted to land aboard "Of Course I > Still Love You," a drone barge that SpaceX uses as a mobile, ocean-borne > landing pad stationed in the Atlantic for its flights departing from > Florida, wasn't recovered.

"Of Course I Still Love You" is the name of a spacecraft in The Culture, a series of novels that inspired Elon Musk when he was young.

<comment><USER11> For anyone who has not yet read of the books from The Culture series, I highly encourage you to do so. They are an amazing - agnostic - version of what our far future selves could be. Iain M. Banks had quite the mind (no pun intended).

<comment><USER12> Yes, great books! I recommend starting with 'The Player of Games', not the first book in the series but the best. They are only loosely connected, you won't miss anything.

<comment><USER13> I started on 'Consider Phlebas', the first book in the series. I recommend starting with that, it's a great book, perhaps not as accessible as 'Player of Games', but more wow-factor.

</comment></comment></comment></comment><comment><USER14> I absolutely love the fact that SpaceX is so open and public about their rocket launches, however, I couldn't help but get the feeling that they were in full 'PR backlash' mode [over the core booster] when ending the live stream, and from skimming many of the instantaneous press releases.

Does anyone else think that they were overly quiet on the core booster landing on purpose, in order to minimise negative PR on the whole operation; or was it simply that they did not have enough information on what happened (seems slightly implausible to me)?

This all being said, what happened today was nothing short of magic.

<comment><USER15> Agreed, at the very end of the feed actually, the 2 presentators didn't really know what to say about the status of the core booster. A space X employee showed up (I assume a PR or an engineer previously briefed by PR, all he had was a SpaceX polo). The guy essentially said: 'best test ever, everything went smooth yadi yadi yada, oh and also, we gotta figure out about that last booster. Otherwise, everything went perfect !"

Which is very true. But it was clearly a move to divert away the audience from that last booster. Which is understandable. it'd be a shame if everyone focused on a lost booster when the everything else was nothing short of magic like you say ;)

Congrats spaceX, guys guys are magicians.

<comment><USER16> "A space X employee showed up (I assume a PR or an engineer previously briefed by PR, all he had was a SpaceX polo)"

I think you're talking about John Insprucker: [https://twitter.com/jinsprucker?lang=en](https://twitter.com/jinsprucker?lang=en)

He's a senior engineer who was present for the whole cast. Look at it again.

</comment></comment></comment><comment><USER17> they also re-uploaded the webcast video with feeds from both booster cameras, I also didn't remember seeing the fairing separation so well when watching it live

[https://www.youtube.com/watch?v=bCc16uozHVE](https://www.youtube.com/watch?v=bCc16uozHVE)

</comment><comment><USER18> SpaceX hasn't failed a landing of a booster in two years (correct me if I'm wrong). I hope a lot of data was collected as to why and how it failed.

SpaceX's biggest advantage is that they iterate quickly. Hopefully next iteration fixes whatever problem they had today. Those boosters aren't cheap.

<comment><USER19> _Musk attributed it to low levels of triethylborane—a hot-burning propellant that essentially acts as rocket engines ' starter fluid—in the center booster._

The center core is a special structurally enhanced model, so I'll give them a pass since they're flying a new bird. I'm sure they'll get it right.

<comment><USER18> That makes a lot of sense. The center core had to ignite for liftoff, then either scale back or stop entirely during the main lift phase, then re-ignite when the side boosters detached, then reignite for the boostback, entry AND landing burns. That's a lot of re-ignition!

I hope they post a very high quality video of that booster having its unplanned disassembly. May not be great for PR, etc, but it sure would be entertaining.

</comment></comment></comment><comment><USER20> > Eventually, Musk hopes to be able to turn rockets around in less than 24 > hours and have reflown boosters go up twice in one day.

This will be amazing.

<comment><USER21> This won't be a huge flashy milestone when it happens, but future historians will point to that achievement as an inflection point in space travel.

Remember a few short years ago when humans always threw away the rocket after every trip?

<comment><USER22> Agreed. It's the Henry Ford moving assembly line of rocket launches. It's a game changer.

</comment></comment></comment><comment><USER23> I'm still impressed beyond words that they were able to glue basically 2 reused rockets onto a 3rd rocket and then not only have a successful test flight but return 2 of the 3 boosters.

<comment><USER24> Fun fact: there is a pneumatic mechanism that gives the side boosters a gently push away at the top, with the bottom attachment point separating milliseconds later. This is to cause the side boosters to fall away from the center core properly.

</comment></comment><comment><USER25> Yes, I remember this. Had stayed up all night to watch the launch. After the side cores landed (which was mind blowing honestly), the cameras switched to drone ship where the center booster was supposed to land. The only thing I could see was mist & fog, and they switched to their commentators 2 seconds after that. Thought I must have not noticed the successfully landed core, & went back to sleep dreaming of the cool life Musk lives.

</comment><comment><USER26> _> and in the live stream SpaceX provided of the launch, you could hear someone say "We've lost the core" but it wasn't clear whether that indicated just the feed, or the booster itself._

That's definitely not set-speak for 'video feed' but I wish it was.

</comment><comment><USER27> And the article does not say / make apparent where this information is from.

<comment><USER28> There was just a press conference where he went into some detail, with the caveat that it's all still very early information.

Basically he said that 2 of the 3 engines didn't start up correctly, and caused the booster to hit the water at a few hundred miles per hour.

I'm guessing the dive into the water was purposely missing the barge when it detected the problem, as they have spoken about this capability in the past when they were first making attempts at landing the rockets.

He also said that as long as they have the footage, and it's cool looking, they will post it somewhere!

<comment><USER29> I previously thought that the middle core on a Falcon Heavy was expendable. So it not landing (while needing 3 engines) makes sense, as it was only a week or two ago that they tried a fast/hard landing with 3 engines over open water (as they weren't sure then that it would work). I would expect this landing mode to need at least a couple more tries to get working reliably.

<comment><USER28> Well that test was unrelated. They only reason they tried that with an ocean landing was because they wanted the barge for the FH demo flight.

This should have been a 1-3-1 style landing burn where it lights up one engine, then 2 more to slow it down, then drops down to just 1 again at the end. In this case it sounds like the the outer 2 engines didn't start up right, so it came in way too hot.

They also have said that the center core of FH is basically a new design from the ground up, so it makes sense that they had some problems on it's first landing.</comment></comment></comment></comment>


How to make Lisp go faster than C [pdf] - <AUTHOR> http://www.iaeng.org/IJCS/issues_v32/issue_4/IJCS_32_4_19.pdf <comment><USER2> This is typical for papers like this: take a toy algorithm, show it's kinda as fast as C/Fortran if you do something really odd, then declare victory. This works better if your C compiler lacks important optimizations, like a loop nest optimizer.

If you really want to impress people, take a significant algorithm, preferably one of the SPECcpu benchmarks, write the whole thing in Lisp, and then see what happens.

<comment><USER3> It's not just Lisp. Language advocates are the snake oil salesmen among computer scientists. Case in point: [http://www.reddit.com/r/programming/comments/bnnoh/fast_auto...](http://www.reddit.com/r/programming/comments/bnnoh/fast_automatically_parallel_arrays_for_haskell/c0npq4j)

<comment><AUTHOR> >It's not just Lisp. Language advocates are the snake oil salesmen among computer scientists.

It's true. (-: I submitted the paper mostly because it's a condensed "how-to" for optimizing Common Lisp code. I think the name of the paper itself is really, really sketchy.

</comment></comment></comment><comment><USER4> I admit to not having read the paper all the way through, but Figure 2 only shows Common Lisp as being noticeably faster than C in one instance, and I suspect that's due to an anomaly of some kind (it's totally out of line with the other C speeds).

</comment><comment><USER5> To those who know the SBCL compiler better:

The paper says that to get the speed they had to turn off run time type checking, but they say "here's how to do it globally" does that mean you can turn of type checking in just one module or function?

<comment><USER6> Yes. The Common Lisp spec says you can put declarations a lot of places, including prog:

<http://clhs.lisp.se/Body/s_declar.htm>

So you can basically wrap it around any S-expression you want.

When I was at ITA we did that a lot. For example, there was a macro i+ that would expand to basically "the two arguments are fixnums, as is the result. Now, please give me the sum."

In debug builds, the type declarations would be come assertions. In release builds, they'd become assumptions, i.e. the whole thing would compile to a single x64 ADD instruction.

</comment></comment><comment><USER7> Why is the scribd version 1) ugly and 2) in Flash? I thought they were all HTML 5 these days, and I'd clicked the scribd link to give them another chance.

</comment><comment><USER8> barely readable

<comment><AUTHOR> What do you mean? Is it the layout, the content or something else? I was able to read it just fine.

<comment><USER4> The scribd version is pretty bad.

<comment><AUTHOR> Ah, you're right, that is bad. I had only looked at the raw PDF.

</comment></comment></comment></comment><comment><USER6> gcc 4.0.3 vs. CMUCL, the predecessor to SBCL.</comment>


Python-based static site/blog generators? - <AUTHOR>

Looking for something like Octopress in Python.  Of course, I came across a number of options with a simple google search:
http://gistpages.com/2013/08/12/complete_list_of_static_site_generators_for_python
Not included in this is Nikola, which looks interesting:
http://getnikola.com/
I am looking for feedback from first hand experience with any of these (or others).
Why Python?  Because I am working with Python for a number of other projects and I'd like to stay in that domain if at all possible.  Avoiding PHP for the same and other reasons. <comment><USER2> I've been using Pelican for over a year and have been pleased. It's mature, but still has active development and a strong community. A few major features:

      * markup languages (reStructuredText, Markdown, etc.),
      * code highlighting via pygments
      * themeable
      * simple to use
      * support for automated deployment

[http://blog.getpelican.com/](http://blog.getpelican.com/)

</comment><comment><USER3> I use Hyde and it works great for me.

[http://ringce.com/hyde](http://ringce.com/hyde)

My webpage uses it, feel free to steal everything besides the content:

[https://github.com/stucchio/homepage](https://github.com/stucchio/homepage)

<comment><AUTHOR> Hide does look interesting.

Nice blog BTW. Any issues controlling comment spam?

<comment><USER3> Nope, I'm using standard free disqus and I rarely see any spam.

</comment></comment></comment><comment><USER4> This works well if you want to start from a dynamic application: [https://github.com/SimonSapin/Frozen- Flask](https://github.com/SimonSapin/Frozen-Flask)

<comment><AUTHOR> I'm mostly interested in writing a series of programming and hardware design tutorials for my kid (and anyone else who might find them useful). I don't think I need anything fancy for this. Images, code with syntax highlighting and a few things. Not even sure if I want comments. He's been learning Python, so I'll use it as a way to expand his application of the language.

The rough curriculum I have in mind at the moment is:

        - Basic electronics concepts
        - Digital logic
        - Design a simple 8-bit microprocessor
        - Program it using the custom instruction set
        - Implement basic mass storage in Flash (perhaps via USB)
        - Implement a Threaded Interpretive Language (Forth)
        - Write a text editor using Forth
        - Design a number of I/O peripherals
            - 7-segment display
            - A/D and D/A
            - Dot matrix LED display
            - Various sensors
        - Build something useful, like a clock driving the n x m
          LED display

<comment><USER5> This curriculum sounds really cool!

Be sure to post your progress to HN!

</comment></comment></comment><comment><USER6> I am using tinkerer [http://www.tinkerer.me/](http://www.tinkerer.me/), its very simple to change and manage and uses well known sphinx-doc with jinja templates. I like it a lot you can check my live site at [http://www.vyomtech.com/](http://www.vyomtech.com/), since it uses boilerplate html 5 template its very easy to customize.

</comment><comment><USER7> I use mynt. [https://github.com/Anomareh/mynt](https://github.com/Anomareh/mynt) It is much simpler compared to hyde. The reason I chose mynt above other similar options is that, it allows for custom urls that made migrating my previous wordpress blog posts easier without having to resort to .htaccess magic.

<comment><AUTHOR> Ah, that's important. I need to look at the other options and see if they all support custom URL's without .htaccess magic, as you put it.

</comment></comment><comment><USER8> I built and use djangothis[1] for my blog. It gives you most of the power and flexibility of django, like templates, arbitrary urls and views [which gets converted to static html], templatetags and a few other niceties.

[1]: [https://github.com/amitu/djangothis/](https://github.com/amitu/djangothis/).</comment>