Postmark introduces easy inbound email processing for all accounts - <AUTHOR> http://blog.postmarkapp.com/post/15687406657/introducing-postmark-inbound-easily-parse-replies-other

<comment><USER2> Some feedback: This is a nice addition to a mail-provider API service, but the way it works is way too complicated. All email I want to parse needs to be forwarded to my special postmark inbound email address? This might be nice for people that have their email setup through google apps (which kindly forwards on all headers, etc when an email is auto-forwarded with a filter), but for other mail systems they might not play so nice with forwarding headers, etc (I know this from running my own email servers for Notifo and dealing with incoming and outgoing email on my own).

With this solution I still have to setup an intermediate mail server solution to first receive the emails from users, and then forward it along. Can't you take that step away from me?

Other providers (mailgun, sendgrid, etc) let me do that and let users send emails directly to my inbound mailboxes (which are then POSTed to my endpoint).

Also, this excerpt from the documentation lowers my confidence that you know what you're doing:

"Rather than leaving this information in plaintext, we recommend using SHA1 or MD5 to encrypt the information after the + sign and include the encrypted value after the +. This will avoid unintentionally exposing any sensitive information."

<comment><AUTHOR> Thanks for the feedback, Chad.

Regarding forwarding: that step is already being taken away - it's in beta for a number of our customers, and we'll be adjusting workflow then releasing to everyone.

There's lots of people who don't have access to their DNS records but still want a tool like this. We want this utility to be simple for more them, too!

As far as SHA1/MD5 encryption - we know better than to use weak encryption on any TRULY sensitive information. In the context of the walkthrough, we're talking about internal IDs and other things that you might want to be obvious to your customers, not things that pose a security risk to you or them. If you even thought about using something like that as part of a unique identifier in your email replies - I think there would be bigger problems at hand than choosing SHA1 or MD5 to obfuscate that information. :)

<comment><USER2> Thank you for the reply!

Regarding the md5/sha1 stuff: The documentation as it stands is both wrong and misleading. MD5 and SHA1 are _hash_ functions (not _encryption_ functions), so while trying to suggest that people could encrypt information to put after the + sign is a good idea, you will utterly confuse newbies by giving examples of hash functions. Of course you could use a unique hash value after the +, but then you need a reverse hash lookup table in your app to correlate the hash and the internal app state/data you need (but that is a whole other beast and not nearly as simple as "just encrypt it in the to address, decrypt on receipt, et voilà!").

In the end it is confusing for beginners and outright wrong for people that know the difference.

Now for the constructive part of the criticism: Please change or improve that portion of the docs, since docs are arguably the second most valuable asset to an API-provider (right behind the service itself).

</comment></comment></comment><comment><USER3> Wouldn't it be great if there was a standard JSON representation of an email, and that multiple providers (SendGrid, AWS SES, MailJet, ElasticEmail, etc) all then provided such an interface.

So not only could you choose (swap out very very quickly) your outbound provider, but also your inbound provider.

After the thread the other day on outbound email, I ended up choosing MailJet who have been wonderful with the questions I had when I hit their new member ceiling within the first few hours.

The things that convinced me on MailJet were real-time outbound stats and meaningful data (recipient + subject), API of status notifications (who bounced), and the fact that they do campaign emails at a cost so low that MailChimp ruled themselves out (one of my email lists has over 23,000 members).

I need inbound handling (currently using /etc/aliases and piping), but would love all the reporting and info which centralisation from an external provider could deliver.

My only concern would be resilience. Email is really good at re-delivery when your end is down or something prevents delivery, would a HTTP REST interface offer the same level of re-try and timeouts to ensure it got through eventually. Would the provider queue for me?

<comment> old-gregg Everything you're asking for (and much more) is handled by Mailgun (full disclaimer - I work there):

<http://blog.mailgun.net/day/2011/11/07>

Moreover, our incoming mail parsing doesn't stop at simple MIME parsing, we're dig deeper into the meaning of those characters and extract user signatures, quoted parts, etc.

Moreover, we retry inbound POST, allow you to filter incoming traffic on the server and (very important) we're the only incoming mail API processor that generates proper bounces for 3rd party servers (spammers) trying to hammer you with non-existent addresses.

<comment><USER4> We use Mailgun at UserVoice.com for incoming mail.

Of particular help to us:

\- All emails are converted to UTF-8

\- There's always a 'plain' (text only) version of the email sent to us, even if the original email only included HTML.

\- They offer a 'stripped-text' field, which lets us easily see the new part of an email thread.

\- Pretty great support

</comment></comment></comment><comment><USER5> $1.50/1000 seems steep and could hurt should a spam attack occur...Someone needs to package charged outgoing, with free incoming. Kinda like the cell messaging systems they have in the UK...

Remember, you can add value by combining valuable services. Sell one as a loss leader to bring in the users and the other is your revenue stream.

<comment><USER6> You could always use Postmarks spam checker service to monitor emails and if you get say 2 or 3 from the same email address that flag up as spam block the email?

<http://spamcheck.postmarkapp.com/>

<comment><AUTHOR> We actually include spamscore information in the parsed emails.

[http://developer.postmarkapp.com/developer-inbound- parse.htm...](http://developer.postmarkapp.com/developer-inbound- parse.html#spam)

Also, if you use Gmail forwarding to foward emails into Postmark, you get the benefits of their spam filtering before it even hits our system.

That said - we watch for spam-like activity across our entire system.

<comment><USER7> Assuming I could craft messages which did not look like spam (to your system), and I could do it really really fast, do you have any throttling options to allow your customer to prevent an excess of mail from adding up too quickly or an alert based on a threshold? (Maybe i'm overthinking it, but I like services that put me in control of how much money i'm spending instead of being at the mercy of botnets)

<comment><AUTHOR> "Unusual" send activity is pretty easy to spot, especially if it's going to put our customer at risk in any way. We're extremely proactive about making sure that you're aware of anything that might be considered a "surprise" to your service or your wallet.

</comment></comment></comment></comment></comment><comment><USER8> Lamson makes email processing easy - it is a very modern framework for writing email applications. I have used it for a while now (almost a year) and have not had huge problems. I have written about it here (including code samples): <http://amix.dk/blog/post/19608>

<comment> old-gregg Lamson is a great project but it has a number of issues with handling international emails, try handling mails that have subjects in two different languages, or have To/Cc fields with non-ASCII names in them. Basically if you're using it you'll have to guard it with some additional code.

</comment></comment><comment><USER9> I read though the docs and couldn't tell: is delivery reliable? If my app is down and I miss some POSTs, will I be able to tell? Will they be re-delivered?

I'm currently using Exchange to get real-time mail notifications and nice parsing, but am looking for other options.

Also, encoding attachements as base64 is an interesting choice. I'd prefer a URL where they can be downloaded instead.

<comment><AUTHOR> Your activity feed will show that there was an HTTP error.

We're working on the workflow for retries, it's coming soon.

Thanks for the feedback about the download URLs. Any particular reason for this preference that I can share with the team?

<comment><USER9> Photos and other large content. Most web stacks don't nicely handle large requests like that, and to the best of my knowledge streaming JSON parsers are both young and rare.

<comment><AUTHOR> Totally fair - thanks!

</comment></comment></comment></comment><comment><USER10> This is awesome, will play with it when I get a chance.

As an aside, I've begun to think that email is/should become a platform for app development. Some sites are already using incoming email for active user engagement (posterous, idonethis, etc)

I think it's a great avenue to fight app/user fatigue. Most people constantly check their email no matter what, but few are making it past the app download to regular usage. The assumptions of emails limitations should be rethought.

</comment><comment><USER11> Parsing email is a real pain point, kudos to Postmark for trying to solve it. However, I _need_ this functionality in a more abstracted way, not locked to a email address. I'm thinking about taking a swing at it myself and GPL'ing it. (Basically full fledged IMAP->JSON.)

For a real world example, at my startup <https://zapier.com/> we have "new email received" or "new email tagged in Gmail" as one of the many, many inputs which you can map to any other write (IE: Tweet something, add a note to Basecamp, add a contact to Highrise, etc...). However, this means I need to log into a IMAP account, not receive an email at a predetermined address. Parsing that raw email is an absolute bear (even with Python's built in IMAP and parse libraries).

Anyone heard of a service or library like that?

<comment><USER12> <http://lamsonproject.org/>

<comment><USER11> Looks interested, without having the chance to give the docs a one over, does it do IMAP as well? Looks more focused on SMTP/sendmail.

<comment><USER13> At it's core Lamson is just a really sweet wrapper around smtpd (<http://docs.python.org/library/smtpd.html>) using asyncore for asynchronous socket IO.

When a mail is received you can optionally process it and when done processing you can drop it on the floor (e.g. you're done with it) or you can send it to a 'relay' which means a IMAP or POP server (or really anything) if you so choose.

I would generally say that yes, lamson is much more focused on receiving email via SMTP and doing [smart] processing on it.

<comment><USER11> I would imagine that its parsing functionality could be a boon to someone wanting to do something higher level with IMAP. Thanks for the response.

</comment></comment></comment></comment></comment><comment><AUTHOR> We've got more in store for this feature set, but we're really excited about this as a start - we use it ourselves at beanstalkapp.com and I know that a number of people HN were using the beta.

I'm happy to answer any questions, or field any ideas you might have.

<comment><USER14> Hey, finally it launched, will definitely come in handy.

I found a bug in the docs: On [http://developer.postmarkapp.com/developer- inbound-parse.htm...](http://developer.postmarkapp.com/developer-inbound- parse.html) the link named 'API developer email list' links to 'file://localhost/Users/alexhillman/Sites/wildbit/postmark- design/docs/groups.google.com/group/postmark-api-developers/'

<comment><AUTHOR> Good thing I don't keep docs in my pr0n folder. That would've been embarrassing.

</comment></comment></comment><comment><USER15> Fantastic! I'm very excited to try this out. I've been using postmark to quickly build out my MVP for a new site I'm working on. This is just the other side of the equation and I am very eager to start playing with it. Integrating with their nodejs library was literally as simple as putting the library in the right place, adding the api key and calling the function. Could not be simpler.

</comment><comment><USER16> As much as I like the simplicity of Postmark, their pricing makes it very hard for me to justify using their service. Sendgrid and Amazon SES are charging $0.10 per thousand. Postgrid is 15 times more expensive.

<comment><USER17> Also, MailGun is $1 per thousand. Which seems to offer a similar proposition.

<comment><AUTHOR> I'll point out that we have no "account minimums", only the $1.50/1000 email credits for both inbound and outbound that never expire.

Also, we're home to many happy customers that are sending very high volume - we offer them bulk-credit purchase rates as low as $0.50 per 1000 when buying 2MM credits at a time or more.

</comment></comment></comment><comment><USER18> This is great! Perfect timing too. I've been using postmark for over a year, and have recently been looking to other providers for an inbound service.... soooo glad they did this!

</comment><comment><USER19> Try EmailYak. It is simple and elegant <http://www.emailyak.com/> It is my friend's but I'd still recommend it otherwise. See: <http://docs.emailyak.com/quick-start-guide.html>

<comment><USER20> It's also quite pricey - same price as Postmark at its __cheapest __, and $5/thousand at the first pricing tier.

</comment></comment><comment><USER21> There are a subset of applications that imo could rely exclusively on email as the platform.

A few months ago I built such an app, a small side-project called betabrokers(email firstname.lastname@example.com with subject:about to signup and learn more). The simplicity of the signup process for this type of app (simply sending an email) is a huge benefit.

Email-based apps especially make sense in a mobile context, which is somewhat ironic because the transactional nature of applications built on them would suggest that they are slower, therefore less likely to be used. In our case however, users keep coming back to it, probably because the email client is the single most used app on their phone, and receiving replies to your actions via email becomes addictive.

</comment><comment><USER22> We've been using Postmark for quite a while now (maybe 2 years now?) and it's fantastic. We recently began using the inbound processing while it was in beta, and it too is fantastic. The API is great and it's absolutely painless to setup and begin using quickly...

<comment><AUTHOR> Thanks for the kudos, Jeremy. If you can share where you're using the inbound API, I'd love to see it in action!

</comment></comment><comment><USER23> This is neat, but it's not too difficult to handle incoming mail yourself with postfix and a simple ruby/python script to process it. The learning curve is slightly steep due to postfix and some _really_ bad suggestions, but the actual work is straight forward.

I setup a simple ruby program to pipe the postfix email into a resque queue (redis backed ruby jobs system) to be handled by my work pool...a couple easy regexes determine what the email is: bounces get auto-delisted after 2x, unsubscribes delist immediately, replies get shoved into the correct in-site comment thread or conversation, etc. This is a day of work....

<comment><USER24> You are correct, of course. Nothing stops you from doing this yourself. The basic functionality in SaaS is always easy to accomplish (hosting, email sending, dropbox, etc). Its the little things that get you and eventually drive you to the bottle.

</comment></comment><comment><USER25> Interestingly we just had to build this type of processing into one of our applications, and I would have much rather sent this through a simple API.

Alex are there high volume discounts?

<comment><AUTHOR> Yup, the same discounts apply for volume credit purchases.

Our credits work for inbound and outbound and never expire.

* 500,000+ - $1/thousand ($500) * 1MM+ - $0.75/thousand ($750) * 2MM+ - $0.50/thousand ($1,000)

Sending even more? Email me - email@example.com and we can talk.

</comment></comment><comment><USER6> I've been using this amazing service <http://mailnuggets.com> for a while now (6 months or so) and the guy is absolutely fantastic (I emailed some ideas and he implemented them :D) buuuut I use Postmark for outbound email, it would be nice to consolidate everything into one account. Will give this a try. I love postmark :-D

Google apps + configured filters + inbound email processing = awesome possibilities.

</comment><comment><USER26> This got me thinking... Are there any startups that are doing something similar with physical mail? I can picture a large centralized mail room with a bunch of industrial sized scanners, mail gets opened, scanned, and uploaded. Customers could access pdfs of their daily mail through a Web interface and could search through archives. Through in some super OCR and I think you'd really have something...

<comment><USER24> You know, when I started working on Postmark I thought that would make a fantastic April Fools joke.

then I discovered there really is a company that tries to do that. I can't remember the name though…

<comment><USER27> Earth Class Mail?

<http://www.earthclassmail.com/>

</comment></comment></comment><comment><USER28> I'd love an email API that you could forward an email thread to, which would be parsed into a set of linked emails with contacts attached. Email to JSON is great, but not when the body is the text of 20 forwarded emails. Seems like a gimme for a CRM app -- does anybody provide this?

<comment><AUTHOR> This is a cool idea, we'll keep it in mind!

</comment></comment><comment><USER29> How well does this deal with the cruft of a "Replied to" email

Are there particular requirements/restrictions for emails to be robustly parsed? Even Facebook had problems early on with their email-response parsing.

</comment><comment><USER30> I can see how this would be helpful but it seems like the best way to do this would be with a POP/IMAP client in your code. But I'm guessing those libraries are probably not as user-friendly as one would hope?

</comment><comment><USER31> We use google for our domain email. In order to handle uploads via email on this same domain we simply create a new account and send emails to this account. You then can have a hook (imap push) in google imap to detect when an email is received per account/box and process is right away. Or you can just run a cron every minute the check if new emails have been received. We then fire a script that pulls the email via imap, and then processes the emails/images into json format for use.

This allows us to keep everything on google and we do not need move our MX somewhere else. Also we keep our "upload" email on the same domain.

Pretty simple solution with no additional costs.</comment>


An Inside Look at Google BigQuery [pdf] - <AUTHOR> https://cloud.google.com/files/BigQueryTechnicalWP.pdf

<comment><USER2> Note that this is from 2012 and many things have been added/improved. Cost has also been dropped by ~80% a year ago.

</comment><comment><USER3> This is so old! (from the author :)</comment>


Vonage makes free international calls standard - <AUTHOR> http://www.physorg.com/print169924439.html

<comment><USER2> When I canceled my Vonage account (several years ago, I used it for my business phone line back when I had a business that needed a phone), I kept having to repeat myself, and ask the guy on the other end to repeat himself. The quality was horrible. At the end of the cancellation conversation, the guy asked, "May we ask why you're canceling?" And I replied, "You know how you and I have had to keep repeating ourselves, and saying, 'What?' every ten seconds?" And he responded, "Oh."

Anyway, I thought Vonage had gone bankrupt a few years back, or something. I don't understand why anyone would use it for anything. Cell phones are pervasive, dirt cheap, and they work reliably. Skype is mostly free, has video and other cool features, and works reliably. Vonage has none of these things going for it.</comment>


What quantitative facts would you like to know about your startup's market? - <AUTHOR>

Hi.
A friend and I are trying to create some sort of resource where founders can share /swap quantitative facts about their (soon-to-be-launched) companies.
Wanted to know what are the questions that YOU would like answered? (I always get to "So how many users will pay for my service?" sooner or later, unfortunately ;) )
Note that I'm talking hard facts (of the sort you could put in a wiki), not opinions.
Thanks  :) <comment><USER2> I'm not sure how willing I would be to share quantitative data about my startup's markets. I would of course like to read my competitors talking about their business, but I would hesitate to contribute.

<comment><AUTHOR> Of course we all would ;)

The idea isn't to talk about personally identifiable data about your startup, but rather to give average "industry" figures.

But that still doesn't answer the question: what exactly would you like to know?

<comment><USER2> Sales numbers for all direct to consumer and via MD genetic tests

</comment></comment></comment><comment><USER3> hmm why dont you just put up a wiki

<comment><AUTHOR> That's the plan :)

I mean, if folks at a forum like this can come up with why they'd use such a wiki (actual examples), it'd make the concept a little easier to explain.

Right now, whenever we're trying to talk about it; we simply run out of examples before anyone "gets it"!</comment></comment>


Tarsnap public beta - <AUTHOR> http://www.daemonology.net/blog/2008-11-10-tarsnap-public-beta.html

<comment><USER2> Too much crypto, not enough value prop! Everyone believes you on the crypto stuff already, Colin.

Move out of privacy --- privacy is a dead end, nobody has ever capitalized on it successfully (see: the original PGP, Zero Knowledge Systems, that stupid data center in the pylon off the coast of England).

Rewrite your content to focus on shared hosting. Emphasize that _you don't keep the keys_ to the backup data. Consider an OEM deal with a Slicehost-like provider. Backup services at hosting providers fucking blow.

People won't pay much for privacy, but businesses will pay a decent premium over basic hosting for CYA security.

Take your crypto content off the front page; ironically, I think you're _decreasing_ its value by fronting it like this. Instead, _brand it_ : 100% audit-ready, fully transparent, something like that: "the only secure backup provider with [blah blah blah]".

I should have to click to see that stuff. For normal users, they should just see the stamp of assurance and a page they can click to with lots of docs.

I'm a lot less concerned about the clientside security here and more concerned with the serverside --- even though I know intellectually it doesn't matter much, because the security has been factored out to the client. That could mean one of many things, including these two:

(1) People are never going to get over their concern about serverside security and you're going to have to do so something to demonstrate that the server code and associated web apps are secure.

(2) You're not doing enough with your pitch to emphasize that your design moves security off the server and onto the clients, where the customers control it.

How do you do dynamic updates? Spend less time sniping at SSL, because SSL is still the weak link in this system.

<comment><AUTHOR> Keep in mind that this is my blog, not the tarsnap website itself. The tarsnap website -- once it's more than a placeholder -- won't have technical details about the cryptography, but will instead (as you suggest) have it on a separate page which only the interested people will go and visit. My blog, however, is the blog of someone who regularly critiques other people's security (and complains about companies which don't provide details about their security), so people would start asking questions if I avoided the question of security in a blog post about tarsnap. :-)

Server side security -- you're right, I should probably talk about that more. The cryptographer in me says that there's no need to talk about it because it's irrelevant... but people tend to care about irrelevant things anyway.

What do you mean by "do dynamic updates"?

<comment><USER2> Start figuring out how you're going to talk about this now, and use the blog to practice. This killed our first product too.

I'm not saying talk up serverside security; I'm saying, after allllllll that content about crypto algorithms, I'm _still_ left wondering whether I can trust my data to your app. Most everyone else who talks about secure backup is going to leave it at "SSL, plus we encrypt the data on our servers". Your #1 feature is that you do the opposite, but all I see is RSA OAEP.

Also, I'm not saying downplay the crypto; I'm saying, reverse pyramid style, find a way to come up with a brand-y feature-y way to summarize (a) custom crypto, (b) open, auditable design, (c) freely available source code. Then use the crypto details to back that up (on a seperate page).

Dynamic updates: how do you update the tarsnap clientside software?

Consider a name other than tarsnap, too.

<comment><AUTHOR> _Dynamic updates: how do you update the tarsnap clientside software?_

I don't. Users do (by downloading the latest version, extracting the tarball, compiling, and installing).

Actually, you're right that I depend on SSL for this at the moment -- but I will be adding GPG signatures for the tarballs soon.

 _Consider a name other than tarsnap, too._

What's wrong with that name? In all seriousness, the people who would be turned off by that name are also people who would be turned off by the command-line interface.

<comment><USER2> _What's wrong with that name? In all seriousness, the people who would be turned off by that name are also people who would be turned off by the command-line interface._

Yeah, and, about that...

<comment><USER3> I think you make some VERY good points, but I'm unsure if you and the creator of tarsnap are on the same page regarding the audience / customer base for this service. I like the name because it makes sense to me and it's easy to understand. Of course the management at Big Co. probably have no idea what "tarsnap" means just by looking at it. That could matter, or not, depending on the author's target audience.

Also, from your example, I've crafted the perfect position statement: _For summer camps that need surface mount soldering, Colin Percival offers tarsnap. Unlike carbonite, tarsnap provides puppy mincing that allows users to make better tacos_

After reading that, I'm buying tomorrow.

<comment><USER2> There's a pretty well recognized trap in the security software space; call it the "IDA Pro Effect". IDA is the industry standard disassembler, easily the most popular commercial reverse engineering tool, certainly licensed by every major security research team. It costs ~$500.

This is a problem: successfully cornering the market for disassemblers and selling to ever security research team wins Hex-Rays literally _thousands_ of dollars!

If you're going to craft a product pitch that appeals to the very high end of the market --- those comfortable with per-host command-line backup who care very intensely about security --- you may need to price way higher than where Colin is today. Of course, then he'll find himself in competition with homebrew PGP+S3 scripts, just like IDA would quickly be replaced by a web UI on "objdump" if Ilfak jacked the price up to $20,000 (which is kind of where it belongs, value-wise).

I'm also wary of the history of "premium" command-line backup tools; remember Bru?

<comment><USER3> I don't disagree one bit. I'm not entirely convinced it will be successful, considering it is a pretty niche product as far as backup products go and the security value-add isn't really priced according to what very security- conscious people/organizations would spend on the solution. All I was really pointing out is that Colin and you may have different ideas of who the audience here is, in which case some of your advice isn't relevant (yet, anyway); if it's going to be marketed to people who <3 cli then tarsnap is a perfectly good name. Otherwise, something _awesome_ like "Fort Knox Backup Vault 2008" may be more appropriate ;)

<comment><USER2> For the record: I'm very bullish on Colin's idea; I just don't think he has the positioning and packaging worked out.

Unless he's not out to make money. Some people aren't. He's a FreeBSD dev, after all. Sucker. :P

<comment><AUTHOR> Don't worry, I'm in this to make money. :-)

I started with *nix and a command-line interface for two reasons: 1. This is what I know, and 2. a CLI can be far more easily used as a component of a larger system.

I'm following a fairly standard startup course here: Build something people want, then build something more people want. I expect a GUI to happen in the future (probably using Qt). I expect a Windows version to happen in the future. I expect tarsnap to be included in web server management interfaces in the future. The name might change in the future, or "tarsnap" might just end up being one of the names like "google" which start out looking dumb but end up sounding normal.

But that's all "easy" stuff -- the place to start is with a solid and secure foundation, which is what I'm doing.

</comment></comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER4> Looks nice. How fast is it? I wonder if you could make this into a simple off- site backup system for databases. It might require some more complex interaction with the db or os to do it right though (since the data is changing while you are reading it).

<comment><AUTHOR> _How fast is it?_

How fast is your internet connection? On a decent CPU the tarsnap code can easily push 50 Mbps -- the largest part of its time is spent in zlib, compressing data before it is encrypted and uploaded -- so the limiting factor is almost certainly going to be bandwidth.

 _I wonder if you could make this into a simple off-site backup system for databases._

Tarsnap isn't likely to be useful for "real time" replication, but if you get your database to write a transaction log you might get interesting results by having tarsnap loop creating archives containing that one file.

<comment><USER4> I was thinking of the snapshot speed, where most of the data hasn't changed. Do you keep a local cache of block checksums or something to avoid having to check the server? How about an option to not examine the data if the file size and mtime haven't changed? (like rsync does)

<comment><AUTHOR> _I was thinking of the snapshot speed, where most of the data hasn't changed._

On modern CPUs, somewhere in the 20-50 MB/s range.

 _Do you keep a local cache of block checksums or something to avoid having to check the server?_

Yes.

 _How about an option to not examine the data if the file size and mtime haven't changed? (like rsync does)_

Tarsnap already does this for unmodified (file name, inode number, file size, mtime) tuples.

</comment></comment></comment></comment><comment><USER5> In case people aren't paying attention, this is well-respected news.yc-er cperciva's big project. Looking pretty good, Colin!

 _edited for clarity_

<comment><AUTHOR> I'm well-respected now? When did that happen?

<comment><USER6> _When did that happen?_

481 days ago. <http://news.ycombinator.com/item?id=35083>

<comment><USER7> I simultaneously feel proud and shamed for making that happen!

No hard feelings, I hope.

<comment><AUTHOR> No hard feelings. :-)

More than anything else, I'm amused that people here care so much about the Putnam -- I personally consider winning the Putnam to be a relatively minor accomplishment.

<comment><USER8> Those of us who pride ourselves at merely having gotten a non-zero score on the Putnam think otherwise :-)

</comment></comment></comment></comment></comment></comment><comment><USER9> Sorry to hear that you aren't up to dealing with the hassles imposed by the Canadian tax systems. Perhaps you can offer a limited storage/bandwidth account free of charge to fellow Canucks. :) There aren't that many of us...

<comment><USER10> Idle thought: Here in Australia, you don't have to deal with sales tax until you've reached about $50,000 per annum income. I always thought this limit should be much higher, say $200,000, since it is in society's interest for small businesses to keep growing to that stage anyway.

<comment><AUTHOR> For Canadian sales tax (GST) there's a lower limit before people have to collect. For British Columbia sales tax (PST) there isn't any such limit.

Rather than allowing in Canadians but worrying about making sure that I figured out how to deal with GST before I hit the limit, while still keeping BCers out, I figured that it was simpler just to keep all Canadians out.

</comment></comment></comment><comment><AUTHOR> Another small step towards launch: After many months in private free beta and a bit over a month in private paid beta, tarsnap has now moved into public beta.

</comment><comment><USER11> You're free to ignore me, but what I see is someone really impressed with their technical prowess and the engineering that went into this software.

It looks like cool technology, but the packaging is all wrong. A command line tool?

I'd build a seamless front-end a la JungleDisk and bill it as an online safe deposit box -- nobody but you has the key.

Just my $0.02.

</comment><comment><USER12> Sounds interesting, this could replace a scheme I use to store files in AWS. However if you're using AWS under the covers I'd rather use my own AWS account via dev pay instead of paypal and I'd rather grant your service permission to store stuff in my own bucket.

<comment><AUTHOR> I am using S3 under the covers, but they are very thick covers. It's not feasible to separate out individual users' data to store it in different buckets via devpay. (Also, I'm in Canada, where FPS and devpay aren't supported yet -- as soon as they are, I'll be accepting payments via FPS.)

</comment></comment><comment><USER13> This is like JungleDisk but at twice the storage price and lacking the GUI or cross-platform clients.

The per-transfer and per-storage fee sounds like tarsnap also stores back to s3.

<comment><USER13> To be slightly more constructive:

You have at least one entrenched, successful competitor with lower prices and more features. This proves there's a market, but it means you're also going to have to be good at marketing to succeed. Especially because the product is something people are very risk-averse about and the switching cost of re- uploading everything is a big pain.

You might enjoy the book The Knack: How Street-Smart Entrepreneurs Learn to Handle Whatever Comes Up by Norm Brodsky and Bo Burlingham. There's a lot of war stories from author Brodsky about his document-storage business (a similar market to yours) and how he differentiated it from competitors.

</comment></comment><comment><USER14> Ok, so I currently use cron + s3sync + s3 buckets to backup servers/sites.

How would this differ (other than the price)?

<comment><AUTHOR> 1\. Security.

2\. Snapshotted backups. With tarsnap you can take a backup every day, but only pay for each unique block of data once -- which is useful if you don't want your nightly cron job to "synchronize" the fact that you just accidentally deleted some files to S3.

3\. I don't know all the details of how s3sync works, but how efficient is it at handling files which have had small changes made to them? (Or, in the case of log files, data appended to them?) Tarsnap doesn't have to re-send the entire file.

4\. Does s3sync compress data before storing it? Tarsnap does (and you pay for the bandwidth and storage used, post-compression).

5\. When looking at the price, keep in mind that S3 has a per-request fee which can get pretty expensive if your average file size is small.

<comment><USER15> _Snapshotted backups_

While we're giving marketing advice, shouldn't that be "source-based de-dupe"? :-)

<comment><AUTHOR> Yes, but I absolutely HATE the word "de-dupe", and I've been hoping that if I ignore it for long enough, it will go away. :-)</comment></comment></comment></comment>


Watercoolr - Gossip for web applications - <AUTHOR> http://watercoolr.nuklei.com/ <comment><USER2> There's an article about pubsub here:

[http://www.igvita.com/2009/06/29/http-pubsub-webhooks- pubsub...](http://www.igvita.com/2009/06/29/http-pubsub-webhooks- pubsubhubbub/)</comment>


Ask HN: Future of Windows OS on Phone? - <AUTHOR> What will be the future of this OS in mobile devices? <comment><USER2> In the short term, it will probably be a very distant third. In the long term, as the wild west gets settled and mobile computing hardware stabilizes toward longer life cycles, I think Microsoft will be more of a player in the business world. BYOD works better when the mobile device is not the primary computing interface within a business. As the mobile device becomes a primary computing device for line of business applications, the vertical integration provided by the Windows mobile platform provides serious advantages.

The competing operating systems are not driven by developer support because the underlying businesses are hardware sales and data aggregation. Microsoft is a tools company.</comment>


Show HN: One year devoted to open source and I've almost made $2 – horrraaaaay - <AUTHOR> https://opencollective.com/property_web_builder <comment><USER2> Don't go into open source expecting anyone to pay you. It doesn't work like that. The reason your open source project is used by anyone is, to a great extent, because it's free and open source. If you charged money at the outset, there's zero reason to believe that anyone would use it. If others are making money or raising funds on the back of your work, there's zero expectation for them to even give you a token of appreciation.

Anecdote: A few of our open source projects (our largest is [https://github.com/SheetJS/js-xlsx](https://github.com/SheetJS/js-xlsx) ) have web-based demos ( [http://oss.sheetjs.com/](http://oss.sheetjs.com/) ) which some people have copied, stripped license and attribution comments, changed the name, raised money, and then turned around and tried to shut us down. On the bright side, at least someone found it useful.

<comment><USER3> > Don't go into open source expecting anyone to pay you. It doesn't work like > that.

It's changing a bit in the crypto space. Take a look at:

\- [https://gitcoin.co/](https://gitcoin.co/)

<comment><USER4> So gitcoin could be really revolutionary or it could be a total flop.

I had a lot of hope for BitHub, a simple tool for paying bitcoin per git commit. What seems different about gitcoin which might increasing its chances of success is that it seems to be focusing on building a community/marketplace.

Would be great if GitHub could add donate buttons to pages and issues and payout. No reason this has to be done with cryptocurrency.

[1] [https://github.com/WhisperSystems/BitHub](https://github.com/WhisperSystems/BitHub)

</comment></comment></comment><comment><USER5> yo dawg, the image (wording) here is off-putting. [https://propertywebbuilder.herokuapp.com/](https://propertywebbuilder.herokuapp.com/)

I'm not uptight or anything, but it bothers me when people allegedly (see, i'm doubting you already) put in so much work and then shoot themselves in the foot with shenanigans for half a laugh. First impressions, and such...

This often bothers me, and obviously when i was cooler and in college I did it to. Aaaaaand it didn't get me anywhere. In this particular case, we're talking real-estate? big money, important purchase, long-term debts. It especially doesn't fit.

<comment><AUTHOR> Fixed. Reset the database and re-seeded it. Come on guys - do you really think I would put such nonsense on my own site.

<comment><USER6> If anyone can deface your site at will, and it takes your manual intervention to fix it, you won't be able to just "set it and forget it" as you will be constantly babysitting your website.

Find a way to limit the damage one can do to the site by localizing changes to a randomized URL or something, because giving anyone on the internet access to your main demo is quite dangerous.

</comment></comment></comment><comment><USER7> All projects are cool, and kudos on going down the open source route.

I'm struggling to determine what you're offering though? How many new real estate websites launch, how many are looking for a "RoR ecosystem solution", how many are genuinely losing out anything by being based on Wordpress?

If you think the niche is there, I'd suggest your monetisation strategy should be like most of the big hitters in the space: continue to give away the product, sell value-added services on top. Offer the hosting, devops, SEO management, branding and advertising, or something.

At the moment there's no compel, it's a solution looking for a problem (I'm not judging, built plenty of them myself, just without any sort of expectation someone would give me money for the sake of it).

</comment><comment><USER8> Not well versed in the open-source realm at all, but my guess is that if monetary gain is your primary goal, you'll probably have a bad time.

Most of the value of open-source work lies in the experience gleaned. It looks good on a resume. (There's also of course the personal satisfaction you may get from knowing a bunch of strangers use, and maybe even love, something you made or contributed heavily to).

<comment><USER9> > (There's also of course the personal satisfaction you may get from knowing a > bunch of strangers use, and maybe even love, something you made or > contributed heavily to).

From my experience this is actually way more rewarding than getting paid.

</comment></comment><comment><USER10> [https://twitter.com/SquidDLane/status/943622664211652615](https://twitter.com/SquidDLane/status/943622664211652615)

</comment><comment><USER11> I'm interested in this project - but I do have a few questions:

Who is the target audience? Developers that need an OSS RoR alternative to wordpress? Property owners/managers that need quick sites? Someone else?

Why does Open Source matter in this case? And how does it affect the target audience?

What disadvantages of Wordpress are you trying to fix/improve/innovate around? (I took a look at the link provided on the website, but assuming the reader is not a developer, it may be better to summarize in your own words).

What do donations go towards? What fees are associated with the upkeep and development of this product? How will my donations contribute to development?

<comment><AUTHOR> Good questions. The target audience for now is developers. The long term vision is that it acts as a platform on which other real estate services can be built.

Maybe the title I chose makes it sound like I'm complaining or begging. I'm not - just a sarcastic dig at myself. I'm not expecting to raise much money but whatever I raise will go back into the project.

The opencollective is actually a great idea because it means I don't have to go to great lengths to prove that I will put the money I raise back into the project. What I raise and spend will be entirely transparent.

</comment></comment><comment><USER12> There are many open source business models. For individual developers, probably the most viable one is to become a domain expert in a popular open source technology and then to find a commercial entity who uses that technology and is willing to sponsor some work on it, in exchange for having someone on staff who can help them use it much more effectively.

Not all technologies are suitable for this business model, and it is best if you are comfortable with permissive licensing. And you may wind up working more and working harder than colleagues who don't choose an open source route.

Nevertheless, there are big advantages: the job security that comes from having a good reputation in open source is valuable, and if you care about open source ideals, you can take pleasure in how much this sort of position allows you to spread them and move them forward in our industry.

</comment><comment><AUTHOR> I'm not bitter. Just saying. I know open source takes a long time to be worthwhile but it has kind of hurt thinking how much income I was losing out on each month (especially as everyone around me was showing of their shiny bitcoins). Hopefully I'll figure out a way to monetize next year.....

<comment><USER13> near as I saw, you were up to $17, so there's that.

"This project has been created to address a glaring gap in the rails ecosystem: the lack of an open source project for real estate websites.

The result is that WordPress has become the dominant tool for creating real estate websites. This is far from ideal and PropertyWebBuilder seeks to address this."

isn't a bad way to sell the widget to developers. but because you've already put some of the work in to set up the one-click deployment, it looks like you're walking the path of going to a wider audience, and if that's your goal (not a horrifying one, it maybe where more dollars are...) it's good to think about how to explain that chunk more broadly.

granted, it's a tough line to walk but a lot of people are doing interesting things --> [https://www.eclipse.org/oxygen/](https://www.eclipse.org/oxygen/) for example.

<comment><AUTHOR> The $17 is the projected annual budget. The guy just started donating $2 this month ;)

Yes, you are right. I should aim for a wider audience. To be honest I'm totally to blame for not making more of an effort with marketing.

I will make more of an effort next year. I actually wasn't even expecting the $2 donation - it did cheer me up.

</comment></comment></comment><comment><USER14> PropertyWebBuilder is currently sponsored by Coddde, Ruby On Rails consultants based in Spain and Chile:

uh they sponsored you for 2 dollars? This might be you problem.

Also until you have users you aren't going to and shouldn't make money.

<comment><USER15> > Also until you have users you aren't going to and shouldn't make money.

How then do you explain crowdfunding, which is effectively what this is?

<comment><USER16> Have you ever looked around Kickstarter? I mean really _looked around_ Kickstarter? There are a lot of campaigns that are going nowhere. Same on Patreon, same on any clone of either of those two sites.

You've got to have something people want, and find a way to tell the people who want it that you're making it. You don't magically have people show up and rain money on you just because you've put up a page on a crowdfunding site.

(FWIW, I've done three comics kickstarters, and have a Patreon that pays most of my rent drawing comics on months when I'm cranking out pages at a good pace.)

<comment><USER17> I agree with egypturnash. I did two failed Patreons and currently have a successful one that barely is enough to live on (in southern Vermont, on Section 8) which was launched off the existence of an existing business about four times its size.

I can't speak for Kickstarter: that seems more of a 'from nothing' place and it'd be a gamble. I can say that if you are running a business charging money for what you do, you can do Patreon for the same thing and make a quarter to a half as much, IF you don't find viral success of some sort.

One big difference: doing the Patreon, you get to give your life work for 'free' to your community, which might desperately need that sort of support. I give tools to musicians, and I passionately believe that there are many musicians/producers/etc who cannot blow lots of money on proper tools they need. And, the ones I want to hear are not necessarily the ones who have the spending money.

So, my Patreon has other than monetary value to me. I take a hit in my income so that I can get the tools out there, across a wide range of host computers that sometimes aren't supported by the commercial sphere, and I get a lot of appreciation for doing it.

Even to do that, you need to already have a following, and it had better be a following that's already paying you or hiring you to do something.

<comment><USER16> I fuckin' _love_ that Patreon earns me more from drawing my comics than ads _ever_ did. My stuff's all ad-free now. Which, yeah, it sure is nice to be able to have media that broke queer femmes can see themselves in as long as they have some kind of Internet connection. And other broke people who may not be seeing themselves in my protagonists but enjoy other aspects of my work.

<comment><USER17> Yeah, I was selling software as retail off my own storefront: I never tried to make it ad-supported, I hate that. Patreon seems like it's a cut above trying to be ad-supported or Youtube-supported. That doesn't make it comparable to operating on a true retail basis selling merchandise… but it's definitely better than 'everything being free' and trying to roll your own support system.

I get people mad at Patreon wanting to send me money by PayPal. The thing is, if you do that, where are you going to be next month? I already know what total chaos of income and not being able to set expectations is like. It's called 'selling at retail and trying to get a next hit product'.

</comment></comment></comment></comment></comment></comment><comment><USER18> As a couple other commenters alluded to, when it comes to making money you should consider open source as the "long game". There are thousands of developers who improved their ability to get jobs and higher salaries by demonstrating their ability via open source projects and commitment.

The more prominent names in open source are an obvious example, but there are countless others who are more marketable, perhaps more visible, and likely had more negotiating leverage upon hire due to open source work.

Your work today might pay off in far higher figures down the road.

</comment><comment><USER19> I'd look at the wording on the opencollective page.

E.g., change the first sentence from

Ruby on Rails website builder for real estate.

To

Real estate website builder based on Ruby on Rails.

Also, put some newlines in, and expand the blerb out a bit.

The example site looks nice, though I notice on clicking the links to different sections in firefox, it draws once and then everything slightly resizes a little bit, so may be worth sorting out.

It might be worth mentioning propertybuilder more in the example, I've been using Wagtail recently, and their bakerydemo does this quite well. It's not over the top, but it is in there.

</comment><comment><USER20> The way to make money with software is to either have a good marketing team or to spend a lot of time doing it yourself.

But releasing a project and hoping donations will come in almost never works.

<comment><USER15> This is quite true.

I have an online games website I work on mostly for my own enjoyment. Basically just my take on classic arcade games like Asteroids ([https://arcade.ly/games/asteroids/](https://arcade.ly/games/asteroids/)) and the little known Star Castle ([https://arcade.ly/games/starcastle/](https://arcade.ly/games/starcastle/)). It's made a grand total of less than £7 from ad revenue over the last two years... because what I want to do for fun is write the games, and not a bunch of boring SEO.

Sadly it's the boring SEO that might make it pay enough to cover its own hosting. And even knowing this I'm still too unmotivated to do anything about it[1].

 _[1] Granted I should probably use Fiverr or Peopleperhour to pay somebody to do it for me, and I probably will, but I at least want to finish my version of Space Invaders first... and, fortunately, I have some time off over Christmas to work on it._

</comment></comment><comment><USER21> as a fellow RoR dev and a real estate agent. My $0.02 is RoR isn't really the right tool for agents.

with WP, you get this huge ecosystem of easy hosting opportunities and addins. You can integrate your MLS directly with a wordpress app with no extra coding.

For the larger brokers, they often provide a web hosting solution for their agents to use that they built custom in house.

It is just too expensive to maintain and build compare to WP.

</comment><comment><USER22> Made $0.49 with a close to 1,000 stars repository :)

<comment><USER15> That hurts to hear.

You know what gets me though? Patreon and YouTube. Granted you have to work your ass off to get to, say, $1000/month - based on channels I watch I'd say you need 30k+ subscribers to get to this level - but clearly there are people out there, in fairly decent numbers, who are willing to donate to support free content.

So, the question becomes: why does this not work for OSS?

<comment><USER17> It's not about the code. I'm not bad at Patreon (around $750/mo, have an open source goal at $800/mo) but it seems like code is fungible, and it's not worth paying a person just to code.

The position to be in (and I think this will work for me) is to have a vision and essentially be the leader for that vision. Then, rather than getting paid to do what might be pretty ordinary code, you're getting paid because you're communicating a vision, one that's coherent and exciting to people.

What helps me is that I'm legitimately one of the vanguard developers in digital audio and have been for many years. It shows in my work, and that work isn't mainstream, and I'm able to communicate what's being discovered (for instance, the significance of repeated quantization of mantissa in floating- point representation: people behave as if you have infinite resolution because you can represent an infinitesimal value, but of course that's meaningless around 1.0f)

Just coding is NOT enough, and in fact having high value is not enough either. You have to get people to want to support your vision (or indeed personality, for the youtubers: and that holds for other types of creators as well).

I will do substantially worse when I go open source than when I was keeping things proprietary, just as I did worse going Patreon than I did when I was selling my stuff at $50 a pop. I'm doing the open source goal (and it's designed to be a trickle of source rather than a sudden throwing open of the gates) because it's the right thing, and that's in line with my brand (or, the way I represent myself).

I just have to manage the transition in such a way that I, personally, survive this doing of the right thing. That's not a given, but I think I can do it.

Never Patreon, or open source, in the belief that it is helping you. It's helping the world. Done unwisely, it can hurt or kill you if your resources can't withstand serious poverty.

<comment><USER15> Thanks for sharing your perspective: good food for thought there.

</comment></comment></comment></comment><comment><USER23> There are many extremely talented artists, musicians, writers, educators, ..., who don't get paid (much). If you're doing this with the intent making money, then you should a) reconsider or b) leverage this project and experience to get contracting or consulting work — which, thankfully, does pay.

</comment><comment><USER24> I'd like to feel your pain, but I have a project I've been working on for seven years that has 60x more stars and 10x more watchers than your project. It's used by a multitude of businesses and tens of thousands of developers.

Here is the open collective: [https://opencollective.com/fakerjs](https://opencollective.com/fakerjs), you'll see it's not even paying a pittance.

I suggest you either give up on writing open-source software or adjust your attitude entirely.

</comment><comment><USER25> Congratulations - you can now put "built software that people paid for" on your resume!

Sarcastic or not, that's a legitimate achievement.

</comment><comment><USER26> You don't do open source because of the money. You do it because you believe software should be free, and if you're lucky and your software is useful then maybe you can eek out an existence. But most open source makes no money.

<comment><USER12> There are lots of good reasons to do open source when you don't expect to make money for it. However, for some technologies there are definitely business plans which can work and yield higher returns than donationware: open core, consultancy, sponsored development, etc.

</comment></comment><comment><USER27> Begging for money?</comment>


The Baseline Costs of JavaScript Frameworks - <AUTHOR> https://blog.uncommon.is/the-baseline-costs-of-javascript-frameworks-f768e2865d4a <comment><USER3> As a US based user of an A53 based phone, where I use firefox. I'm here to tell you more than 50% of the sites I visit with it (HN being one that actually works well) are completely unusable. There was a saying back in the 1990's that developers should be forced to use obsolete hardware to assure that their software was usable for the regular user, and its even more true today. Developers running around with the latest iphones does nothing to actually represent what the average user actually sees.

Picking on amazon, I generally don't even bother going to their site unless i'm on my desktop, where I can physically watch them frequently spike my 4+ Ghz CPU for a few seconds when I type in the search bar, or click a page. Doing the same on my phone or cheap tablet frequently results in a 10+ second waits. The other day I was typing on my tablet and the amazon search completion was literally taking 20+ seconds to display each _character_ and a completion list for it.

<comment><USER4> And this is the first time I heard someone complain Amazon UI being slow. And Amazon sales or stock are not dropping.

Probably you are not part of the 99% of users that Amazon is targeting.

<comment><USER5> Amazon is succeeding _despite_ of this. Imagine where it could be it if was also fast!

As a workaround, now when I want to search something in amazon I just google for "amazon [what I want]"

</comment></comment></comment><comment><USER6> >Or consider not using a framework at all. For websites that primarily display content, it's more efficient and cost-effective to just send some server- rendered HTML down the wire. If there are areas of your website that require interactivity, you can always use JavaScript to build those specific parts.

This should be the takeaway from the article. There's absolutely no need for javascript on most websites that use javascript. I understand if you're forced into it by the time and economic contingencies of your work but if you bring JS frameworks home to your personal sites you're doing it wrong.

<comment><USER7> Exactly. I avoid JS as much as possible. Unfortunately most of web developers want to develop SPAs regardless of the actual need of the projects.

<comment><USER8> I prefer the MPA (multi-page application) approach for most projects. If interactivty is needed, I use REST services and a data-binding framework on the front end.

<comment><USER9> Can you recommend any such frameworks?

<comment><USER10> Stimulus is is a pretty good example of a simple JS framework that binds via data attributes. It, coupled with Turbolinks or Pjax, should let you do nearly anything you'd typically use a more popular front-end framework for, for much cheaper. Check it out: [https://stimulusjs.org/](https://stimulusjs.org/)

<comment><USER9> Thanks, that looks really cool.

</comment></comment></comment></comment></comment></comment><comment><USER11> I'm not surprised that Angular ships a larger bundle than React and Vue, it intentionally covers a much larger surface area of functionality. But Angular also includes build tools meant to reduce the size of the final bundle - <USER2> anyone tell if those were used in this comparison?

<comment><USER12> Yes, his Angular page [0] uses AoT compilation which results in the bundle, instead of individual files by default (meant for development only).

Also I agree with you. It's a bit unfair to be comparing a barebones Angular app to a barebones React/Vue app (even with a router library for each of those). Angular also has a forms and validation library already installed, and its dependency injection framework, etc. The baseline React/Vue apps don't have those pre-installed, which you would typically install when building an actual app.

[0] [https://angular-ngrx-router.netlify.com/](https://angular-ngrx- router.netlify.com/)

<comment><USER13> That's kind of the point though, right? If you don't need forms, can you remove them in Angular?

While we're on the subject, does anyone know a good forms and validation library for React? :-)

<comment><USER14> If you're using redux, I can't recommend redux-form enough. It's genuinely one of the best designed libraries I've ever used, including other languages/stacks etc. I think a little hard to grok at first but once it clicks you realise that it's such an elegant way to manage all form state you could ever have a need for, and it just works out the box.

Comes with an inbuilt validation mechanism in the form of a function with errors piped through to fields etc, if that's what you meant? The actual validations you have to write yourself though. I typically do this by hand/regex rather than use a library for that on the client, to be honest, as it's usually pretty simple.

<comment><USER15> The author of redux-form created another library, final-form, that doesn't need redux or react and has most of the features of redux form and has built in lessons he learned from writing redux-form. There's a separate React-final- form package with easy React bindings as well if you're using them together. It's worth a look as well (I've used both).

<comment><USER16> [https://github.com/final-form](https://github.com/final-form) ?

</comment></comment></comment></comment></comment></comment><comment><USER17> One thing it should be noted with Angular is that there is currently work on a new renderer that should allow framework bundle size to be optimized with the whole framework being tree-shakeable.

I wonder how Angular would compare once that is done, as this has long been known as probably the main negative with Angular performance-wise, the initial load (but it should be faster than most of the rest once it does load).

<comment><USER18> Really faster than react? Having a hard time with that one...

<comment><USER17> For all numbers excepting initial load, everything I've seen shows Angular is significantly faster than the rest - the difference is more pronounced with more DOM nodes. Angular also has the advantage of incremental update, although React will also gain that benefit within a year as well.

Note: I used to work primarily with Angular, but I currently use React since the coding style encouraged with React works better with the team I'm on & React is fast enough to us.

<comment><USER19> >everything I've seen shows Angular is significantly faster than the rest

That contradicts every published test.

It is also hard to imagine. Angular is both reading and writing the DOM.

And i haven't touched it after they switched to a new version, but in the old version they noticed state changes by contiously comparing all of your state. (so more state meant it would be slower).

I'm assuming they switched to a more native implementation of the observable pattern in the newer version, rather than that pathetic hack.

Of course neither React or Angular are as fast as hand written code. Since, what both take responsibility away from the developer. In the case of React: don't worry about what you need to update, just re-render the whole thing again. Simpler code, and the performance price you pay for it is somewhat limited because it then diffes a virtual dom.

Angular's approach, annotating HTML however, simply reinforces what we learned later: that is was written by someone with absolute no experience as a front- end developer. Because performance wise it's about the dumbest thing you could do. Every time you switch from writing to the DOM, to reading the DOM you force the browser to do another layout&paint and for all of that to happen synchronously, instead of the concurrent model that modern browsers actually offer.

What has happened is that these frameworks have become the new platforms. People don't realize that everything from online chess games to google maps isn't written using these frameworks and it's questionable any of them could do that with good enough performance.

At our company when recruiting we now distinguish actual front-end developers from angular or react developers. They may know these frameworks, that does not imply they could write them or understand enough of front-end development in general to debug actual cross browser issues, performance issues or rendering issues. And we give them a test that requires building something complicated in pure html/css/js.

<comment><USER17> > And i haven't touched it after they switched to a new version, but in the > old version they noticed state changes by contiously comparing all of your > state. (so more state meant it would be slower).

Are you confusing AngularJS and current Angular? AngularJS is certainly wholly slower than React, but not Angular 2+.

</comment></comment></comment></comment></comment><comment><USER20> I used to write all my static sites using Hugo, until I saw a site that was built using Gatsby. Gatsby has a larger overhead on the initial page load but feels much faster and subsequent page loads have less overhead (or maybe more, if it's prefetching a lot of adjacent pages which you'll never navigate to).

It seems to me that the next logical step is a site that ships the initial page request in vanilla JS, then async loads react and then async fetches the overhead and then runs Gatsby. This would be the ideal if we're optimizing for _perceived_ latency. So the first page load is fast, then as long as you stay on that page for a second or two before clicking elsewhere, everything after that point will be snappy too.

Ofcourse, many other devs are optimizing for network usage, etc, so to each their own.

<comment><USER21> You can attain Gatsby's SPA feel/speed with Hugo, via Turbolinks[0]. A much smaller overhead on initial page load, and you can hook up a service worker to do any prefetching/caching.

[0] [https://github.com/turbolinks/turbolinks](https://github.com/turbolinks/turbolinks)

<comment><USER20> Awesome, thank you. I dislike working with Gatsby and much preferred Hugo.

But what about prefetching?

<comment><USER21> From my experience, it seems like you'd have to jump through a few hoops to get prefetching working with Turbolinks. I just relied on having my service worker do all of the prefetching/caching of the site's important pages + static assets.

</comment></comment></comment></comment><comment><USER22> Tangental, but it's frustrating creating a website and optimizing it for performance down to the bone; and then having 'mission critical' plugins totally rip the potential performance gains to shreds.

My ecommerce site [0] (built on a custom stack, not shopify; making it orders of magnitude faster). I got it down to 600kb with 27 network requests total (with it being heavily media/image based). Now find me any other ecommerce site which can boast a comparable network payload.

Unfortunately, after throwing google analytics + fb ad trackers + a chat widget + a youtube video, the network requests have ballooned to 100+. It's very ironic how these companies promote the development a performance driven web, but the tools they want us to embed do quite the contrary. They do, however, offer a lot of value. What to do?

[0] — www.getfractals.com

<comment><USER23> Your site is still really fast compared with almost all ecommerce sites. I built the backend for a friend of mine's blog and I always use it as an example of how fast the web could be but yours might be a more impressive example:

[https://www.lukew.com](https://www.lukew.com)

<comment><USER22> Thank you kindly! The site you have attached is also rocket speed!

CMS's like shopify offer great value, and are a great starting place even for technical folks. However I think the lack of prioritization they've placed on the performance of sites on their platform has overall been a net negative for e-commerce at large. The mobile experience of most shopify stores is usually atrocious and the inclination to bounce before page load is all too real.

I understand they are working with an old stack, and changing course may be nigh impossible at this point. However, I built my stack in 2014, and the dividends of doing something custom vs the status quo have more than paid off.

</comment></comment></comment><comment><USER24> Damn, Svelte is not on there. I'd be curious what the author thinks to it as an approach [https://svelte.technology/](https://svelte.technology/)

<comment><USER25> I was about to say, this is precisely the reason Svelte exists. I rewrote my personal site in it a year ago [1], and the development experience was similar to a Vue or Ractive app, but with compilation (ie, the resulting webpage doesn't have Svelte included - it's just _my app_ specifically compiled) the result was tiny.

[1] [https://mikemaccana.com](https://mikemaccana.com), probably has a lot of small bugs since I spent my time working on more important things.

<comment><USER26> Couldn't SSR with later JS hydration be just as effective at getting instant startup?

<comment><USER25> Svelte also does SSR (if you set it up, which I haven't bothered), but you don't have to pull down a whole framework, so the hydration step is much faster.

<comment><USER26> Right, but with SSR the other frameworks basically have no downside. Their loading is also instant. A 1 second long hydration is not a big deal.

<comment><USER25> 1000ms hydration may be a big deal, hydration may also take more than 1000ms.

<comment><USER26> What matters to people is the content loading and being able to see the page. For the majority of apps the hydration is almost irrelevant because the time it takes to read the page and decide to interact with it takes vastly more than 1000ms, though there could be exceptions. It's certainly the case for all the apps I've worked on.

<comment><USER27> It can still cause things like janky scrolling while the JS is parsed. Less JS is never ever going to be a bad thing

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER28> > your React application will never load faster than about 1.1 seconds on an > average phone in India, no matter how much you optimize it

What about cache? In that 1.1s figure, 0.9s are for the download of react.

Is it reasonable to assume that for most subsequent loads the website would load in 0.2s?

<comment><USER29> HTTPS everywhere kills caches, in the sense that only end devices can provide them, but other network nodes not.

Check Eric Meyer's complaints.

<comment><USER30> Does it also affect caching via service workers? Pretty standard practice in PWAs.

<comment><USER29> No, but it impacts the initial download as it needs to be end to end, instead of the closest access point.

Also a very good way to waste traffic quota, e.g. a school network connection vs students computers.

<comment><USER31> Which interestingly is a problem IPFS would solve, right? But IPFS potentially introduces other drawbacks.

It would be cool to able to build dual-distribution applications, where publicly available assets and content blobs (e.g. app.js) could be distributed in an immutable, cache-able manner, and server interactions and private assets could still be handled in a more traditional client/server way.

Perhaps then we might even be able to realize the original intent of including dependencies from a CDN, being able to share a cached version of React across websites.

(I'm not and IPFS advocate or detractor)

</comment></comment></comment></comment></comment><comment><USER32> A nice way to improve the time-to-interactive is to prefetch the framework after you load an initial basic page.

Here's Addy Osmani explaining how it works: [https://medium.com/dev- channel/a-netflix-web-performance-cas...](https://medium.com/dev- channel/a-netflix-web-performance-case-study-c0bcde26a9d9)

<comment><USER33> Used that on my first Ajax app. It was authenticated so we pruned the JS on the login page like crazy and added a script that injected the main bundle into the page after page ready.

Worked great for everybody not using a password manager...

</comment></comment><comment><USER34> That's assuming you don't do any server side rendering right?

I can send completely readable html over the wire that becomes interactive once the JS finishes loading.

What am I missing.

<comment><USER35> The days of work needed to set up server-side rendering?

I realize I must look stupid saying this, because when you know your tech stack inside out the setup might be 15 minutes instead of days. But a large proportion of developers only learn to do things when they have to, and so you learn server side rendering the first time you have a customer willing to pay for the time it takes you to learn.

<comment><USER36> "SSR" is the default behavior of most pre-2016 web frameworks. Only when ReactJS became hip did it appear on lists of nice-to-haves that people weigh the pros and cons of.

<comment><USER35> Yes, I'm talking in the context of SPA frameworks (which the article referred to).

</comment></comment></comment></comment><comment><USER37> I'm the only person on my current team (an internally-focused engineering team) who has experience building websites/apps for __external paying clients __(e.g. people who will freak out if anything fails to load faster than they can blink, on their phone 's 3G connection).

I was taken aback by how many times I had to argue with my team, our sister teams, our consultants/contractors, and even our (non-technical) leadership about NOT building a SPA when we came across 1 form in our large codebase that required some extra interactivity.

For some reason many developers seem to be fine turning a blind eye to the actual page-rendering time, maybe because it's hard to benchmark with tools? (although I'm skeptical of that). I find an overlap however in that those same developers seem to not understand why multiple database query roundtrips are worse than 1 query followed by some in-memory munging.

I'm sure I'm biased by my background, where we were profiling like mad, doing crazy things like purposely modifying images/hand-building sprites to pack better, minifying CSS before SASS/LESS existed, and every millisecond to finish rendering mattered. Good times.

<comment><USER4> Really depends on what is your product and how users use it.

If the user just visit a few pages every time, you probably don't need SPA.

However, if your website has hundreds of pages and users need to navigate around a lot, SPA would actually reduce load time by cutting down the amount of data being transferred over network every time users goes to a different page. It also makes the experience navigating between pages smoother.

<comment><USER38> browser cache solves this problem

<comment><USER4> Curious what's the strategy you use to invalidate browser cache for non-SPA? When deploying new changes.

<comment><USER38> versioned url

<comment><USER4> Sorry if it is not clear, I was referring to html files returned by servers when user visits old-fashioned server-rendered web pages (non-SPA pages) like `/page1` or `/page2`.

How do you version urls like those?

Something like `/page1?t=201811292359`? That seems ugly to the users.

<comment><USER38> rules i use:

\- content under certain url should never change

\- if it has to change then use ESI or/and smaller cache expiration times

\- if it has to change very often/realtime, then use static html as skeleton and load only required information via javascript. f.e.: info about currently signed in user, eshop basket

</comment></comment></comment></comment></comment></comment></comment><comment><USER39> This is why I'm a happy user of Preact - the core feature set of React minus the bloat. Especially with some recent additions to React (context, hooks) which I don't have any need for, I'm glad to stay with the smaller surface area of Preact instead.

<comment><USER40> Preact is the most React-like small framework, probably the best for your resume and for hiring people who want a good resume.

Mithril is the small framework that most directly addresses the issues brought up in this article. It has routing built in and comes with a small streams library that, for apps I work on, is all the state management I need.

For those who really love JSX, Inferno is a really clean, small React-like framework. Its advantage over Mithril is that JSX expressions evaluate to components, not vnodes, so your HOC code will be slightly cleaner.

</comment></comment><comment><USER41> > If you want your application to become interactive on your users' devices in > under 5 seconds, can you afford to spend a fifth of that time just booting > up React?

If you care about load time you should pick a framework because of the value it brings, rather than what it costs. That one second overhead might make your application code twice as efficient.

For example Next.js and React will give you route based bundle splitting and navigation preloading for free. Yes, a vanilla JS app could implement the same, but requires a lot more boilerplate to achieve. So in practice most devs are more likely to skip that part and their lightweight no-framework app becomes a bloated mess.

<comment><USER42> I switched a classical html stack to a next.js app and although the first page load is definitely lagging, the speed gains of inter-page navigation is incredible. People navigate 70 pages within a couple of minutes.

Definitely worth it for many modern-day usages.

</comment></comment><comment><USER43> I wonder how much bootup time Ember.js takes nowadays. I mean, a few years back Ember was probably the worst when it came to initial loading times (it has other strengths), but I heard it got better over the years.

</comment><comment><USER44> > Frameworks exist for a reason.

People keep saying that...

</comment><comment><USER45> > Truth is, when you build your application on top of a modern JavaScript > framework, you agree to pay a certain baseline performance cost that can > never be optimized away. In exchange for paying this cost, you gain > maintainability, developer efficiency, and (hopefully) better performance > during runtime.

All of that is completely subjective except for that final bit about runtime performance. I bet nobody can prove runtime performance improves in correlation with layers of abstraction.

<comment><USER46> It depends. ML or ATS code can perform faster than hand-written C code, despite the fact both MLton and ATS compilers actually generate C code, effectively adding the layers of abstractions.

And you can have the verification for free also.

> We introduce a new approach for implementing cryptographic arithmetic in > short high-level code with machine-checked proofs of functional correctness. > We further demonstrate that simple partial evaluation is sufficient to > transform into the fastest-known C code, breaking the decades-old pattern > that the only fast implementations are those whose instruction-level steps > were written out by hand.

[http://adam.chlipala.net/papers/FiatCryptoSP19/FiatCryptoSP1...](http://adam.chlipala.net/papers/FiatCryptoSP19/FiatCryptoSP19.pdf)

<comment><USER45> Compiler output isn't an abstraction. It's what is ultimately executed where the compiler's source sample is not. For an apples to apples comparison I am claiming the code compiled from a framework is going to be slower than well written code without a framework. I am also claiming you well be painfully challenged to prove otherwise.

Frameworks don't exist to make code faster. They exist to provide an abstraction layer.

</comment></comment></comment><comment><USER47> It's simple. Don't use frameworks for content consumption. It adds no value to the end user. No, I don't need your full fledged my-data-sucking SPA just to read an article on your site. Keep it simple.

The best example?

[https://www.react.rocks](https://www.react.rocks) is a site built on react to advocate people to use React.

This is how it looks like without JS.

[https://webcache.googleusercontent.com/search?q=cache:WRll6h...](https://webcache.googleusercontent.com/search?q=cache:WRll6h8NQc8J:https://react.rocks/example/react- chrome-redux+&cd=6&hl=en&ct=clnk)

A site built on React to advocate React doesn't serve its primary purpose. The irony.

<comment><USER48> I don't understand your complaint here. Your issue is that a site written with a JS framework doesn't load without JS? The HN anti-JS brigade will never cease to amaze me.

You think the 0.001% of people who browse with JS disabled are the primary target for such a website?

<comment><USER49> >You think the 0.001% of people who browse with JS disabled are the primary target for such a website?

Because you obviously grabbed this number out of thin air, I would like the HN community to know that the actual percentage of users without JS is much closer to 1%[0].

[0] [https://gds.blog.gov.uk/2013/10/21/how-many-people-are- missi...](https://gds.blog.gov.uk/2013/10/21/how-many-people-are-missing-out- on-javascript-enhancement/)

<comment><USER50> Do you have any updated sources? This data is from 2013.

<comment><USER49> No I don't, unfortunately. But I imagine that this figure will be fairly stable for the foreseeable future. There are always going to be people with disabilities who turn off JavaScript for accessibility reasons, and privacy- conscious individuals and corporations that have it disabled.

I think the parent comment's exaggeration of the number is bad form for web developers, and goes against the spirit of the web, which I believe is based on the principles of universal access and inclusiveness.

<comment><USER50> I have no data here, I would guess that number would be lower today than in 2013. My reasoning being the proliferation of smart phones in second and third world countries and the general trend of more web 2.0 users, though I fully accept that this is just an educated guess.

As a web developer, for me the problem is that writing anything modern without JavaScript is a much harder task that only really benefits a small minority of users. I'm unfamiliar with any disabilities that would require JavaScript to be turned off but I could see this being a valid usecase.

<comment><USER49> Just found this from 2016: [https://blockmetry.com/blog/javascript- disabled](https://blockmetry.com/blog/javascript-disabled)

That post also implies that the figure is stable.

Your second paragraph is a perfect example of the kind of attitude that I think is problematic. Yes it's more work to accommodate non-JS users. But so is building ramps for people in wheelchairs. The discrimination gets "baked in" to your/our development processes, and you/we justify the discrimination based on the idea that the only ones who will suffer from it are a "small minority".

It's like assuming that everyone uses a mouse. Clearly some don't, and if we develop the web based on that assumption, we're going to leave a lot of folks in the lurch. We should hold ourselves to a higher standard[0].

[0][https://webaim.org/standards/wcag/checklist](https://webaim.org/standards/wcag/checklist)

</comment></comment></comment></comment></comment></comment></comment><comment><USER51> This isn't about JS frameworks, it's about rendering data on page load.

Whether the server renders the data into HTML it responds with or there's some XHR request made by a client library (including VanillaJS) or framework, you're always incurring some cost. But data-on-load what users generally want and expect from modern web apps.

Having recently built an almost 100% HTML-based web app, I can tell you the user interactions feel unpleasant. Removing JS from web apps is silly at this point.

</comment><comment><USER52> Comparing a barebones view layer libraries (Vue, React etc.) with full frameworks (Angular, Ember etc.) is not comparing apples to apples.

<comment><USER53> I'm assuming you didn't read the article? He was comparing react + redux + react-router using create-react-app compared to Angular. So it is an apples to apples comparison.

<comment><USER52> Even if you add redux and react-router you still don't have feature parity. The comparison is inherently arbitrary. All I really know from his experiment is if I load up a feature rich framework vs. a bare-bones view+route+state combo the framework is going to be heavier. Is that really news?

</comment></comment></comment><comment><USER4> I am surprised by the fact that author completely ignores client side caching using browser and service worker, as well as SSR.

One key point of PWA is that it works offline and loads faster after after first load (even after you deploy new changes, with basic service worker configs from sw-precache). So you need 1 second for _first load only_. Subsequent loads are much faster.

</comment><comment><USER54> Why would you compare React/Vue vs Angular??? I don't think the author understands the difference between frameworks and libraries.

React vs Vue can be a valid analysis, same as Angular vs Ember. It's silly to benchmark React/Vue vs Angular/Ember.

Lets create a benchmark of ponies vs zebras/horses!

<comment><USER55> Seems pretty clear they understand the difference judging by what was written in the article.

<comment><USER54> Do they though? have you read the article? Medium says its a 5min read. 5min of a completely wasted time. Btw its says right here that react is not a framework [https://reactjs.org/](https://reactjs.org/).

Always amazed how people calling themselves "engineers" while being unable to grasp such simple, basic nuances.

</comment></comment></comment><comment><USER56> This is why obsoleting frameworks by expanding the native component model of the web is good for the world.

<comment><USER57> I'm really loving just casually writing websites for Rust web frameworks because I get to try out all these new tricks in html / css / ecma7 to avoid pulling in any third party libraries. Then you get goodies like compile time templates and regexes while having only 30ms from initial request to tls handshake to routing, request handling, response and rendering all done (on the same network).

</comment></comment><comment><USER58> Including megabytes of unrelated .gifs in the blog post kind of undermines the point here...

<comment><USER55> I don't agree. Those gifs are explicitly added. The author's point is about load times that are relatively unavoidable.

</comment></comment><comment><USER59> Please stop using gzip size for these sorts of comparisons. One, gzip affects download speed, but _not_ JS evaluation and compilation. Two, it makes the libraries seem much smaller than they actually are.

<comment><USER60> _> One, gzip affects download speed, but not JS evaluation and compilation_

They measured scripting time directly, which is a much better measurement than uncompressed bytes.

 _> Two, it makes the libraries seem much smaller than they actually are_

Would you prefer they gave unminified sizes as well? The gzipped size of a library lets us predict how long it will take to download, and how much that will cost for users.

</comment></comment><comment><USER61> Of all the client-side JavaScript libraries available for making responsive web applications, I've found that Elm suits me best. I haven't done any benchmarks, but it feels faster than React and Angular to me and the Elm compiler eliminates virtually all runtime issues before I ever deploy my code.

Another alternative to large client-side libraries I've been looking at lately is Phoenix's LiveView. It is a server-side rendering library that uses web sockets. I've only just started using LiveView, but so far my pages seem to perform better than React or Angular.

<comment><USER62> Given Live View was only announced two months ago, is in development, and hasn't been released in any form whatsoever, I'm curious as to how you've managed to use it to make that comparison?

<comment><USER61> Duh! Sorry, I meant Drab. I had LiveView on the brain. You can check it out here: [https://tg.pl/drab](https://tg.pl/drab)

</comment></comment></comment><comment><USER26> He's not using SSR which is wrong in 2018. Vue's startup performance will also supposedly double in the next version. Looking forward to that.

<comment><USER63> Do you mean halve? That's what I took away from the twitter post by the creator Evan You.

<comment><USER26> Performance double, time halve. I think my comment was fine.

</comment></comment></comment><comment><USER64> To better make sense of this data it needs to be compared against the time needed to build a non-trivial interface using each of the frameworks.

<comment> 10-6 This is exactly right. A lot of posts regarding loading times or bundle sizes for frameworks/libraries forget that there are trade-offs when it comes to building an application.

The correct way to frame this is: use a _slow /clunky/large/etc_ framework vs. build everything from scratch (which comes with its own costs).

Sure, you can optimize parts of your application to speed up the JS portion or even remote it completely [1], but it's not always as simple as "your framework is making your application slow so you should think about ditching it."

This article actually ends with a very reasonable conclusion in the section "Are Frameworks Evil?" but I've seen plenty of articles where the author doesn't offer an alternative to some library/framework [2].

[1] [https://twitter.com/netflixuie/status/923374215041912833?lan...](https://twitter.com/netflixuie/status/923374215041912833?lang=en)

[2] [https://dev.to/gypsydave5/why-you-shouldnt-use-a-web- framewo...](https://dev.to/gypsydave5/why-you-shouldnt-use-a-web- framework-3g24)

</comment></comment><comment><USER65> The main conclusion from this article isn't accurate. This is one reason to load common libraries from a known CDN so that if any website has downloaded that library version previously, and the user's cache has not been cleared or expired, the next time that library can just be loaded from cache, even if from a different website.

<comment><USER66> Historically that was a major security issue. It's less of an issue now that SRI exists (you're using SRI, right?), but it's still a problem when you have a broken pathway to the CDN but your site loads OK. Well, assuming you're not building with progressive enhancement. You're building with progressive enhancement right?

<comment><USER67> sorry what is SRI ?

<comment><USER66> Subresource Integrity. It's designed specifically for this scenario: a resource included into your page from a seperate server that could potentially be compromised. Your hash the known good resource and include that hash in your script or link tags as an integrity attribute. When the user's browser downloads the resource it'll hash it again and check for a match. Only if the two match will it parse and execute the contents.

[https://developer.mozilla.org/en- US/docs/Web/Security/Subres...](https://developer.mozilla.org/en- US/docs/Web/Security/Subresource_Integrity)

</comment></comment></comment></comment><comment><USER68> It doesn't mention framework versions..

</comment><comment><USER69> Not using server side rendering, not using a cdn or some form of caching. Not sure this is a good test of anything other than how to fail at implementing SPAs.

</comment><comment><USER70> This person doesn't know a thing about Angular and he is benchmarking it?! Angular is not loading the whole RxJS and I doubt AOT was setted properly.

This is a misleading paper again. I would like that 3 experts on each framework do their best to provide the same app and then we could compare.

<comment><USER71> This isn't a research paper.

</comment></comment><comment><USER72> > in exchange for paying this cost, you gain maintainability, developer > efficiency, and (hopefully) better performance during runtime.

You also gain responsiveness to user interactions, a programmatic way to reason about your application/website & and some animation-sugar if you want. Don't throw the baby out with the bath water simply because people follow trends instead of good design practices.

<comment><USER73> And in my case, less Java I have to read or write on the back end :-)

I agree, though - it's much easier to reason about the data integrity and security when there is a smaller code base on the back end, by virtue of splitting user interface concerns off to a front end code base.

I've been programming since the 80s, but I'd rather write FP code in JS than OOP code in Java - so long as I'm not doing anything to make the app work badly for people. So I've gravitated towards front end work the last 3 years or so.

Another rant for another time...

</comment></comment><comment><USER74> > your React application will never load faster than about 1.1 seconds on an > average phone in India, no matter how much you optimize it

Weren't your assets deployed on Netlify's CDN? Shouldn't that make location roughly irrelevant (assuming enough PoPs)?

<comment><USER75> He's referring to the typical phone performance (parse and execute) and network bandwidth for an Indian user, not the network latency.</comment></comment>