Mailpile: Let's take email back - <AUTHOR> http://www.mailpile.is/ <comment><USER2> I'm in the crowdfunding business, so I feel strange saying this, but I'm really turned off by this model of 'pay us in advance for developing a product that hasn't been market tested or validated,' especially when asking for a sum as large as is $100,000.

What happened to building something and selling it?

According to the campaign page, the need is so great because:

"We're asking for a lot of money, so of course you should know why. $100.000 means paying two people $4166 a month for a year, including all taxes, insurance and other fees."

I suppose people get what they pay for, but I find it insulting to ask me to pay your salary for a year so that you can avoid risk.

If you were truly concerned about online privacy, you'd build it anyway. So is privacy the mission, or the pitch?

<comment><USER3> > What happened to building something and selling it?

Since they are already providing it for free to those who need it, and producing at as free software, selling it as a finished product is unlikely.

> If you were truly concerned about online privacy, you'd build it anyway.

Many people do altruistic work during their free time. crowdfunding means you can do the same work, but not be limited by what scraps of time that exist after work.

The extremely few people in the world that would quit their job to do altruistic work is few. They are so few that almost every time it happens, it get posted here as _news_.

If I made a poll, asking how many people here cared strongly about something in the world, I would get close to 99% hands that said yes. If I then asked how many of those people would agree to quit work to work altruistic on that subject, how many hands would I see?

<comment><USER4> I have actually been working on Mailpile "altruistically" for a while, but sorely missed the ability to dedicate myself to it full time. Thus the fundraiser.

<comment><USER2> Totally get it, but why not share some of the risk with your supporters and meet them halfway? Asking for a full (decent) salary for two people for a year places all of the risk on your supporters and makes you look uninspired.

Edit: I can't reply to child, so I'll do it here.

It has nothing to do with a free ride. I don't want the product because the solution isn't right for me, but I would much prefer to pay a monthly fee for something than give a 'free ride,' as you call it, to a developer, in advance.

I have no sympathy for the family argument, because I spent a year building a business on the side while I was employed full-time, so that when I quit my job, I could support my family.

I believe there's something to be gained when the venture does not include a parachute.

<comment><USER5> I don't understand your objections at all. These guys are making a proposal, which individuals are free to accept or reject as they see fit. Those who accept it know what they're getting themselves into and accept the risk. No one is getting scammed. If these guys want a parachute, that's their prerogative. Just because you wouldn't take the same approach doesn't mean the approach is a bad one.

<comment><USER2> Point blank - I'm tired of people abusing crowdfunding as some sort of money grab (Spike Lee's recent Kickstarter as an example) and because I'm involved in the industry, I have a vested interest in seeing crowdfunding, as an industry, prosper.

These guys probably have a great product, and are probably great people, and for that reason alone, I feel bad that my comment leads the thread. However, I never said anything that suggested this was a scam, and I don't believe it is. Like I said, people get what they pay for. It's just that I see this trend getting worse, and I think long-term it will hurt crowdfunding.

<comment><USER5> I can see where you're coming from and I think that your argument would be stronger if you took your own feelings out of it. For example, you say that you find it insulting that they'd ask you to pay for their salary. Well, why should it matter to anyone else that you feel insulted? If that's your reaction then they're probably not talking to you. Besides that - why should anyone care about whether you feel insulted? You also mention that you feel no sympathy for the family argument because of your experiences. In this context, why does that matter? I don't mean this to be dismissive of you personally; I'm just trying to share my minor analysis.

You also say that you're not suggesting they're running a scam but at the same time you compare their outreach to a money grab.

I don't know, perhaps your arguments are very good ones and I am a heartless bastard because they don't sway me. At their root, it seems like you're saying that these guys are proposing something that violates your sense of fairness somehow. I'd be interested in seeing an argument that applies more universally, and not just that you personally find it unfair.

Myself, I think the cool thing about crowdfunding is that it's so open and free. I might personally find it frustrating or unfair that money gets allocated in a way that doesn't match my values, but too bad for me!

<comment><USER2> You make some great points, and I can't say that any of them are off-base. My comment is obviously filtered by my experiences in the industry, which might be unfair to the founders in question.

However, my comment seems to have resonated with many others, so could be offered as advice for ways to improve the campaign.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER6> This is exactly what I've been looking for! It seems the product is aimed at technical users (at least initially), though there are some technical details I didn't see in the video, description or FAQ:

\- This is an MUA, correct? Based on the features on the project page, it sounds like MailPile will not act as an MTA or MDA, and is predominantly interfacing with mbox/maildir. I see features for IMAP and POP3 on the roadmap, but its not clear if using those protocols is idiomatic for MailPile.

\- How is PGP/GPG handled? The server-side code for MailPile must have access to my secret key, correct? Is MailPile's web interface then accessible via HTTPS (given the proper cert)?

\- Is there a plan for a key management interface?

\- It sounds like the MUA itself (MailPile) is a server, and it would access maildir/mbox directly. Is there any API planned for accessing that data through MailPile's programatically, or is MailPile's main goal to provide a browser interface?

\- This might have been covered, but will the web interface support mobile as well?

Thanks for working on this...it really sounds like a great project!

<comment><USER4> Thanks! The idea is you run Mailpile on a machine you control (it is an MUA), either your laptop (and connect to localhost) or a home server like a plug computer or something like that - basically somewhere where you feel comfortable storing your secret keys.

And yes, API access to mail is something we already support, every "command" can return either HTML, plain text or JSON. Probably XML to come as well.

We'll have to help with key management, otherwise it won't be usable by normal folks.

Mobile web support, yes, probably sooner than later.

Hope this clarifies!

<comment><USER7> I'm not sure how you plan on doing it, but I think you should tie everyone's public keys to their names/id's. So let's say you know someone who uses Mailpile. You shouldn't have to ask him what's his key. You should just "enable PGP" (if it's not default, though maybe it should be), and he should just get the e-mail.

So try to do the key management as automatic and "out of the way" for users as possible. That's the biggest hurdle with using PGP right now.

<comment><USER8> But that just defeats the whole purpose of PGP. If you don't even know where the key is coming from, how do you know it's secure?

<comment><USER9> Easy: [http://www.xkcd.com/1181/](http://www.xkcd.com/1181/) :)

</comment></comment></comment></comment></comment><comment><USER10> I applaud these goals however if we were to ever consider moving from Google Apps (which we would) it would only be for an open source project given that privacy is becoming the more critical issue as the years go on. It would be great to see a subscription based email service such as this where the source is fully available for scrutinizing and patching.

Edit: Great, so the source is already available.

<comment><USER11> This is an open source project.

Also, if privacy is important for you, you shouldn't look into subscription services, whatever software package they use.

<comment><USER10> That's fantastic. I didn't see any information about this on your page so you might want to make this clearer (I assure you I read it first before commenting, then went back and saw the Fork on Github ribbon, the only indication).

By subscription I mean that I am happy to support an open source project with regular funding. Initial funds from Indiegogo are not sufficient to maintain a long-term service and we'd be hesitant to move our most important business infrastructure to something that may not be updated regularly as the security environment changes. If you were to look into making this a business with recurring revenue I believe you would find there is a lot of support.

<comment><USER12> there's a big red line across the top right corner pointing to github (and licencing info there). there's another link to github near the end. apologies for a previous post that used strong language, but you really can't have looked very hard.

(you know, i just read chapter 2 of 'how to win friends and influence people' which tells me that criticising people never works because they simply work harder to justify themselves. it's the most depressing thing i have read in a long time, but it seems to be horribly, universally true. and it's so frustrating when the world seems to be populated by incompetent idiots. how do you convince someone 'nicely' that they complained about something without even reading it?)

<comment><USER13> Regarding your addendum, if you really view the people as incompetent idiots, I'd argue you're not all that interested in winning them as friends.

In any case, just give the person the information they need without attacking them -- they won't feel defensive and they'll probably feel pretty stupid on their own.

For bonus points, try to empathize and think about something stupid you once did -- surely you've done some idiotic things in the past.

<comment><USER12> i don't deny doing stupid things. look, here's an example from my recent posting history - [https://news.ycombinator.com/item?id=6121287](https://news.ycombinator.com/item?id=6121287) and the follow-up [http://www.acooke.org/cute/HyperloopW0.html#Mon29Jul20132135...](http://www.acooke.org/cute/HyperloopW0.html#Mon29Jul20132135420400)

what depresses me is that people don't admit it, learn, and move on. instead we get "well, i am showing how important it is to make this information more explicit because everyone else is as dumb as i am". no. we are not.

ps and no, i am not interested in winning you as a friend. what makes you think i would be? i am interested in influencing people. i want people to be smarter. it's frustrating that this can't be done by simply pointing out the correct answer. instead, people need to be molly-coddled into changing their minds without noticing. by pretending to be friends. an enormous amount of effort is needed to effect the smallest change.

i don't deny it. i think i need to read and learn from that depressing little book. but god i wish people would think a little more.

<comment><USER14> When you view the world as being filled with incompetent idiots, you're seeing the world through your own filter...not anybody elses. Realize that everybody on the face of the planet has their own filter. Human relationships are built upon being able to empathize with others and perhaps glimpse life through their filter.

Not seeing the fork ribbon in the upper-right isn't about someone being "dumb" anymore than you not knowing how to change your own oil, fix a leaky faucet, put in a new electrical outlet or build a fucking house, makes you dumb.

You didn't just "simply point" out the correct answer. You added, "you really can't have looked very hard." which is an assumption on your part; a conclusion you reached based on your filter of the "world seems to be populated by incompetent idiots".

<comment><USER15> Very much this. As someone with a strong interest in everything neurology/psychology/intelligence/learning/teaching/etc, this is the most accurate description of what is going on.

"Incompetence" isn't toxic, viewpoint and attitude are. There will always be someone "dumber" than you, and there will always be someone "smarter" than you. Once you realize this and internalize it into your world view, you will come to the same conclusion that book teaches, and that is one of _cooperation_.

Like my parent here alluded to, you didn't just point out an answer, you projected your thought process onto another person. Why do some people get depressed at what seem like tiny insignificant details, while others live happily in the worst of global conditions? The answer is _Perspective_. The pain/difficulty anyone feels is very much real to _them_ , it is not an objective measurement -- so when you project the idea that a person just _couldn 't have tried very hard_, you make the assumption that you know what it's like for that person to 'try' anything in that context.

If you look at my own comment history, you'll see that I have dealt with the difficulty of explaining myself to others quite a bit, but also that I try to turn it into a productive thought-provoking discussion with a neutral tone and strong points. For example, what I am saying right now is very similar to a direct attack on your ego (and I wouldn't blame you if you took it as such), but I try to present my words in such a way that will invoke action from your rational 'executive brain' long before it hits your emotional 'reptilian brain', if that makes any sense?

The only real course of action that makes sense _is_ cooperation; any other interpretation would basically implicate that either the world revolves around you, or that everyone is like you, and those ideas should be clearly false to everyone.

</comment></comment></comment></comment></comment></comment></comment></comment><comment><USER16> Here's why I want something like Mailpile:

Right now, _every single email I receive is encrypted._ I have my public GPG key on my mail server, and every incoming email that's not already encrypted is encrypted using that public key. That way if the anyone compels my VPS provider for access, they just get a bunch of encrypted email.

So my problem isn't receiving or encrypting email, it's reading it. The only real option I have right now is Thunderbird, which isn't great, and is no longer under development. Mailpile doesn't look like a mail service to me, it looks like a browser-based but locally-hosted MUA, which might be the remedy to Thunderbird that we need.

</comment><comment><USER17> I just switched back to Zimbra (also open source) after years with Google Apps. It's been a spectacular experience so far, and it's nice to have email fell like email again (not to mention being in full control).. It has everything I need: a rich web client, good smartphone phone support, good spam filtering, and a good community.

<comment><USER18> Another vote for Zimbra.

Seriously, it's not that hard to install and setup.

Once you have it all going the capabilities are immense. It also lives on your machine, using your SSL certs, and using as much in-place HDD encryption as you want.

This is how you take your privacy back. You care enough about it to do it yourself.

<comment><USER19> It's also how you give yourself a headache having to sysadmin a machine. I myself used to run Zimbra and switched to Google Apps. Too much irritation with backups, monitoring, security, et al.

</comment></comment></comment><comment><USER20> They talk about privacy, but MTAs like Postfix can already encrypt mail via TLS when moving it to another host¹. GPG gives us a better guarantee, but more user overhead, and unreadable by many recipients.

Additionally SPF gives us a way to check if the sender address has been forged. GPG signing is more robust, but again, more user overhead.

Not to mention, I already have S/MIME support in my mail application and can get GPG support via a plug-in, but I use neither, because few recipients can handle it.

So what is new with Mailpile? What is it supposed to change?

In my opinion, if the goal is to make email more secure, we should look into ensuring that all MTAs is setup to support TLS and use it when delivering mail to other hosts (AFAIK Exim4 only announces STARTTLS when connecting to its submission port).

Getting SPF records setup would also be a plus.

This would go a long way in making email more secure, and only requires action from administrators of mail domains.

¹ [http://www.postfix.org/postconf.5.html#smtp_tls_security_lev...](http://www.postfix.org/postconf.5.html#smtp_tls_security_level)

<comment><USER21> Exim can announce TLS to all comers. Set the tls_advertise_hosts option to '*'. (I believe Debian sets this by default.)

[http://www.exim.org/exim-html- current/doc/html/spec_html/ch-...](http://www.exim.org/exim-html- current/doc/html/spec_html/ch-encrypted_smtp_connections_using_tlsssl.html)

</comment></comment><comment><USER22> I'm super-happy that someone is at last trying to build a self-hosted competitor to Gmail.

Sure, I can set up my own mailserver already, but the amount of effort it takes is too much compared to just setting up a Gmail account. If they can get set-up and spam-protection right, then this could be huge!

<comment><USER23> Bonus points for it being installable with one "docker build" command. I would definitely give it a shot, at the very least.

</comment></comment><comment><USER24> Nitpicking:

Thumbnails shouldn't be high-res pictures that actually are scaled to stamp proportions. It's slow to appear on poor connection and for a moment I thought there were no screenshots and just words like "compose". And it slows down my poor 2nd gen asus notebook.

</comment><comment><USER4> Bjarni here, lead dev. Happy to answer any questions.

<comment><USER25> "Mailpile is a modern web-mail you run on your own computer."

I think this sentence should be the first thing anyone sees when they go to the page. The text at the moment assumes everyone already knows roughly what Mailpile is, so I didn't figure it out until I read HN comments.

Also, are there screenshots somewhere?

<comment><USER26> I think that this sentence needs to be gotten rid of completely. \- Webmail is fundamentally not something that can run on your own computer. The word is "web" mail, not desktop mail, it's just going to confuse users.

In addition to it being confusing, because of dynamic IPs and residential port blocking you may be able to run it on your computer, you just won't be able to do anything with it due to other internet infrastructure... \- Sending mail from home is almost guaranteed to fail sometimes/often, due to dynamic ip ranges which are frequently blocked. \- Port 25 and port 80 are blocked by most major american ISPs these days for residential services. Making this unusable from a home server. Not to mention it's against many ISPs terms of service to run a server from home without paying for a business package. (That's right, it's not just google fibre) \- SPF records and other forms of email authentication? You would also need a third party DDNS service if using a dynamic IP.

So with all of that said, I like the interface pictures. It could be a good competitor to webmail clients like roundcube and friends.

<comment><USER21> > Sending mail from home is almost guaranteed to fail...

It's an MUA - it can sent mail just as well as any other MUA. Use your ISP's smarthost if you are using it from home.

> Webmail is fundamentally not something that can run on your own computer.

This statement is plain wrong, even if by "your own computer" you really mean "your desktop box". I can run any server I like on my desktop box. If it has a well designed installer, then it would be as easy to set up as a "normal" app - the end user might not ever know the technical details.

> The word is "web" mail, not desktop mail, it's just going to confuse users.

Now I do agree with you. The product seems to be a little bit of both, so there is some potential for confusion. "Web-mail you run on your own computer" does seem to explain it pretty well though.

</comment></comment></comment></comment><comment><USER27> Weird that the description of what it actually _is_ is below the fold :-) However: _Mailpile is a modern web-mail you run on your own computer._

</comment><comment><USER28> I don't get it, how is it better than RoundCube[0], Horde[1] or any other webmail clients? Out of the six "reasons", three of them are already there in the wild (privacy, self-host and no ads), I'm pretty sure two more can be found/coded (search & encryption) and the last one (speed) is completely subjective to one's usage.

I mean seriously, what's the appeal to paying for Mailpile when I could just use an open-source webmail client on my server? (Which I will by the way, thanks for the idea)

0: [http://roundcube.net/](http://roundcube.net/)

1: [http://www.horde.org/apps/imp](http://www.horde.org/apps/imp)

PS: Here are more: [http://www.noupe.com/ajax/10-ajax-webmail- clients.html](http://www.noupe.com/ajax/10-ajax-webmail-clients.html)

</comment><comment><USER29> It is very good you are attacking this problem, but is this the right way to attack the email problem?

According to Paul Graham frightening startup ideas [http://www.paulgraham.com/ambitious.html](http://www.paulgraham.com/ambitious.html), email is a bad ToDo list, How are you going to implement a better Todo list?

I have created a alpha prototype which attacks these problems conceptually, namely, Message Classification, Message Sharing, Bidirectional Messaging, Pull Messaging, Sender Revocation, Message Expiry, Centralized Attachment etc as a Mobile App but approached this as a separate todo protocol using the Push Messaging Infrastructure.

You can download the working app from Google Play [https://play.google.com/store/apps/details?id=priya.pullgrid...](https://play.google.com/store/apps/details?id=priya.pullgrid.v1) and can see a website created [http://www.pullworld.com](http://www.pullworld.com) (Undocumented - You can download the Frontend App Html Source - Have not even shared in HN as Show HN because it is incomplete in documentation). The purpose of the Prototype is just as a proof of concept and not really to solve the email problem.

I would love to share my knowledge/architecture if you are interested, so that you can really attack this problem as envisaged by pg and since you are planning to do it open source and with email, would love to contribute if you are thinking of mobile in the future using Html/OpenGL based client.

<comment><USER30> I'm not sure about that idea. A tongue-in-cheek evaluation: [http://pastebin.com/Xf1RffNV](http://pastebin.com/Xf1RffNV)

</comment></comment><comment><USER31> This project has roots at least some years back, so it's not starting from scratch with lofty ideas that are unlikely to come to fruition. Solid team with a great ideology!

</comment><comment><USER32> Urrgh, if you're going to ask for money, at least ensure you use 'its' correctly on the fundraising button (i.e. it should be 'reach _its_ fundraising goal'. I really wish everyone would read the Oatmeal piece on apostrophes [http://theoatmeal.com/comics/apostrophe](http://theoatmeal.com/comics/apostrophe)

</comment><comment><USER33> I like the concept - but it's missing a discussion of how they handle what I consider to be (by a pretty large factor) one of the most important feature in email - Anti-Spam.

Gmail (which I'm not suggesting is an ideal, just what I happen to use) doesn't even bother to show me my spam folder - and, in the last week, it's redirected 1178 messages there.

<comment><USER4> Absolutely. I'm the lead developer - I spent almost 6 years working on anti- virus and anti-spam, so rest assured we know this is a big deal. :-) There is alot of quality stuff in the open source world already which we will be able to build on.

Update: To clarify why we left it out of the pitch - we just took it for granted that you can't have a functional e-mail client without dealing with spam.

<comment><USER34> Nothing I tried ever handled spam as well as Gmail, I was drowning in it, it's the number one reason I use Gmail. I see so little spam now I think if everyone used Gmail the spammers would have to give up.

There are plenty of nice-enough webmail UIs around, and people complain about the speed of Gmail but I find it perfectly fine. Spam's the thing.

</comment></comment></comment><comment><USER35> Noble goals. With all the NSA action, this is something whose time has come. Where do I donate?

<comment><USER36> I get what you're saying, but what's to stop them from monitoring the transmission and how can you be sure the receiver of your sent email is running a secure email as well?

I don't think this does much to stop snooping.

<comment><USER37> It doesn't guarantee that any specific e-mail is snooping-proof, but it makes it hard for someone to trivially look through all your e-mails from the last five years.

</comment></comment></comment><comment><USER38> if you need free site hosting PM me. i run an infrastructure company and are looking to support open source projects (we rely heavily upon open source).

</comment><comment><USER39> It uses CGI eh? Old school, I like it!

* It's a bit hard to find where the HTML/JS templates are. I think that's important for folks who are good at design and willing to help the project.

* search.py, looks like it stores indexes in memory, is that correct? Will it work if I run multiple instances of httpd?

</comment><comment><USER40> Two things: 1\. IMO sup ([http://supmua.org/](http://supmua.org/)) is more mature and does about the same. However it's interface is a curses based one which is a plus and a minus. 2\. Why not use an existing search products (like xapian)?

<comment><USER4> This is actually answered in the FAQ on our Indiegogo campaign: [http://igg.me/at/mailpile](http://igg.me/at/mailpile)

Mostly, the reason we do things the way we do, is because one of our primary goals is to make an end-user desktop app. Packaging is therefore a significant task and minimizing dependencies will help a lot.

We still have quite a bit of work to do on our website and message.

</comment></comment><comment><USER41> This reminds me of Zoe. [http://zoe.sourceforge.net](http://zoe.sourceforge.net)

Back in 2001 or 2002, I was following the progress of Zoe, what I thought to be a very promising new approach to email archives. It kept your email in mbox or eml files, used Lucene for indexing and search and then provided a web interface on an embedded web server.

The Zoe project is no longer active (the homepage is dead and the files are gone), but this MailPile sounds very much like it. Almost exactly like it. I hope it does better.

</comment><comment><USER42>
        Mailpile stores in RAM about 180 bytes of metadata per message (actual size depends largely on the size of various headers), but Python overhead brings that to about 250B. This means handling a million messages should consume about 250MB of RAM - not too bad if you consider how much memory your browser (or desktop e-mail client) eats up.

Totally killed my interest, I want to run this on a small server (e.g. NAS), for everyone in my family

</comment><comment><USER43> I'm not sure what responsive framework you're using, but the layout doesn't look good and the logo is messed up on mobile browsers.

</comment><comment><USER44> Hi guys,

If you're interested in this, you might also want to check out:

[https://echoplex.us](https://echoplex.us) (overview) [https://chat.echoplex.us](https://chat.echoplex.us) (in action) [https://github.com/qq99/echoplexus](https://github.com/qq99/echoplexus) (github)

Lead developer here, FYI

</comment><comment><USER45> This looks great!

I have a couple of questions:

\- Any plans for a plugin/extension system?

\- Are there any screenshots of the interface anywhere?

Thanks! : )

<comment><USER46> At around 1:30 in their Indiegogo video they show what I assume is the user- interface of the app. Looks nice.

[http://www.indiegogo.com/projects/mailpile-taking-e-mail- bac...](http://www.indiegogo.com/projects/mailpile-taking-e-mail-back)

</comment></comment><comment><USER47> Thumbnail images under the interface section are coming up broken.

</comment><comment><USER48> Should they succeed, let us hope that the mobile version of the webmail client will function better than the mobile version of your static site. Pretty unusable in MobileSafari.

<comment><USER49> Hey topherwhite I'm the UI/UX designer of Mailpile- totally aware of the poor rendering on mobile at present. Working on fixing that- was swamped with other aspects of the campaign launch. Thanks for your feedback :)

</comment></comment><comment><USER50> Very timely. Might be a viable alternative to NSA's Gmail.

The tech lead is from the Empire though - an ex-Googler - not exactly a privacy caring company :) Maybe that's why he left :)

</comment><comment><USER51> It's still decidedly aimed at techies though. I guess it'll be another few years before the benefits of a Free Software Gmail competitor will be accessible to the layman.

<comment><USER49> Hi there. I'm the UI/UX designer on the Mailpile team. Making every aspect of MP easy for non-techies is my primary concern. We aim to be easier than Wordpress with our Alpha release in Jan 2014. Please give me any feedback concerns with out to achieve this goal :)

<comment><USER52> Feedback: IMAP and POP3 would let everyone continue to use the native apps that they know and love for their UX. The web sucks for apps. It just does. Making _robust_ dedicated native apps for each major platform would be a real differentiator.

<comment><USER49> Thanks for your feedback! We definitely would like to build native (mobile) apps for Mailpile- small (and important) steps, first though :)

</comment></comment></comment></comment><comment><USER53> this looks awesome, lots of luck.

Am I understanding this correctly that this is basically a selfhosted imap/pop/smtp frontend?

I am curious how you guys would use it, i.e. where would you actually host your emails? Running your own mailserver on a vserver sounds fun and not to expensive but I don't know if I want to maintain something like that in the long run. If something breaks this just sounds like a lot of work.

You guys pay for mail hosting? Use the one that comes with your website domain?

</comment><comment><USER54> What is wrong with e.g. Dovecot/Postfix + Roundcube ? How would this be any better? Is this intended to be a "close as possible" gmail clone?

</comment><comment><USER30> How is this any different from every other desktop mail client (eg. Thunderbird, Inky, Evolution, Notmuch, Sup, Mutt, ...)? What makes this better?

<comment><USER4> This is a desktop mail client. It is also a web-mail you can run on a server somewhere if you so choose. It depends on how it is deployed.

Contrasting Mailpile with other tools, one difference is that the basic design is that of a search engine, not a tool for reading mail from folders. Most current desktop mail clients are built on top of a bad paradigm, in my opinion.

Another exciting thing about this model, is that since the UI is a website (of sorts), we can leverage the collective experience and creativity of the web design community. That is a much, much larger pool of talent than UI designers who know C++ or Objective C, or whatever.

Finally, making the app a web server means you get an API to interact with your e-mail almost for free.

<comment><USER55> I've been working on an extensible console-based mail-client for the past while. ([http://lumail.org/](http://lumail.org/))

I abhor the idea that clients only ever interact with messages from a single folder.

</comment></comment></comment><comment><USER56> Site is barely readable on the iPhone:

[http://imgur.com/IbwipKQ.jpg](http://imgur.com/IbwipKQ.jpg)

<comment><USER49> Thanks! I'm working on fixing mobile rendering ASAP :)

</comment></comment><comment><USER57> We all need to back this project to get rid of the monopoly of Google.

<comment><USER58> Google doesn't have anywhere near a monopoly on Email. They are barely ahead their two largest competitors.

<comment><USER57> and what planet do you live in? everybody I know has a Google account only because of either Gmail or Youtube.

<comment><USER58> I live on the planet not defined only by your personal experiences.

<comment><USER57> Ok then go research how many companies in silicon valley give their employees Gmail. All the companies I've been worked provided Gmail only because there is really not a good open source option. And Google accounts is a huge monopoly that obviously sounds bad enough to hurt your feelings. But this is the reality. Google is a valley of shit and most of us want to move on from it.

<comment><USER58> Criticizing Google doesn't "hurt my feeling". I've moved away from Gmail and other Google services (not all) a few years ago because I disagreed with their options. But that doesn't mean I have to agree with nonsensical claims such as Google having a monopoly on email.

Maybe they do on Silicon Valley, but I'm sure you realize that's a pretty insignificant subset of email users.

<comment><USER57> I believe we aren't in the same planet.

<comment><USER21> You sound like you're getting pretty emotional. Time for a cup of tea and a sit down?

<comment><USER57> yeah, I'm really upset to not have any alternative of Gmail. What do you recommend ?

</comment></comment></comment></comment></comment></comment></comment></comment></comment><comment><USER59> i like it, and im willing to pay $5. to make it happen

</comment><comment><USER60> I wonder how long until they put a NSA rootkit in it.

</comment><comment><USER61> Prime member... hope this project goes through.

</comment><comment><USER62> This project desperately needs a logo.

</comment><comment><USER63> You have got to get rid of that logo.

<comment><USER49> Mailpile UI/UX designer here- thanks for the "feedback" I guess... What bothers you about our current logo, specifically?

</comment></comment><comment><USER64> Who are we taking it back from?

<comment><USER49> Firstly, the centralized cloud webmail companies (Gmail, Hotmail, Yahoo) by giving people the ability to self host their email with an awesome webmail solution! ~Mailpile Designer

</comment></comment><comment><USER65> not another shitty mail service

<comment><USER49> Wow, that's a useful comment. Thanks.</comment></comment>


Place Kit – The future of augmented reality mobile apps - <AUTHOR> https://medium.com/inborn-experience/place-kit-the-future-of-augmented-reality-mobile-apps-25a20907316b <comment><USER2> This is a really great UI kit, very consistent and much needed in the space.

<comment><AUTHOR> Thank you. Let me know if you'll have any requests.

</comment></comment><comment><USER3> Please improve your written english. This is hard to read. Eg: "when" != "once" ...

<comment><USER2> may not be the intention but this came across as an arrogant comment, not everyone's first language is english.

<comment><USER4> A quick proof read by an English speaker might have helped.</comment></comment></comment>


Rails is not your application - <AUTHOR> http://blog.firsthand.ca/2011/10/rails-is-not-your-application.html

<comment><USER2> Sounds like you shouldn't be using Rails if this is your philosophy. I'm of the school of thought that if you're going to use a framework when building an app, then you should lean on it for just about any task that it will let you. Especially if you're using a well-maintained and widely used framework like Rails. There are multiple benefits to this approach. For one thing, it makes it easier to onboard new team members. There are tons of people who are Rails experts, and are very familiar with its paradigms and features. People like this can very quickly get up to speed with just about any Rails project. A large team can also communicate and work much more efficiently if they are all using the same code structure/helpers/etc. If you're a one man show, it might work just fine for you to build specialized gems for all of your "application" (though the line you've drawn between application and framework seems pretty vague) though there is going to be a fundamental lack of structure to your entire project. It'll make sense to you for a little while until it gets so big that even you don't understand it. It's going to be difficult to perceive test coverage, and in practice, all those little gems are going to vary pretty wildly in structure.

You'd be better off just using Sinatra and including ActiveRecord, and whatever other Rails libraries you want to use.

<comment><USER3>
      > Sounds like you shouldn't be using Rails if this is your philosophy.

      > You'd be better off just using Sinatra and including
      > ActiveRecord, and whatever other Rails libraries you want
      > to use.

What seriously? Something doesn't fit inside Rails' neat cutouts so obviously the solution is to get rid of Rails? This smacks of "if you don't like Rails, go somewhere else." Just because you're using a framework doesn't mean you can't do anything outside of it, ever!

I mean, take Rails. It gives you a LOT of convenient stuff that you wouldn't want to throw away wholesale, but if you have a Customer model and a Card model, where do you put the checkout code? It could go in either place.

A great solution is to put it into lib/checkout.rb and write a custom Checkout class that manages how Customers and Carts interact. Not only does this make it super-easy to test outside of ActiveRecord, but it consolidates the code in one common place, and you haven't tossed out Rails for no reason.

<comment><USER2> > if you don't like Rails, go somewhere else

Exactly. Makes perfect sense to me. If you're not going to lean on the framework, why even use the entire stack? You can pick and choose the parts of it you like, and want to use under the context of something like Sinatra which is meant for that sort of thing.

> but if you have a Customer model and a Card model, where do you put the > checkout code? It could go in either place.

You can put it in either place. I'm not sure I understand why just because there's some flexibility on where you can put method X you should just work completely outside of the framework.

Edit: But this is a different point than the original debate though. I think the blog author goes way too far in avoiding using the framework.

<comment><USER4> You can pick and choose the parts of rails as well. I have a rails 3.1 app that doesn't use active_record at all, but it does use active model and encapsulates the service layer in model objects.

</comment></comment></comment></comment><comment><USER5> It seems that people forget what "model" (in MVC) really means. A model (in general) is a _simplification of the world_. The world is so complex that you can't possibly reason about it at once, so you split it up into smaller models that are easier to understand. It's exactly the same in MVC: the model is the _definition_ of your application's world; a simplification that your computer can work with.

Yes, Rails ship with ActiveRecord, which makes it dead simple to define data models that you need to store, but the app/models/ is far from limited to _only_ data storage. app/models is about the _business logic_ of your application. It seems to me that all of the files the article wanted to place under lib/ actually belongs under app/models.

Yes, I agree that it's nice to be able to test your application outside of Rails, but whether you place the files under app/models or lib/ doesn't change that. Yes, it might be useful to abstract it out to a gem, and _then_ you can move it into a lib/-directory (since there is no MVC of a Ruby library), but as long as you're inside the realm of a Rails application you should put everything that's related to the _business logic_ in app/models.

So, in summary:

If it's related to the BUSINESS LOGIC of your app: Put it in app/models.

If it's just GENERAL stuff: Put it in lib/ (or better: create a gem out of it).

<comment><USER6> Say you're writing an app that has simple business logic. Easy, you put that logic into model. Then you realize you need to write much more business logic code. No problem, model. But your model classes are getting fat and it looks ugly. So you refactor the code into modules.

Now somebody is browsing your code and they are ripping their hair off, because there's a method on Order object and there are 6 modules included in the Order object.

Before you realize it, you're violating not only MVC but also OOP.

If you used Services, you could model nice controller/manager classes in OOP manner, with responsibilities nicely distributed. These are your controllers. Rails controllers are just the HTTP interface.

<comment><USER5> I'm not suggesting that you should create modules and include these into your ActiveRecord-models. Feel free to create any kind of class/module/object/whatever, but as long it's related to the business logic put it under app/models.

<comment><USER7> +1 app/models is for domain models

</comment></comment></comment></comment><comment><USER8> I'm not particularly familiar with rails (or even web or client/server in general -- I'm more of an embedded/systems type) but this post gives me a strong feeling of the youth of today rediscovering the lessons of yesterday, to wit, properly abstracting the application from presentation.

<comment><USER9> Indeed. "Services" often sound suspiciously like well-dressed subroutines.

</comment></comment><comment><USER10> I'm not typically a Ruby/Rails programmer, but this seems to me like a different take on good advice I was given a while back: "Whenever possible, build libraries, not apps."

If my code does anything at all which might be decoupled from this particular program or webapp, I try to break it out into a library, which I then call from the app I'm working on. This sometimes results in some extra work when working on that project, but it's saved me plenty of times in the long run.

edit: typo

<comment><USER11> It's one thing pull elements into libraries it another thing to put the whole model layer into the lib directory,

</comment></comment><comment><USER12> I completely agree! As a "framework-less" Java developer where I had written apps with 100k+ LOC, the application code is always separated from the web layer. When I started Rails I scratched my head for days trying to figure where the application code should reside.

I hope this turns into a best practice.

</comment><comment><USER13> i go one step further, and put the domain layer behind a network interface (http or other) with a versioned API. IMO, trapping behavior inside a single application is just as bad as tying it to a framework. the domain layer needs to be available to the entire organization.

rails is just a tool to build a front-end app. i have many of those on my project. some rails, some sinatra. if they want to access shared behavior or data, they include the appropriate gem which knows how to access a particular API.

<comment><USER14> I would only reach for service-oriented architecture after the application is very mature and it is apparent which parts will benefit by moving to their own services.

<comment><USER13> or the moment you have more than one application running. (which for many products is day #1)

<comment><USER15> I'd like to hear more on how you setup your service architecture afex (if time permits). Sounds like exactly what I'm trying to accomplish. Thanks in advance!

</comment></comment></comment></comment><comment><USER16> In Rails, the application belongs in app/, not lib/ or a gem. If this does not meet your admittedly obsessive design concerns then I suggest not using Rails.

<comment><USER3> You do understand that when the author says "your app" what is meant is "the domain or business logic that drives decisions."

Seriously, what's with the attitude? There's more than one way to architect software and Rails doesn't cover every single base. Pulling business logic out of your ActiveRecord models and into separate classes makes it easier to test and easier to manage code that interacts with more than one model.

Lighten up.

</comment></comment><comment><USER17> I think the bottom line with this is you shouldn't be using a framework as the entire architecture of your application. Frameworks usually are designed to do a few things very well, in the case of web frameworks, they provide the mechanisms needed to receive http requests and return responses elegantly along with rendering output from the server-side language into an HTML format. Beyond that, there is a much larger world of application architecture and design that goes beyond any "canned" framework. Web frameworks are great and very much fill a need but they aren't a panacea to make every project trivial.

</comment><comment><USER18> Totally agree on the concept, but not on the implementation. It's time for a #norails revolution. Rails expresses the needs of domain-driven applications very poorly. My response to this article: <http://bit.ly/n9DFi6></comment>


Show HN: Spayce – A Raw and Real-Time View of What's Happening Nearby - <AUTHOR> http://spayce.me

<comment><USER2> It certainly seems like a great way to plug into the "collective memories" of people and get a sense of their experience of a place, which could inform our own. Cool alternative to text reviews...sort of a vine like twist.

</comment><comment><USER3> This is a great idea, even better execution. I think they have messaging and strategy right, brand, design, etc. Seems like they have a chance in a very difficult space, so if they solve the problem, I can see this being hugely valuable and have exponential growth. The college space is really great to introduce this with because it's just such a dense geographic area and people are so connected to each other within the region.

Astroman rocks.

<comment><AUTHOR> Thanks for the kind words! It is indeed a tough market to crack. Doing our best. Thanks again for the support.

</comment></comment><comment><AUTHOR> Would love to hear any feedback from the community..

</comment><comment><USER4> Love this App</comment>


Books I recommend to my software engineering students - <AUTHOR> http://web.eecs.utk.edu/~azh/blog/booksformystudents.html <comment><USER2> I've advocated for a "canon" of books for software engineering and computer science that could be taught in school. The canon would accomplish several goals: it'd inform students about the less technical but profoundly important ideas within CS (Brooks' No Silver Bullet, The 10x cost incurred when moving from stage to stage in waterfall, how to treat and manage failure, etc.), it'd teach students how to think about problems (How To Solve It by Polya, The Design Of Everyday Things), and it'd provide history/culture (Coders At Work, Soul of A New Machine). One should, of course, learn how to program in a CS major. But there's diminishing returns on teaching programming in a class; ultimately students must make the shift from learning in class to teaching themselves. Perhaps teaching more of the "soft" aspects of CS would give better returns, especially as the students move up the ranks and start managing people.

<comment><USER3> I disagree.

I think books on software management and process are mostly lost on someone who hasn't been involved in the process. (Hell many of those books are lost on project managers)

I think the most important subjects to teach students are javascript, css, & sql (or mongo). Before you teach a beginner wood worker the "Zen of wood" you teach them how to cut a piece of wood without sawing off a finger.

The rest will come later.

<comment><USER4> Any books to recommend for the topics you mentioned? Especially sql or mongo.

<comment><USER5> I really liked 'The Art of SQL', Stephane Faroult

</comment></comment></comment></comment><comment><USER6> > [ _The Mythical Man-Month_ ] Key takeaway: You'll soon fall victim to the > problems identified by this book, even after reading it.

Yeah, that pretty much describes all of software development in one stroke. But you should read it anyway (if only to experience the feeling of someone predicting your future before (most of) you were born).

<comment><USER7> I've always like the MMM but there is also an undercurrent of this being a re- telling of xeno's paradox, and its important to remember the rabbit actually does win.

So, adding new bodies does incur cost, but the belief that always adding new bodies always adds more cost than adds impetus to the outcome is not always true.

The classic example is "the hump" which was the cost in fuel terms to ship fuel to China, to be able to fly from China to bomb Japan. Economically ruinous, people forget that missions were nonetheless successfully conducted: it was ruinous but the outcome was achieved within limits.

So the MMM does not say "never add a body" it says "know what your incurring in overhead, choosing to add a body"

<comment><USER8> From what I recall, it talks about 3 somewhat related things.

First is that adding people slows things down in the short term.

The second is that due to overhead, adding people won't give linear speed up in completion time.

And third, adding people won't help a project that is linear in nature. The common example is using 9 women to get a baby in 1 month.

<comment><USER6> That's a good summary.

My years suggest that he wasn't nearly pessimistic enough.

<comment><USER7> You mean if you add 9 women you get a baby in 81 months?

<comment><USER6> I mean that in the limit, adding the n+c'th person isn't just useless, it's devastating. If the optimal (in terms of calendar date of completion) size for a project is (say) three people, it is entirely possible, if not almost certain, that putting 30 people on the project will push the expected completion date to infinity. In other words, the project will fail.

Now, you might say, "I will just bench those other 27 people" and have them play cards or something. In most organizations, though, you _cannot do this_. Instead, they must be _seen_ to be _useful_ somehow. And if you have 30 people with their hands in a three-person project, no matter how peripherally, failure is all but guaranteed.

I have seen this many times.

<comment><USER9> 100% feel you.

Last time I worked at a big company, there was lots of dead wood, but one guy in particular was just a massive liability. Nice guy, tried hard, but utterly incompetent, and (I think there is a term for this?) he was unaware / ignorant of how incompetent he was. He would often check in code that would break the build (team of 300 engineers, large telecom system), he would write and run scripts that would bring computing clusters to their knees (this was late 90s, there are probably ways to mitigate that now), he would consume lots of high- quality talent's time with basic questions, etc.

I literally asked my manager if we could pay Leo to sit home and play video games. OF course, as you said, everyone's gotta look busy.

Here's the punchline: I learned a new term (to me, at the time... not sure I've heard it again) from a greybeard/wizard there -- this guy was a genius. He had a very appropriate term for Leo: negative producer.

Bingo!

</comment></comment></comment></comment></comment></comment></comment><comment><USER10> I would also recommend two more good books. They are relevant to almost every aspect of life and creation process:

\- Antifragile: Things That Gain from Disorder by Nassim Nicholas Taleb[1]

\- Skunk Works: A Personal Memoir of My Years at Lockheed by Ben R. Rich[2]

[1] [https://www.goodreads.com/book/show/13530973-antifragile](https://www.goodreads.com/book/show/13530973-antifragile)

[2] [https://www.goodreads.com/book/show/101438.Skunk_Works](https://www.goodreads.com/book/show/101438.Skunk_Works)

<comment><USER11> Can I dive straight into Antifragile without having read the author's previous books?

<comment><USER12> I started from Antifragile and didn't regret it. If you feel like drawn toward the earlier ideas, read Antifragile -> Skin In the Game -> Fooled By Randomness -> Black Swan. This might also be related that I usually have more interest in starting with practical knowledge and then digging deeper into the theory/more basic knowledge.

</comment></comment></comment><comment><USER13> I'm disappointed not to see the pragmatic programmer there. I would think that it's great reading, not only for every software developer, but also for stakeholders and product managers.

<comment><USER14> Peopleware is another good one.

</comment></comment><comment><USER15> Software Tools by Kernighan and Plauger was the first programming book to blow my mind, and changed the way I approach working in difficult/legacy systems. As a programmer, you don't have to accept the limitations of the system you're working with and by building simple tools can make even the worst system bearable.

And of course The Elements of Programming Style is a classic.. The lessons seem like clichés now, but there was a time when things that seem obvious now were hotly debated.

[https://en.wikipedia.org/wiki/The_Elements_of_Programming_St...](https://en.wikipedia.org/wiki/The_Elements_of_Programming_Style#Lessons)

</comment><comment><USER16> "The Design of Everyday Things", I would consider the digital versions to be

Code and Other Laws of Cyberspace

[http://codev2.cc/](http://codev2.cc/)

and

Free Software Free Society: Selected Essays of Richard M. Stallman

[https://shop.fsf.org/books-docs/free-software-free- society-s...](https://shop.fsf.org/books-docs/free-software-free-society- selected-essays-richard-m-stallman-3rd-edition)

</comment><comment><USER17> I've read only the _The Mythical Man-Month_ from this list. It is amazing how the project management really did not change much. Similar ideas (sometimes same) to similar problems.

I would recommend watching this talk [0] from Kevlin Henney to anyone who likes that book.

[0]: [https://youtu.be/AbgsfeGvg3E](https://youtu.be/AbgsfeGvg3E)

</comment><comment><USER18> A Discipline of Programming, by Edsger Dijkstra.

"For lack of a bibliography, I offer neither explanation nor apology."

</comment><comment><USER19> I take it that the goal is to list books of general interest for the software engineering student. It is not clear what the author means by classic CS material, so it is hard to know what is excluded. Surely, _Mythical Man Month_ and _GoF_ qualify as classics of software engineering. Or does the author mean classics from the entirety of a CS program?

I've been teaching a software engineering class for a few years. Being conscious of the high cost of textbooks, I find that _Applying UML and Patterns_ by Larman achieves the best balance of practice, design, and process.

For students who want to dig into agile methodologies I would recommend _Extreme Programming Explained_ by Beck and the Poppendiecks on Lean.

In my opinion, a philosophical grounding is helpful for the programmer. Accordingly, I'd want to suggest something like Locke's _Essay Concerning Human Understanding_ or a secondary source on Aristotelian categories. Moreover, a broad knowledge of different types of ethical systems will benefit a developer.

Lastly, as a kind of curve ball, I would recommend any STEM student to work through an LSAT preparation book. Technologies will come and go, but, throughout a career, the problems a developer encounters will likely have dimensions that require general critical thinking.

</comment><comment><USER20> I don't think you can teach software engineering. Writing software is more like writing proofs than building airplanes. Take a formally verifiable language like Agda for example. Proofs are indistinguishable from programs in that language. The Curry-Howard correspondence shows this is true for program- proofs in general.

Large software systems fail continuously in countless ways all the time. Google search will still throw 404 or 5xx occasionally. We're not building airplanes that either fly or crash (although even the software running airplanes is now increasingly fault tolerant). We are effectively getting from point A to B through a variety of logical operators, increasing the set of valid A through trial and error. We find an input that throws error, our theorem was faulty and we update our proof. A seemingly valid input can't produce the results we want, we double check to make sure our proof is still valid. Dilettante engineers will begin to introduce unbound complexity at this point for making fixes without understanding root causes.

For the above reasons, we should abandon any attempt at comprehensive software engineering curriculum or canons. We are not architecting a skyscraper to be delegated to a construction company to be built one and done for all of time. We are continually exploring and staking out a specific problem space. Our greatest compass in these new journeys will always be foundational computer science. We must take the specific and make it general. Careful application of elementary data structures, algorithms and discrete analysis will move and has moved us continents further than any convention to "best practices" ever will.

<comment><USER21> Funny, I'd say that writing proofs is a lot like building an airplane, and software engineering might be far away from both.

In writing a proof, you provide an argument that justifies some claim is true. Your argument has to be airtight, so that no matter what kind of counter example or ugly scenario is proposed, your argument still holds.

The same seems true for an airplane. It needs to do its job (fly) and do so on the face of a myriad of different external factors. If you build a plane, it should "never" fail.

Software on the other hand, can be built without thinking about ALL of these edge cases. Obvious it's better if it always works, but it's usually okay if there is some bizarre scenario that causes an error. For some pieces of software it might even be okay if these errors never get fixed (ie: a very small number of users ever experience them). Software can work, even when it (kinda) doesn't.

The Curry-Howard correspondence might be technically true, but we're just as bad at writing specifications for programs as we are at writing the programs themselves. It could be good to acknowledge that and just write the software that solves most, but maybe not all, of our problems.

</comment></comment><comment><USER22> Blech. Nothing here that's obscure or likely to alter the path of some one's knowledge. No philosophy, no history, and the list includes Gladwell. Actually, the inclusion of Gladwell probably distinctly colors my thinking on the subject.

Maybe "How to read a book" or "The Alchemist" or I don't know, something that's not just so totally typical.

<comment><USER23> I personally don't like philosophy books. I think there's more to be learned about philosophy by reading about specific topics, rather than actually reading about philosophy. Some exceptions like Plato, etc.

Absolutely agree with history being useful, though. Any particular periods of history you think are essential to know about, and any particular great books on them?

<comment><USER24> I'm not the OP, but I agree that history's an essential subject.

History has many themes. These appear in every time and place, sometimes in the forefront, sometimes in the background. I believe the most "essential" period is the one that answers your questions.

So, I'd recommend picking any period that you have a vague curiosity for. I'd go a half step further and recommend avoiding recent periods (late 20th Century).

IMO, it's too recent for there to be consensus on what constitutes good scholarship. There are obviously exceptions to this, but if you're new to history reading, it's difficult to disambiguate the good from the bad.

Examples of periods and geographies that are particularly well-studied, with good accessible literature:

\- Late antiquity in the mediterranean (fall of the roman empire)

\- Inter-war period in continental europe (Weimar, etc)

\- Revolutionary period in France, United States

\- Antebellum period in the United States

\- Early Russian Revolution (there aren't many good syntheses imo, because this period was incredibly complicated)

\- Europe during the reign of Louis XIV (1643-1715)

\- Napoleonic wars and aftermath

This list is pretty euro-centric. IMO, these are the safest place to start, as the plurality of english-language scholarship is in these places and periods. After developing a good nose here, you'll feel comfortable reading in areas where the scholarship isn't as deep.

</comment></comment></comment><comment><USER25> Creativity Inc is a wonderful book, but Pixar's culture and work environment are exceedingly rare if not entirely unique. For a more realistic, yet still useful example of corporate culture, I would recommend "The Phoenix Project" to students.

<comment><USER26> I absolutely love The Phoenix Project, but it is 100% fiction and not based on any real-world evidence. It's great for laughs but I wouldn't use it to "teach" anyone.

<comment><USER23> My opinion: There's more we can learn from fiction than we think.

By analogy, creativity, perhaps even reality distortion. I don't think we should solely rely on non-fiction and proven methods to teach or grow. Indeed, if we did that, we wouldn't grow at all.

</comment></comment></comment><comment><USER27> Scroll to the bottom of this page and watch the lectures on conceptual design [https://stellar.mit.edu/S/course/6/fa18/6.170/materials.html](https://stellar.mit.edu/S/course/6/fa18/6.170/materials.html) interesting analysis on why some software ideas fail on launch. He had a book as well but the lectures are better so posted them.

</comment><comment><USER28> I had a professor make us read Death March by Yourdon and Peopleware. Both were great, especially Death March. It helped me notice bad patterns at an employer and gave me the motivation to quit. Its also very well written and funny, but a bit too real at times. You'd think all of it is fake or exaggerated, but then you're living it and it's so depressing.

</comment><comment><USER29> I bought my boss two copies of _The Mythical Man-Month_ so he could read it twice as fast.

<comment><USER30> And he hired 4 people to read to him for each book. Now he could learn 8 times as fast.

<comment><USER31> Not if he outsourced the reading of said book to a foreign company. He would would have to check if their reading was the same as his every hour for 18 months that would have taken him 3 days to read it himself. Slowly.

</comment></comment></comment><comment><USER32> I would swap out Outliers for Dark Pools, which better highlights how a revolution driven by Software Engineering in its purest form looks like, and is still ongoing to this day.

Outliers is kind of entrepreneurship junk food and 10,000 hours has already been debunked.

<comment><USER33> I'm not sure it's debunked and I don't understand the rabid hatred of it (like bringing it up where it isn't mentioned at all).

<comment><USER34> Outliers is debunked. The authors of the primary sources that he quoted disavowed it as an inaccurate representation, and when you look outside of kind learning environments like sports its becomes even less correct.

</comment></comment></comment><comment><USER35> It's interesting to see "The Mythical Man-Month" in a list like this, since I've never once seen any team implement what it suggests. Is it meant only as a cautionary tale?

<comment><USER36> What I took away from MMM was less concrete, actionable suggestions you can implement, and more "things to watch out for" in the development process. Things like the second system effect, and Brooks' Law (adding people to a late project makes it later) are red flags. You can't always avoid them (management will add warm bodies in desperation, you probably can't convince them otherwise). But at least when you see second system effect happening you can often try to mitigate it by design/architecture decisions.

I would also say the lessons around keeping teams small have been taken to heart in industry somewhat, e.g. in the notion of the two-pizza team at Amazon.

That said, some of Brooks' suggestions, such as the "surgical team" concept, are more products of their era and haven't aged well.

</comment></comment><comment><USER37> " Note: This book does seem to overgeneralize and oversimplify complex situations, but there are still good points." describes everything by Malcolm Gladwell.

</comment><comment><USER38> If they want to get started with ML, I recommend Hands On Machine Learning with Scikit Learn and Tensorflow.

<comment><USER39> Hey I have no background in ML. Would you still suggest it ? I do systems.

<comment><USER38> Yes

</comment></comment></comment><comment><USER40> My list for all engineering students:

Design of Everyday Things Founders at Work Antifragile

</comment><comment><USER41> Clean Coder is great for developers just starting their career.

</comment><comment><USER42> I'm surprised to see nothing on negotiation there.

</comment><comment><USER43> fwiw, i found John Ousterhout's "A Philosophy of Software Design" pretty good actually (despite having misgivings about it earlier)

</comment><comment><USER44> These books will tell them nothing.

I legit think that there is only one way of engineering complicated software and that is Entity-Component-System ([https://en.wikipedia.org/wiki/Entity_component_system](https://en.wikipedia.org/wiki/Entity_component_system)).

<comment><USER45> ECS systems are great, but claiming they are the only option available is more than a tad unreasonable.</comment></comment>


Show HN: I Wrote a RPM Packaging Guide - <AUTHOR> https://github.com/maxamillion/rpm-guide <comment><USER2> Haven't looked at it in detail, but have you mentioned docker builds? I see you wrote about mock, which is immensely useful for building RPM packages. But docker seems well-suited for this task, for example: [https://github.com/alanfranz/docker-rpm- builder](https://github.com/alanfranz/docker-rpm-builder)

What's your take on this?

<comment><AUTHOR> I hadn't gone down that far into the rabbit hole yet but I'm absolutely open to adding new content around using docker for rpm building. Thanks for the feedback!

<comment><USER2> bookmarked, thanks for the guide!

<comment><AUTHOR> Certainly! I hope it's helpful, feel free to share ;)</comment></comment></comment></comment>


The Friendship That Made Google Huge - <AUTHOR> https://www.newyorker.com/magazine/2018/12/10/the-friendship-that-made-google-huge <comment><USER2> What a lovely profile. I worked at Google early enough that I got to know Jeff and Sanjay, although never worked directly with them (to my regret). Every aspect of this article rings true to me, the way they are, the way they work together. I always admired how incredibly collaborative they were. They never competed with each other. Or really with anyone else, either, they just quietly did excellent work and helped others do their work too.

</comment><comment><USER3> I'm so glad this this article (unlike so many other representations of Jeff and Sanjay's work) does not overlook Sanjay's contributions.

I don't know what it is; maybe Sanjay is a weird (read: non-English) name, maybe it's that computing culture's obsession with lone hackers leaves no room for a partnership like Jeff and Sanjay's. Anyway, kudos to the New Yorker for not falling into that trap.

<comment><USER4> I think a big part of it is something the article touches on: Jeff is an extrovert, Sanjay is an introvert. Jeff is the much more visible of the two; the one who people are more likely to have heard speak personally or to have dealt with. It sounds like this is how they both prefer it, but it definitely means that Sanjay's contributions get passed over in the common discourse. I was also glad to see the article avoid that trap.

<comment><USER5> You mean like Steve and Woz? j/k :)

</comment></comment></comment><comment><USER6> I was so delighted by this piece and then I got to the byline - James Somers strikes again!

You might know his writing from "You're Probably Using the Wrong Dictionary" or his amazing article about New York's subway system told through the lens of the countdown clock: [http://www.theatlantic.com/technology/archive/2015/11/why- do...](http://www.theatlantic.com/technology/archive/2015/11/why-dont-we-know- where-all-the-trains-are/415152/)

Highly recommend all his work: [http://jsomers.net/](http://jsomers.net/)

<comment><USER7> seems to be a awesome engineer too.

</comment></comment><comment><USER8> > Most of the fragile insights that laid the foundation of a new vision > emerged not when the whole group was together, and not when members worked > alone, but when they collaborated and responded to one another in pairs

The article gives several examples of this dynamic, but one that came to mind for me was how Monty Python members created sketches for _Flying Circus_ :

> Writing started at 9 am and finished at 5 pm. Typically, Cleese and Chapman > worked as one pair isolated from the others, as did Jones and Palin, while > Idle wrote alone. After a few days, they would join together with Gilliam, > critique their scripts, and exchange ideas. Their approach to writing was > democratic. If the majority found an idea humorous, it was included in the > show.

[https://en.m.wikipedia.org/wiki/Monty_Python](https://en.m.wikipedia.org/wiki/Monty_Python)

</comment><comment><USER9> Interesting tidbits:

 _To solve problems at scale, paradoxically, you have to know the smallest details_ -Alan Eustace

 _Today, Google's engineers exist in a Great Chain of Being that begins at Level 1. At the bottom are the I.T. support staff. Level 2s are fresh out of college; Level 3s often have master's degrees. Getting to Level 4 takes several years, or a Ph.D. Most progression stops at Level 5. Level 6 engineers—the top ten per cent—are so capable that they could be said to be the reason a project succeeds; Level 7s are Level 6s with a long track record. Principal Engineers, the Level 8s, are associated with a major product or piece of infrastructure. Distinguished Engineers, the Level 9s, are spoken of with reverence. To become a Google Fellow, a Level 10, is to win an honor that will follow you for life. Google Fellows are usually the world's leading experts in their fields. Jeff and Sanjay are Google Senior Fellows—the company's first and only Level 11s._

<comment><USER10> > Level 2s are fresh out of college; Level 3s often have master's degrees. > Getting to Level 4 takes several years, or a Ph.D.

I..don't think this is still accurate. New grad SWEs are L3's.

<comment><USER11> Yeah it's mostly accurate but drop level 1. Lowest I've seen for technical employee of any kind is 2. New grad is 3. 4 is about 2 years later. 5 is the point where you have "no expectation if advancement" and is the most common level for mid-career people.

</comment></comment></comment><comment><USER12> If you want to see an example of Sanjay's elegant code, take a look at his Go stream package: [https://github.com/ghemawat/stream](https://github.com/ghemawat/stream)

<comment><USER13> Sanjay's "experimental" directory in Google's repo -- a place where users can commit unreviewed code that are just ideas, or park some code that isn't ripe yet -- is full of gems.

</comment></comment><comment><USER14> Good stuff. Every sport has great partnerships (Jordon - Pippen, Brady - Gronk) and I always wondered if such a thing would even be possible in SE as I've felt overly self conscious and inefficient while pair programming.

Though what really makes this my favorite long read of the year is how it follows their story over the course of Google's rise and the subtle juxtapositions between early Google of "doors across sawhorses" and today with "Level 0-11". Given all the recent news with engineer walkouts and protests it makes you realize that Google have probably done the best job of handling the scaling challenges of being a unicorn and making the necessary cultural adjustments as they grew in size.

<comment><USER15> Don't forget Sam and Frodo

<comment><USER16> Yeah, teams are good but every once in a while you need that lone crazy Tom Bombadil genius to come along and save your butt from disaster.

<comment><USER17> And sometimes you need eagles

<comment><USER16> The really bad times are when your entire universe screaches to a halt until Brandon Sanderson helps it limp across the finish line.

</comment></comment></comment></comment></comment><comment><USER18> After reading this article, I'm surprised that Google doesn't encourage pair programming / developing developer relationships at the level that Jeff and Sanjay have here. One might even argue that the manner in which interviewing and hiring is currently done in the tech industry in general might actively work against building the sorts of relationships where the output is greater than the sum of the inputs.

Edit: added a missing "the".

<comment><USER19> In a way, Google does encourage it by requiring all code to be reviewed before check in. If you're pair programming, you already have a reviewer and can check in right away. One-person projects can be more frustrating due to the delay.

It doesn't seem to be enough to encourage widespread pair programming, though.

<comment><USER20> Every company I've worked at as a software engineer, even my first where we emailed patches to each other like cavepeople (and no we were not using git) has had code reviews before check in. It's not at all the same thing as pair programming though.

</comment></comment></comment><comment><USER21> Holy moly, a mention of Bogdan! He was a legend of his own. Some of the most consequential changes, like elevating your service to higher QoS levels, which something like e.g. Gmail would require, were gated on his approval and his only. Someone made him a SW Easter egg, but the story already on the Internet gets some basic facts wrong.

<comment><USER22> Do tell! There have got to be a few good ones. He interviewed me for his current company.

<comment><USER21> The internal Easter egg was basically a random fortune, which cycled through his greatest "hits" (#1: "No"). You really had to know where to look.

</comment></comment></comment><comment><USER23> Does anyone have a reference for "In 1966, researchers at the System Development Corporation discovered that the best programmers were more than ten times as effective as the worst." ?

I'm seeing the year 1968 at [http://thecomputerboys.com/?tag=black- art#fn-156-1](http://thecomputerboys.com/?tag=black-art#fn-156-1)

Is there a 1966 reference out there somewhere? Or perhaps the actual study was done in '66 and didn't make it into press until '68? I didn't pull the original paper.

(The fact checkers at the New Yorker are typically very sharp.)

edit: I pulled the original paper (10.1145/362851.362858). The studies were done in 1966!

Interesting to see the genesis of the myth of the 10x programmer.

<comment><USER24> "Interesting to see the genesis of the myth of the 10x programmer."

How could it be a myth? Ever see a professional sports player compete against a non professional player? Why wouldnt this dynamic exist in programming?

<comment><USER25> Maybe I'm unimaginative, but in sports where performance is objectively comparable, I can't think of any where professionals 10x amateurs.

Amateur marathons are about 4 hours, professional are about 2 hours. Amateur deadlifts are about 500lb, professional are about 1000lb. Amateur long jump might be 6m, professional almost 9m.

<comment><USER26> Cristiano Ronaldo and Messi are extreme outliers in soccer/football, even compared to other greats. They're probably 10x. Trouble is amateurs don't play in the big leagues, so we don't have data on how a normal person would peform. There is an amusing Ronaldo vs Ronald video. I think it's called "Tested to the Limit."

Occasionally a 1st league team plays a 3rd league. I saw a video of Napoli vs Nowhere go 26-0. They scored about as fast as the game could be reset to kick- off. That's more than 10x.

<comment><USER27> You are confusing game scoring with some concept of "x times ability". In many games, someone who is only 10% better than opponents can win close to 100% of the games.

<comment><USER26> What's your productivity metric? Perhaps having only 10% fewer bugs per hour makes someone 10x faster at writing complete programs.

</comment></comment></comment></comment></comment></comment><comment><USER28> This is a wonderfully told story of two very senior engineers who developed a lifelong friendship and wrote some of the core of the Google search engine I use countless times per day - I really enjoyed this piece.

</comment><comment><USER29> Wow, this article is pretty amazing to me. I have a joke (err, kind of a joke) that I like to say, "It's never a cosmic ray." What I mean by that is that when you are debugging and getting incredibly frustrated, right when you feel that you want to give up because you can't find the bug, that you shouldn't blame "cosmic rays" because 99.999% of the time it's a logic problem.

Here was a case where _it actually was cosmic rays_ (at least, not a logic problem), and these guys found it. I also agree with the title and intro of the article - I think it's very possible that if these 2 exact people hadn't been at Google at the time than Google's path could have been very different. Makes me also appreciate how much luck is involved in any successful indeavor. Not luck that they found the bug (that was hard work), but luck that these 2 guys found themselves working at Google right when Google needed them.

<comment><USER30> It's never a cosmic ray but it's often bad ram. You don't have to inspect the binary to diagnose bad ram. The system would be behaving erratically in other ways due to regular memory corruption. It's an amazing story but by no means the only way it could have played out.

</comment></comment><comment><USER31> "In terms of sexiness, compilers are pretty much as boring as it gets,"

That bit surprised me. For me it doesn't get much sexier than compilers.

<comment><USER16> Yeah. Maybe that's the outside view, or view from the surface level folks concerned with the next shiny framework, but I think the best engineers are always at least a little enamored with deeper nuts and bolts.

<comment><USER32> I think he meant the business side isn't going to give a damn about a compiler.

<comment><USER16> That makes sense. At least, outside of massive companies like Google, who would see the value in a compiler that optimized code for their use even a fraction of a percent better. Then compilers become the "shiny object" for them.

</comment></comment></comment></comment><comment><USER33> "I don't know why more people don't do it," Sanjay said, of programming with a partner.

</comment><comment>=

Maybe because most employers won't pay for that. Two people? For one level of output? You'd have to prove yourself a level 11 googler before most places would give you the luxury of working as a long-term pair.

<comment><USER20> Pivotal Labs has had across-the-board pair programming for years[1]. Although the folks there are undoubtedly talented, I'm fairly certain most aren't at Jeff and Sanjay's level.

I don't know why companies don't do it, but it's not cost alone. Here are some of my hypotheses:

1\. It seems like it would be more expensive - which might deter companies from thinking about it

2\. If you're not already doing it, it's hard to convince all or even a large fraction of your engineers to do it. It's a completely alien way of working to most people

3\. It's not obvious to employees who haven't done it that it's clearly better. And it might not be - I honestly don't know

4\. For 100% pairing, you need 2 programmers' schedules to line up exactly, 5 days/week - so goodbye flextime, work from home, remote working and all the other arrangements that people structure their lives around

5\. It must be exhausting to sit with, and talk to, another person constantly your entire working day

1\. [https://pivotal.io/careers](https://pivotal.io/careers)

<comment><USER34> There've been studies on the "more expensive" part, and IIRC it's about a 15% premium in terms of development time - there've been a few studies. Here's one I could turn up quickly: [https://collaboration.csc.ncsu.edu/laurie/Papers/XPSardinia....](https://collaboration.csc.ncsu.edu/laurie/Papers/XPSardinia.PDF)

Simultaneously, you get a bunch of benefits. (Most importantly, there's no bus factor of 1 for anything)

<comment><USER33> i'd think another benefit might be 'better code' overall. has there been any study about the quality of code from these setups? number of bugs or security issues, for example?

<comment><USER34> Also covered in that paper

</comment></comment></comment></comment></comment><comment><USER35> > The world's most robust computer systems... used special hardware that could > tolerate single bit-flips. But Google... bought cheaper computers that > lacked that feature.

> Together, Jeff and Sanjay wrote code to compensate for the offending > machines.

I'm guessing the article is referring to Google buying servers with non-ECC RAM.

How do you write code to compensate for non-ECC RAM?

<comment><USER36> > When a car goes around a turn, more ground must be covered by the outside > wheels; likewise, the outer edge of a spinning hard disk moves faster than > the inner one. Google had moved the most frequently accessed data to the > outside, so that bits could flow faster under the read-head, but had left > the inner half empty; Jeff and Sanjay used the space to store preprocessed > data for common search queries

So Google was a company that was too cheap to buy ECC RAM, but willing to let half of their hard disks go to waste. I'm struggling to understand the logic behind this; can anyone explain?

<comment><USER26> RAM is expensive. Disk is cheap.

</comment></comment></comment><comment><USER37> these are the kind of articles i like reading, well written, interesting subject and good content length. Too many times i read bold headlines on HN, "Why X is bringing america down", then go to read it and just when they get to the crux of the matter i find the article has ended smh.

<comment><USER38> The New Yorker does this very consistently every week at an extraordinary level of quality.

They will even mail you a copy in exchange for a subscription. It's magical.

<comment> forgot-my-pw Only $6 for 12 weeks for print + digital. Subscribed!

</comment></comment></comment><comment><USER39> Site's down for me. Archive link: [http://archive.is/w4oWL](http://archive.is/w4oWL)

</comment><comment><USER40> > On Sanjay's monitor, a thick column of 1s and 0s appeared, each row > representing an indexed word. Sanjay pointed: a digit that should have been > a 0 was a 1.

I was curious how he can see "a digit that should have been a 0 was a 1" in a "thick column of 1s and 0s"? Can you give a specific example. I'm not questioning that he did but I'd like to know a specific example.

<comment><USER13> For example the high bit in blocks of ASCII text will always be zero. In a binary dump it would appear as a column of zeros.

<comment><USER40> Ok. So you don't need to actually "read" and understand the binary code like you are reading a programming language. You look for out of place numbers.

</comment></comment></comment><comment><USER19> I wonder if more couldn't be done to encourage pair programming and collaboration in general? What are some other great partnerships?

<comment><USER41> Would highly recommend checking out `The Undoing Project` by Michael Lewis. It's a biography (of sorts) about Amos Tversky and Daniel Kahneman that has excellent stories about their fruitful collaboration in behavioral economics.

</comment></comment><comment><USER42> Was surprised to hear the audio version of the story has a different title - "Binary stars". I think the editorial board went with a more click friendly title.

<comment><USER43> That was also the title of the print version. The New Yorker tends to re-title online "reprints" with less vague/terse/artsy titles.

</comment></comment><comment><USER44> Anyone know where folks like Peter Norvig and Urs Hölzle fit on the L1-L11 scale? I assumed that all of the early employees/now legends were at the top level, so I was surprised to see the article state otherwise.

<comment><USER13> Urs doesn't need a level on the individual contributor scale; he's a Senior Vice President.

<comment><USER45> Urs however does have one, he's listed as a Google fellow (L10) if you go looking.

As a general comment, to compare, a director is L8 equivalent, and VP L9, (but on the management ladder) although at VP+ it gets tricky because there are subdivisions.

Also worth noting that the management ladder goes up more. But the total number of people who are >L11 is probably <11, so it's not like there's any data on how exactly that's broken up.

<comment><USER46> Director is L8, Sr Director is L9, VP is L10, SVP is L11.

<comment><USER47> Not exactly; there are 3 levels of VPs and 3 for SVPs (at least in 2015). It's just not publicized.

</comment></comment></comment></comment></comment><comment><USER48> It is refreshing to see an article celebrating collaboration.

</comment><comment><USER49> This article will queue up a slew of startups that pair program.. perhaps a good thing!

</comment><comment><USER50> There was an earlier thread in HN on them here - [https://news.ycombinator.com/item?id=16744353](https://news.ycombinator.com/item?id=16744353)

</comment><comment><USER51> It is such a beautiful story of friendship. Even married couple don't this level of match in mind. Only a few a luckily like that. Their partnership has resulted monumental contributions.

</comment><comment><USER52> Great article! Two powerful minds working on analyzing and solving on same thread of thought. Kudos to the writer, very well presented.

</comment><comment><USER53> Serious question - is this an ad? For the company, showing they have a solution to new growth problems which will keep them a growth company? (custom chip, architected for AI)

And how they solved the problem before with this team?

</comment><comment><USER54> Does pair programming only work with friends?

<comment><USER55> Nah. You can do it with anybody. I've been part of long-lived teams that pair. But there we rotated pairing partners once or twice a day, so that over the course of the week you'd pair with everybody.

I think that's much more sustainable for most. Pairing with the same person every day is sort of like sharing a snowbound cabin with them. There are very few people I get along with so well that they wouldn't grate after a few weeks.

</comment></comment><comment><USER56> Awesome article

</comment><comment><USER57> "Jeff Dean once shifted a bit so hard it ended up on another computer."

<comment><USER58> "Jeff Dean puts his pants on one leg at a time, but if he had more than two legs, you'd see that his approach is actually O(log n)."

</comment></comment><comment><USER59> Great article. Glad to see Sanjay get the credit he deserves.</comment>


Escape from the Heap: Low-Level Programming in Common Lisp [video] - <AUTHOR> https://www.youtube.com/watch?v=S7nEZ3TuFpA <comment><USER2> This is a presentation by Ahmon Dancy, a software engineer with Franz. The talk was originally prepared for the 8th European Lisp Symposium, April 20-21, 2015; this recording was a presentation to the Bay Area Lisp and Scheme Meetup.

He discusses the implementation of a variety of data structures outside the Common Lisp heap in the storage engine for AllegroGraph, a RDF triple-store entirely implemented in Common Lisp.

</comment><comment><USER3> I had never heard of Memory-mapped Files before but I caught on right away that they're the inverse of a RAMDISK. With a RAMDISK you perform file access which actually goes to memory. With Memory-mapped Files you perform (virtual) memory access which actually goes to a file.

</comment><comment><USER4> ObQwe1234:

> you mean you have to rewrite implementations for basic data structures just > to get in lisp what c++ gives you for free?

> oh hell no. do not want.

</comment><comment><USER5> What's the too long, didn't watch?

<comment><USER6> TL;DW: Hide (static, big, public) Lisp-objects from garbage collector.</comment></comment>


Show HN: Exp, a tool/lib to configure, run, and optimize experiments in python - <AUTHOR> https://github.com/davidenunes/exp <comment><AUTHOR> I created this tool to design parameter spaces, run functions in parallel and optimize hyper-parameters for machine learning models, it's pretty generic and a work in progress, but it gets the job done. There are possibly corner cases I didn't consider, but I'll add/fix things as I need/find them.

<comment><USER2> Is this like a simple version of Google Vizier-style blackbox optimization?

<comment><AUTHOR> Yes, I just wanted a way to separate the model logic from the experiment design. So you design a parameter space in a configuration file, and either use the optimizer to find the best parameters within that space, or run every possible combination of parameters (grid seach)

There are plenty of ways to optimize models from evolutionary algorithms to bandit models, in my simple case I'm working with limited resources, so I included a wrapper to a global bayesian optimizer for sequential search. Grid search is out of the question, but the tool is still handy if I want to freeze all the parameters and vary a couple of them to study some aspect.

bonus: the parameter space file is a nice way to document my experiments --I can track exactly how each experiment was configured.</comment></comment></comment>