Why designers don't like A/B testing - <AUTHOR> http://www.ghostinthepixel.com/?p=549

<comment><USER2> As a designer who has written a yet-to-be-published book on A/B testing... (And as posted on the site, but awaiting moderation...)

 _> And, quite frankly, the first victims of A/B testing are beauty, elegance, charm, and grace._

Right, and the first casualty of a designer's-opinion-only approach is "beautiful" designs that users hate. (Of course, neither premise is necessarily true, this is just a baseless appeal to emotion. Who could be against "beauty, elegance, charm, and grace"?) Can the author demonstrate from, say, ABtests.com or Whichtestwon.com that this is the case? Or did he simply have a bad experiences at highly data-driven companies? Is Netflix, for example, guilty of sacrificing "beauty, elegance, charm, and grace"?

 _> Instead we get a unsightly pastiche of uneven incrementalism lacking any kind of holistic cohesiveness or suggestive of a bold, vivid, nuanced vision that inspires users._

Such as? Is the author suggesting we go with poor-performing designs because in one or a few people's opinion, they "inspire" users? How do you know users are inspired if their performance suffers?

 _> It is the implicit charter of a high-quality design team (armed with user researchers and content strategists!) to propose something a user may not be able to imagine, that is significantly better, since they are so conditioned by mediocre design in the mainstream. _

IMO, it's implicit that a high quality design team could come up with 2 or more such designs, or design variations, to test. Not only that, that they would come up with such design variations to test that _are_ cohesive, elegant, and beautiful.

 _> A/B testing may only be as effective as the designs being tested, which may or may not be high quality solutions. Users are not always the best judge of high quality design. That's why you hire expert designers of seasoned skills, experience, judgment, and yes the conviction to make a call as to what's better overall._

Make a call based on what? Gut feel? The quality of designs being tested is not a problem of A/B testing -- if you only want users to choose from "high quality design", only test "high quality design". I don't understand the objection here. If a designer is truly an expert with seasoned skills, they should be the first to push for A/B testing of their best ideas, not retreating into vague notions of what they feel is best. There is no "One True Solution". Designers should own and drive the A/B testing process, not feel like they are victims of it.

 _> As is true with any usability test, you gotta question the motives behind the participants' answers/reactions. Instead, biz/tech folks look at A/B test results as "the truth" rather than a data point to be debated. Healthy skepticism is alwayswarranted in any testing. Uncovering the rationale for a metric is vital._

People buy or they don't. They click or they go. You can't "question the motives" or hundreds or thousands of users, that doesn't make sense. You can't wish away the data.

 _> A/B testing is typically used for tightly focused comparisons of granular elements of an interface, resulting in poor pastiches with results drawn from different tests._

Poor A/B testing practice may be a problem, fine, but let's not throw out the baby with the bathwater.

 _> How do you A/B test novel interaction models, conceptual paradigms, visual styles (by the way, visuals & interactions have a two-way rapport, they inform each other, can't separate them–see Mike Kruzeniski's talks) which may vary wildly from before? _

Uh, you run an A/B test and measure what you're doing.

 _> Would you A/B test the Wii or Dyson or Prius or iPhone? Against what???_

Category error. We're talking about graphic/interaction/web design. All those companies absolutely should A/B test their web sites, for example.

 _> A/B testing locks you into just two comparative options, an exclusively binary (and thus limited) way of thinking. What about C or D or Z or some other alternatives? What if there are elements of A & B that could blend together to form another option? Avenues for generative design options are shut down by looking at only A and only B._

Seriously? Even basic free tools like Google Website Optimizer let you A/B/n or multivariate test.

 _> Finally A/B testing can undermine a strong, unified, cohesive design vision by just "picking what the user says". A designer (and team) should have an opinion at the table and be willing to defend it, not simply cave into a simplistic math test for interfaces._

Sure, and not A/B testing can undermine a business by costing them thousands of dollars by not "picking what the user says". Pitting design teams against a "simplistic math test" is a silly way to polarize the issue. The designer or design team should be thrilled they have such a wealth of data at their disposal to come up with the best design possible. It's not about fighting, defending, or caving, it's about using the data to know -- and not blindly guess about -- what's going on.

 _> A/B test results perpetuate a falsely comforting myth that designs can be graded like a math test, in which there's a single right answer._

The author asserts this several times throughout the piece but never provide any evidence. Will they shed some light on real case studies where this happens?

Indeed, if there is any "falsely comforting myth" doing the rounds, it's that designers can guess and know what thousands of disparate users will prefer within a single percentage point of accuracy. If you can guess that well, I'd like you to pick my stocks.

 _> And you risk dissuading top quality design talent from joining the team's cause for good, beautiful, useful designs that improve the human condition._

Who's "human condition" are you improving by potentially harming client's businesses by blindly rolling out designs which may in fact cost them thousands (or hundreds of thousands, or millions) of dollars, and therefore jobs, and so on...? This is another silly, polarizing argument.

tl;dr: If you want "good, beautiful, useful designs", then A/B test "good, beautiful, useful designs". It's not us vs them. We can have our cake and eat it too!

<comment><USER3> A couple of companies ago, I worked with a designer who believed "users like a challenge" (his words) and every interface he designed was like a little puzzle. For example you couldn't tell what was clickable or where without studying the page for subtle clues.

Except we weren't making games... We were making corporate intranets.

<comment><USER4> Isn't "Discoverable" another way of saying this? It can and does work. It's fun to find a feature or learn how to do something new.

Examples: iPhone. No manual (probably there is now). It's unbelievable, really. But, you have to admit it works!

Anti-example: fire extinguisher. No tricky interface, please.

One reason I didn't see mentioned is user level. Beginners want/need simple. Later, they want more power. How do you keep the advanced features out of the way of the new users? Hide them! Advanced users will find them when they are ready.

<comment><USER3> You have fundamentally misunderstood the motivations of the average corporate user. The system is a tool to them, nothing more, they use it when they must, then get on with their real jobs.

</comment></comment></comment></comment><comment><USER5> Why don't (some) developers like, e.g., the Lean Startup? Because you can execute your job perfectly, and drive the business right off an effing cliff. It strikes at some of our core narratives, like "successful startups are made on the backs of technical competence, largely by upstart engineers who are a) ridiculously competent and b) scorned by people in positions of power."

You can't tell someone who really _feels_ that narrative that, by the way, technical competence is not sufficient for success. (To say nothing of saying it isn't necessary.)

So why don't designers like A/B testing? Well, what's their narrative? User- loving highly-skilled artist scorned by people in positions of power. (Wow, we have a trend here.) They don't want to be told that a) users provably _don't care_ that this design looks better or is more intuitive or b) that beautiful/responsive/intuitive/etc design is not always worth actual money.

<comment><USER6> You're right. Nobody wants to be told that they're wrong about their most preciously held beliefs about the value of their skills.

But for everyone who's championing split-testing, they should remember a little bit about where it first became popular: Direct Mail. Direct Mail copywriters/designers were the first people to truly use split-testing as a major part of their professional arsenal.

Split-testing gave us long-form sales letters with typewriter fonts, blue backgrounds, yellow highlighter effects, and pictures of Ferraris and McMansions and screenshots of commission checks.

Why? Because they all work better.

<comment><USER7> That's interesting though, because you seem to be implying (forgive me if I'm wrong) that we shouldn't have typewriter fonts, yellow highlights, etc. Or at least that it somehow isn't worth the trade off. But as you say, they do work.

Don't get me wrong, I find those design elements you're talking about to be far from my tastes, but provably I'm not "normal" within the markets under test. At what point what you say to a business "you might make more money like this, but don't do it anyway because it's awful"?

Apologies if I'm reading more in to your statement than was intended. I've seen enough of your work to know that you get both design and business hence I think that statement interesting!

<comment><USER6> I'm not saying the yellow-highlighter crowd is wrong, not at all. But would YOU buy from them?

Your question exposes a really important fact about split-testing:

People are complex. Different stuff works for different people.

The yellow highlighter thing tends to make people like us wanna hurl… it triggers immediate distrust. Meanwhile, information marketers still find that it works.

You can't know who a person is when you split-test against them.

More importantly, you need a LOT of traffic, and even minor temporary changes in traffic composition can affect the quality of your data.

You will see that traffic from HN responds to & converts very differently than traffic from Lifehacker, and again different from Small Biz Trends. Initially you may find certain changes increase your conversion rates… and two months later you find you have lost almost all those customers, or that a disproportionate amount of them have asked for refunds, or been shitty to you in the support channels. Net result: a loss, which you only found out about months after you were so confident that your testing showed a gain. Would the other half of the split test do better? You can never know, because you cannot replicate the exact scenario ever again.

Traffic is not created equal; traffic is made up of different individuals. You can only ever get a feeling of what works for some of them some of the time, depending on several factors.

Take the yellow-highlighter example again: Why does it work for infomarketers when it makes people like us flee in the other direction? They sell to different people. Why would the Hacker News crowd be worse customers than the Lifehacker crowd? Because they are different types of people.

Unless you run a months-long test covering all parts of your business (refunds, avg turnover rate, length of customer stay, # of support inquiries/nastiness in support inquiries, resource usage, server bills, etc), tho, split-testing can't tell you which types of people are better for you.

And if you DO do that, when exactly will you have time left to expand and market your product?

I'm not saying split-testing is wrong or useless, but people use it to try to create scientific certainty when it's absolutely NOT possible to do so.

<comment><USER7> Thanks for the clarification - I think to some extent we're arguing on the same side. Points about traffic I would certainly agree with, and to some extent it's as much about attraction as about increasing conversion metrics once obtained.

I'm not sure I'd agree with the last point about time - I think that balancing business generation, expansion and refinement is possible - but perhaps not so much amongst startup size businesses, which is more HN territory. In that case it should certainly be looked at as "is it better to try and get 0.1% more people to click buy or 20% more people to arrive" - or whatever you think your likely ratios are, hence cost/benefit call.

</comment></comment></comment></comment></comment><comment><USER8> Reading the article, and realizing that these are probably the _best_ arguments against A/B tests that designers can offer, has actually made me change my opinion in favor of A/B testing. I'll try to go for a point-by-point reply:

        > A/B testing may only be as effective as the designs being tested

True, but taken on face value, these words mean that A/B testing becomes more valuable when you have great designs made by great designers. Presumably not what the author wanted to say.

        > As is true with any usability test, you gotta question
          the motives behind the participants' answers/reactions.

True, but how does the designer know in advance that "questioning the motives" and "applying healthy skepticism" will yield conclusions favorable to the designer?

        > A/B testing is typically used for tightly focused
          comparisons of granular elements of an interface,
          resulting in poor pastiches with results drawn from
          different tests.

This point is correct, as far as I can see. It's actually the strongest argument against A/B tests that I know.

        > Would you A/B test the Wii or Dyson or Prius or iPhone? Against what?

This point doesn't have any bearing on whether you should A/B test websites.

        > A/B testing locks you into just two comparative
         options, an exclusively binary (and thus limited)
         way of thinking.

This point is wrong. A couple months ago we ran an A/B test with four alternatives.

        > A designer (and team) should have an opinion at the
          table and be willing to defend it, not simply cave
          into a simplistic math test for interfaces.

This point only asserts what the author is trying to prove.

<comment><USER6> "Reading the article, and realizing that these are probably the best arguments against A/B tests that designers can offer..."

What makes you say that? Did you split-test them? (Serious question.)

<comment><USER8> Serious answer: because I haven't seen any designer offer better arguments against A/B testing, though they often write about disliking it.

If someone holds an opinion but doesn't articulate the reasons for holding it, I'm willing to give them the benefit of the doubt. But if they try to articulate the reasons and the result is emotional and full of fallacies, I start to think that it's just rationalization and the actual reason is something else, perhaps some opaque emotional reaction that they're trying to justify instead of examining it.

<comment><USER6> If you read a lot, and you probably do, you know that the vast majority of people can't express their way out of a paper box. (Especially if you read stuff written by "amateurs," such as almost all of the internet.)

That doesn't mean that what they are saying is incorrect, just that they are very bad at explaining it.

Ex: Most people "believe" in seasons, and evolution. But you ever try to get them to explain what makes those things WORK?

</comment></comment></comment></comment><comment><USER9> As a senior graphic designer / web developer:

A/B testing is massively useful, but often misused in business. Commercially its become a bit of a buzzword. Which has led it being badly implemented.

I think designers often get frustrated when A/B testing is applied to their designs post-brief.

Designing a website for A/B testing purposes is a completely different brief. You are designing a toolkit for testing purposes. So that the next version will be informed.

A/B is quite is lot more complicated than some people appreciate. Ive often seen people create two completely different designs and then sort of let them fight it out. Yet they learn nothing from these experiments. There could of been elements that work well for certain persona's in each design!

Fundamentally, designer can be quite precious. They don't like criticism like this. After all its impossible to measure all the factors and how does one measure 'creativity'. Plus some will feel like this lowers there input. why not just throw all the elements in bucket and see which comes up trumps.

As a designer I love using this sort of testing. Design by data is a growing trend, that i think will get wider recognition. Sadly the A/B testing phase can create some slightly odd combination at first, but when you go back a re- design the page. You know its going to convert better.

</comment><comment><USER10> I agree with lukestevens; there's a ton of value in A/B testing. A willingness to destroy your attractive but unusable mock is what separates the graphic designers from the interface designers.

That said.... A/B testing can be _extremely dangerous_ :

\- It's VERY EASY to design your test incorrectly. The most common result is that your test really ends up measuring some other factor of the design, or is chock-full of confounding factors. Any conclusions you draw from it will almost certainly be incorrect.

\- It's EVEN EASIER to over-generalize from your data. People clicked on the green button more than the red button...people must like green buttons! Quick, redo the design with a green palette! These conclusions are always terribly tempting to make, since it feels like you've uncovered some deep truth about your users. You probably haven't, and it probably can't be generalized beyond _the exact thing that you tested_.

\- It's easy to substitute A/B testing for critical thinking and self-editing. Two options? Don't worry about deciding! A/B test them! Another decision needs to be made? We'll let the users decide! The problem here is twofold: a). When designing something interactive, there are simply too many decisions to be made. You better get comfortable with making them, because there isn't enough time in the world to A/B test them all. b). (related to below) Your product will end up looking like it was designed-by-committee. Which it was.

\- Related to the previous one, you can get trapped into doing "design by A/B testing". It's a simple algorithm. Get something that looks okay, then do an A/B test on some small changes. Pick the best one. Repeat. Unfortunately, the nature of A/B testing is that it is inherently _isolated_. If you're testing correctly, you've eliminated all your confounding factors. You're just looking at _one_ aspect of your system. The problem is that interfaces are inherently tightly coupled. Everything tends to depend on everything else, and then you start throwing in abstract concepts like "feel" and "friction" and general "look" of the site (how well does it fit together? does it seem to have personality? Does it respect the metaphors it creates? Is it consistent?). A/B testing can't measure that kind of stuff. You also get into problems of local maxima. This is essentially a hill-climbing algorithm. Unless you're willing to occasionally start descending a hill, you're going to get stuck in an okay- but-could-be-better hump forever.

In short: A/B testing is powerful. Don't abuse it. And for godsakes, design your damn tests correctly.

</comment><comment><USER11> This designer doesn't understand A/B testing.

In Ye Olden World of Direct Mail, everyone worked to beat the control. The control, your most successful mailing piece to date, was the benchmark against which everything was measured. You could modify the control, you could take a completely novel approach, but your work only counted for anything if it beat the control. Clients learned to ignore what they thought about a piece and care only about the results.

What the industry learned was that the only sane way to develop a brand new mailing piece was to imitate established controls. Books like Who's Mailing What and Major Mailers provided an index of successful mailings. Copywriters and designers came to understand the Darwinian nature of mailing and gave up on trying to predict what works. They learned that artistry was at best useless, at worst harmful. Their intuitions were completely wrong - the best- performing mailings were almost universally ugly, stupid and reliant on cheap tricks.

Outside of DM, designers almost never think in this manner. Throughout their education, they are taught to produce work that is aesthetically pleasing to the design world, with perhaps some concessions to accessibility. Aesthetics don't pay the bills.

The purpose of A/B testing is to measure the effects of a decision on the bottom line, but also to act as a driver for decisionmaking. A good DM designer has no hesitation in making a mailing piece look like a payslip or a bill, because he knows that such pieces work. His favourite font is Courier, his favourite colour is fluorescent yellow, his favourite adjectives are "splashy", "bold" and "eye-grabbing". He is blind to aesthetics and seeks only to drive the recipient to the next stage - to open the envelope, to read the headline, to the body, to the insert, to the coupon.

If you're looking to build a brand based on aesthetics, a designer is probably the right choice. If you just want to sell some product, 99% of them are a liability.

</comment><comment><USER12> The author is right about incremental A/B testing, though for the wrong reasons. If I do tests:

      A vs B -> B
      B vs C -> C
      C vs D -> D

there is no guarantee that, say, A plus the changes in C wouldn't be better than D. This is the danger of local optima vs global optima.

I don't buy the rest of the arguments.

<comment><USER13> I agree. The article isn't really arguing with A/B _testing_ , it's arguing with hill-climbing. In other words, it's not the questions that are the problem, it's what you do with the answers.

HN is (at this time) programmer-centric. It ought to be obvious that you cannot design the best program on a large scale by optimizing functions. You have to start with a good design and then relentlessly optimize the bits. From time to time you have to consider larger redesigns that would throw all your previous optimizations right out.

Web/UX design is roughly analogous. Customer conversion, for example, is a minefield. You might have two archetypical personae, Alice and Bob. If you start with a design that appeals to the Bobs and the Alices equally, a small change (sexist image?) might alienate the Alices but increase the appeal to the Bobs. If the _immediate_ gain outweighs the loss, it looks good.

But further gains run out of steam when you have all the Bobs but no Alices: You have reached a local maxima as you state so economically. Had you tried a different optimization that marginally increased appeal to both the Alices and the Bobs, you might be able to continue to acquire both the Alices and the Bobs over time, far outstripping the Bob-centric improvements that alienate Alice.

But you'll never know if you went with the biggest immediate gain.

</comment></comment><comment><USER14> It seems like this person's answer boils down to "because sometimes other people use it wrongly," which strikes me as weak. Sure, it's true; he catalogs a few ways the clueless can misuse A/B tests. But you can over-weight any evaluative result. Including (cough, cough) a designer's own opinion of their work.

He's wrong on one point, though: some designers definitely hate A/B testing because they don't like being proved wrong.

A fair number of designers apply a genius-beyond-fathoming approach to presenting designs. It works especially well for design firms; like many consulting companies they sell to people who can't really evaluate their output. As long as they act with confidence, clients think they're brilliant.

Those designers (whose inflexibility, lets be fair, we encouraged by putting them through waterfall hell) have a hard time adjusting to iterative processes. They have to stop pretending confidence in individual designs, and instead develop confidence in their ability to fail skillfully. They are forced to confront the fact that they aren't artists, but instead are professionals who are there to accomplish particular business goals. And they have to learn to compromise gracefully with people of different backgrounds.

Hating all that forced learning is perfectly natural.

</comment><comment><USER15> As a designer myself I know where he is coming from. But it's also obvious that he is building up a strawman.

First of all when you do A/B testing you don't ask the user. You test what composition get the most clicks.

Second A/B does not mean beautiful/ugly but for instance high contrast on call to action / low contrast on call to action.

Third you can't claim that there is no right design and then go on to claim that A/B testing is wrong because it forces you to choose between two alternatives. Then he is basically contradicting his entire claim.

Fourth. There are cases when design is important and cases where it's not. It's not always important and good design is not always beautiful in the way he seem to be talking about it.

At the end of the day better conversion must be the aim for any design. If you want to do art you should become an artist.

Design matters a lot, aesthetics less so. In fact aesthetics are a rather contextually relative thing and should be judged as such.

Only designers who think about A/B as the OP don't like A/B testing. There are plenty who understand that it can actually lead to better design.

</comment><comment><USER16> _Users are not always the best judge of high quality design. That's why you hire expert designers of seasoned skills, experience, judgment, and yes the conviction to make a call as to what's better overall._

In the end, aren't the users the only judge who matter? You hire expert designers to create an interface that your users judge to be pleasing, but also intuitive and easy to use. The user's judgement is king. If your users don't like your interface, it's game over. It doesn't matter how "cohesive" your designer thinks it is.

 _How do you A/B test novel interaction models, conceptual paradigms, visual styles (by the way, visuals & interactions have a two-way rapport, they inform each other, can't separate them–see Mike Kruzeniski's talks) which may vary wildly from before? Would you A/B test the Wii or Dyson or Prius or iPhone? Against what???_

Rephrased: "I don't like A/B testing because you can't test everything using A/B testing." Huh?

 _A/B testing locks you into just two comparative options, an exclusively binary (and thus limited) way of thinking. What about C or D or Z or some other alternatives?_

C'mon. The "A/B" in "A/B testing" isn't meant to limit choices. It's just a convenient way of expressing the idea. There's nothing stopping you from doing "A/B/C/D testing", or even starting out with 10 different ideas, and then using other methods to narrow them down to a few choices that you can then A/B test.

 _What if there are elements of A & B that could blend together to form another option? Avenues for generative design options are shut down by looking at only A and only B._

Sure, if you think of it in a ridiculously restrictive way. So do an A/B/C test between A, B, and some A+B blend that you think might work. Or make it an iterative process: do an A/B test, which let's say shows you B is better. Then do another A/B test, with the original B pitted against a B with some elements of A blended in.

If you look at A/B testing as the be-all-end-all and consider it the only tool in your arsenal, of course you're going to get bad results. But it has its uses in some places, and any designer who refuses to do A/B testing _at all_ just seems uninformed and unwilling to examine hard data about how an interface might work.

</comment><comment><USER17> It boils down to "because they don't get to be in control." Or perhaps "they have to answer to someone else." Or lastly "they don't like being told they're wrong."

I'm not sure any developers like any of those situations either.

"My data structure is _killer_!. What's that? Oh, we now need to track more info at a different level? But... that'll KILL all my ORM work! Users should just deal with what's there - it already works!"

The 'ugly pastiche' issue - I'm not sure why it'd be an ugly pastiche. The author seems to be suggesting that sections that 'won' from multiple disparate tests would be thrown together in to one interface without testing _that_ interface. The little a/b testing I've seen done tended to be more iterative and sequential - design a vs b, then b vs c, then b vs d, then d vs e, etc.

</comment><comment><USER18> I used to think that the only possibly important metric of a page is how many sales it leads to, and with an A/B test you can prove which design is better and that is the end of discussion. I still mostly believe this.

Lately however I realized there is another metric which most A/B tests don't measure (but which perhaps could be measured). That metric is, if a person sees this variation of the page, how likely are they to tell people about the service.

For example if I include some sort of "human interest" content on my landing page such as my personal backstory on why I started the business, that might make it more likely to get mentioned in blogs. But such a backstory really doesn't mean much to people who are contemplating making a purchase and could hurt conversions. Not including it could deprive the landing page of thousands of leads. Including it could raise the conversion rate a bit, but there would be less potential customers to convert.

When a customer looks at the page, you are trying to make a sale. When a member of the media looks at the page, perhaps they would value different things, maybe in that case those site aesthetics would be more important.

</comment><comment><USER19> Of course you shouldn't judge the character or quality of a professional with an A/B test, that would be ridiculous (although I can imagine people doing it).

BUT you should do A/B tests, if you do an A/B test with 1000k+ people and one design converts 20% more than the other, why on earth would you keep the other design? Just because the "professional" designer of the other design wouldn't feel bad?

</comment><comment><USER20> It seems that the author isn't disaffected with A/B testing, but rather with improper implementations thereof. A/B testing is intended to be scientific, so it must therefore adhere to rigorous scientific principles of validity. Sure, there are those who implement it incorrectly, but a properly implemented A/B test should always have a (large) control group, against whom you can measure the purported improvements. Such complaints as A/B testing inherently fostering incrementalism that leads away from proper design can be countered by implementing a "proper" control at each stage. Similarly, his complaint of A/B testing being a binary proposition ignores the 0/1/many axiom of good engineering. An A/B test that forces the user to choose between two unattractive options is a stupid test. An A/B test that shows many options to be unattractive is a good test with a dissatisfying result.

Rather than question the underlying validity of the methodology, the author would be better served by questioning the validity of its implementation in the instances he's encountered.

</comment><comment><USER21> "Instead we get a unsightly pastiche of uneven incrementalism lacking any kind of holistic cohesiveness or suggestive of a bold, vivid, nuanced vision that inspires users."

Uh, you lost me at that point. Try an A/B test by varying "holistic cohesiveness or suggestive of a bold, vivid, nuanced vision that inspires users" with "consistent design integrity that inspires users".

</comment><comment><USER22> > _Users are not always the best judge of high quality design._

Yes they are, actually. If your users don't like the design, the design is bad because the users are all that matter.

</comment><comment><USER23> The author misses the goal of A/B testing: it's to improve a measurable business objective (signups, retention, etc), not to pick a more aesthetic design.

A/B testing may tell you to pick a "worse" design that results in better business. That's perfectly ok. There are plenty of design showcases one can browse for purely aesthetic appreciation of design.

</comment><comment><USER24> Surprised no one has mentioned Doug Bowman's reasons for leaving Google, which seems in part because of extensive use of A/B testing. I don't really know enough to comment on it but it seems relevant: <http://stopdesign.com/archive/2009/03/20/goodbye-google.html>

</comment><comment><USER25> Is A/B testing a greedy algorithm?

I don't know much about A/B testing, nor have I done any. My first thought is: how do you avoid getting stuck on local maxima?

(The wikipedia article on A/B testing mentions A/B..Z testing and 'multivariant testing', which may be an answer to my question, but they don't seem to get much mention.)

<comment><USER7> That's actually a valid point with A/B and MV testing - you can get stuck on local maxima. It's why the process still requires manual intervention and design skills. Most flaws with A/B implementations are people/process or misuse.

</comment></comment><comment><USER26> I found that developers edge out designers at predicting the results of A|B tests:

<http://method.ac/blog/design/devepers-designers-results.html>

</comment><comment><USER27> I think A/B testing in relation to design is a bit like living by the dice (a la the diceman).

You only pick alternatives you'd be happy to see play out.</comment>


Ask HN: How do you setup test clustering/RAID in single computer? - <AUTHOR>

Pls advice. <comment><USER2> I just want to say one word to you: VMware.

<comment><USER3> Yes, I've set up several clustering situations for our training videos to demo the how to, etc. VMWare is the easiest although MSFT has their own as well.</comment></comment>


GO2P, a simple but full configurable P2P framework - <AUTHOR> https://github.com/v-braun/go2p <comment><USER2> Anyone else think about going to the bathroom? But seriously Go projects are always fun to go through. Go code is usually very legible so even if you dont write much Go code it doesnt hurt to look at how Go developers write code.</comment>


Don't design like a programmer (2010) - <AUTHOR> http://www.uxdesignedge.com/2010/03/dont-design-like-a-programmer/ <comment><USER2> > Exposing representation invariants In object-oriented programming, a > representation invariant is a set of constraints that a concrete data type > must uphold to correctly represent an abstract data type. Simply mapping a > data structure to a UI would expose the invariant and put the burden of > enforcing the invariant on users instead of the code. A typical example is > having to enter a phone or part number in a specific format instead of the > program handling any reasonable input. > Warning sign: Users required to > enter data using a specific format.

This one drives me nuts. Soooooo many forms say "enter your phone number no spaces or dashes or parens" or cc number. Please just remove the dashes, spaces, parens when checking validation and maybe after submit. Don't force the user to have to do this. They might be copying and pasting

Also don't make some 4 field 16 digit custom form that ruins backspace behavior and makes it impossible to copy and paste. That also means don't limit the field to 16 characters. If a users pastes in "xxxx-yyyy-zzzz-wwww" and only get "xxxx-yyyy-zzzz-w" you're only going to frustrate them.

This is also an issue with many phone apps. Some you can't paste a number in. Try copying a number from Safari iOS into Google Hangouts Phone dialer. There's no paste option so I'm forced to write down the number on paper so I can manaully type it back in. Even the built in iOS phone dialer sucks. While you can paste you can't edit. So if you copy something off a webpage and because of the crappy smartphone copy/paste issues you get something like "dial:123-4556" there's no way to remove the "dial:" part. At best you have to paste into some other app that will let you edit, edit, then copy and paste again. Sheesh, does no one ever use this stuff?

<comment><USER3> Another one is password restrictions: if I want to use a one-character password, that's my business. If I want to use my birthday as password, that's my business.

If the app developer is concerned about security in case an attacker gets hold of the raw database, there are better tools for that (i.e. proper salting and hashing)

<comment><USER4> Salting and hashing isn't going to help you much if you're prone to using one- character passwords. Unfortunately, people will use one-character passwords if they are allowed to, so systems should require strong passwords to protect users.

That's not to say there aren't problems with password restrictions, but they are mostly at the other end of the spectrum. For example, I have seen far too many systems requiring that passwords must be no longer than ten characters long, or only allowing a very small subset of characters.

<comment><USER3> > Salting and hashing isn't going to help you much if you're prone to using > one-character passwords

Why not? OK I'm no cryptographer, so I'm probably talking out of my arse but why not store a random salt PER user, and the hash of (password + salt)? To an attacker, a one character password will be as hard to crack as a complex one.

<comment><USER5> > hard

A 1-char password can be guessed in as many attempts as the size of the allowed character set, and in half the guesses on average. Even less guesses are required on average if passwords aren't distributed uniformly and this distribution is known (i.e. from previously cracked password databases you know most users pick "e" for a password, so you start with that character and get into the accounts of 40% of all other users in a single guess).

More generally, the average number of attempts required to crack a password with a known length is 1/2 * charsetsize ^ length. So with alphanumeric case- sensitive 1-char passwords the number of attempts required on average is 31. That's not a lot.

The only thing that salt + hash does is make it possible to check the password without having to store the passwords on server; it only serves to protect user passwords _after_ the password (hash) database has been stolen by attackers.

The only way to keep a semblance of security when user passwords are very short is to aggressively rate-limit password attempts, but in the case of 1 char passwords that doesn't help, you'd have to lock the user out after a single wrong password entry, and even then attackers would have a chance of 1/62 (26 lowercase letters, 26 uppercase letters and 10 numbers) chance of getting into your account.

<comment><USER6> Bravo.

</comment></comment></comment></comment></comment></comment><comment><USER7> This is actually pretty well thought out here. I feel like the most important one is the one that I consider to be the most difficult: Optimizing for the most probable.

To me, it's always a struggle between probability, predictability, and shortest possible paths between states. Probability being that at any given point, the path to what the user is most likely trying to do is as short as possible. Predictability means that from any given state, there is a consistent and predictable way that the user can know how to get to any other state. And finally, that from any state, the ability to switch to any other state in the application is as short as possible (which obviously conflicts with probability).

In my mind, almost every UX should be the equivalent of tabbed, nested folders. This takes care of predictability and short paths between all application states. The area this falls behind is in probability, in that there is no effort made to guess what the user wants. Given that your average user in a large application is unlikely to care about anything outside of the given task they're trying to accomplish, I feel like it's arguably more important than the things I traditionally prioritize for, and therefore something that I have to work on the most.

</comment><comment><USER8> As a 'programmer' I find I often have a better idea than end users about what should be implemented. I sit in front of the system I am building all day, and I notice if performing certain actions is too slow or frustrating. I take an interest in UX concepts and will read articles on them, as opposed to "can we get a new button here to do X" without any real thought going into the process.

<comment><USER9> That only works if you work on apps that you actually use, and more importantly use in the same way that your end users use it.

Having worked on specialized engineering applications where I'm not the end user, I'll admit to on more than one occasion making some really stupid UX decisions since it turns out that the engineers actually working with the app on real projects where using it very differently than I was when testing it on a tiny made up toy project.

Spending just 20 minutes sitting next to someone trying to get real work done can be a real eye opener some times.

<comment><USER8> I watched the users of my last app cut and paste data from the application into excel, mess with the data in excel, upload the excel back to the database and ask me what was causing the errors when the unvalidated data in excel would not go back in to the DB. It was eye opening how they accept such a crap system. I tried to change it, but it was never given high enough priority.

<comment><USER10> Well, then, it's your responsibility to make sure that your application can cope with data pasted in from Excel, since that's obviously a critical part of the users' workflow.

<comment><USER8> It was a "temporary solution".

"Yes, you can built a web form with proper validation to replace it next month after you complete this URGENT feature".

If it can be replaced by a web form then Excel isn't a "critical part of the workflow". Getting data into the database was the critical part of the workflow. The users actually preferred the web forms that I did manage to replace Excel sheets with (more checkboxes, and far less cut and paste).

Anyway after 3 years of complaining about the same issues I realised it was time to leave.

</comment></comment></comment></comment></comment><comment><USER11> This is a very good article. We should generalize this to quote Steve Jobs : 'We have to start with the user experience and work backwards to the technology'[1].

A lot of companies try to get around this by hiring external design teams who decide the user experience first and then asks engineers to implement the experience. Of course then problem is then if they are not technical enough they might not take into account how hard something could be ( [http://xkcd.com/1425](http://xkcd.com/1425) )

[1] [https://www.youtube.com/watch?v=1SIeTmORl0E](https://www.youtube.com/watch?v=1SIeTmORl0E)

</comment><comment><USER12> There's also a follow up where the wget UI is redesigned [0]. It's important to also read this because that's where the argument falls completely apart: most of the features have been sacrificed to the altar of "simplicity": the redesigned application is essentially useless.

Not only that, the design includes a really nice looking progress bar that is, unfortunately, impossible to implement.

[0] [http://www.uxdesignedge.com/2011/11/don%E2%80%99t-design- lik...](http://www.uxdesignedge.com/2011/11/don%E2%80%99t-design-like-a- programmer-part-3/)

<comment><USER3> I do not agree. He explained the thought process two times:

1) "The target users for SiteNabber are intermediate computer users. They have enough knowledge about computers and the Internet to want to copy a website's files, but they aren't so advanced that they would prefer using a more complex existing solution. Experts who want to control every detail aren't target users!"

I agree with that. If you need a GUI you're not comfortable playing with wget from the command line and/or spending time reading the wget documentation. But you're advanced enough to know it is possible to download whole websites (!)

2) "Step 12: Test with real users, so there's a good chance that I have some details wrong. This is normal and expected."

Design, like programming, is a never ending process. Last 20% take 80% of the time, yadda yadda. I am a programmer, and I desperately want to learn what is wrong with my designs. This article explained very clearly what is wrong in my (and most programmers) thought process when designing an UI:

Programmer flow: Write code -> Design UI around code -> Change code -> Apply changes to UI -> Rinse & Repeat

Correct, designer flow: Think long and hard about your target users and why they'll use your product -> Design the UI around their goals -> Write code -> Test with humans -> Adapt design to better suit their needs -> Rinse & Repeat

In the "Programmer's flow", code changes drive UI changes. When the code is feature complete and bug free the app is done.

The correct way is to have end users drive your UI changes. And till users aren't satisfied you are not done, even if the code is perfect.

<comment><USER13> "If you need a GUI you're not comfortable playing with wget from the command line and/or spending time reading the wget documentation. But you're advanced enough to know it is possible to download whole websites (!)"

I disagree. Sure, I _could_ run wGet from the command line, but then I have to either know what various switches/options are available and what kind of input it's expecting or consult the help parameter/man page. Maybe it's a tool I only use every once in a while.

The GUI puts all those options in front of me at once, with the ability to select them right next to the description of what they are, and the "ugly" GUI doesn't require me to navigate to sub-forms to discover/change other options available to me.

...but then, I'm a programmer, so what do I know?

</comment></comment></comment><comment><USER14> I think there is an interesting pattern that every larger software interface that is designed mostly by the developers converges against the IDE in which it was developed. But then on the other hand, superficial designers often just adjust towards the look and feel of their preferred environment as well (e.g. tendency towards the old iPhone look back then, then suddenly switch towards the flat new look of iOS, etc. And even in non-mobile applications like web applications).

<comment><USER15> The flat look of ios was a catch-up design.

But you are right that there are very strong trends in at least visual UI

</comment></comment><comment><USER16> This misses some crucial observations.

One, good UI takes time, even for experts. UI redesigns that are easily described can be hard to reprogram (e.g. "replace those dialog elements with a table!" may mean switching to entirely different programming interfaces, there may be no sane data model that matches both without rework, and the new method may have new bugs). If your manager is asking you to finish something and move on, you need to make trade-offs.

Two, historically there had been no serious shell in _Windows_ environments for a very long time. When you have _no_ other way to easily set a bunch of options, a crappy dialog that maps GUI elements directly to all possible options would have been a quick and practical way to expose all functions. I am not saying that these GUIs are _good_ in any way but it makes perfect sense why they were created this way.

Third, low-level interfaces such as the command line are _an EQUALLY important part of all UI_. A "design" should not just focus on how to rearrange some buttons. Low-level options for tweaking a program can help immensely to edit down what belongs in the GUI. On Mac OS X for example, it is really cool that one can use "defaults write" for a bunch of stuff because you can implement a lot of options without cramming them into your GUI. On a web site, a thoughtful layout for service URLs and JSON data formatting is far better than something that seems hastily-constructed and full of structural problems and other inconveniences.

And also, this post is not showing convincing counter-examples.

That "SiteNabber" redesign from part 3 is not really fixing the _real_ issue. If you're going to create a _graphical_ replacement for a tool that deals with web sites, exposing "site url" to users is still broken. Why not make it a browser plug-in where you can simply _view_ the web site you want, and click a toolbar button to save what you are looking at (or long-press that button for a menu that exposes a dialog version of the save with a few common options)?

</comment><comment><USER17> I've seen several applications that look like wget UI. They often are internal engineering applications, and, in the main work fairly well for the target audience. I think programmers know exactly what a good UI might look like, but (a) they don't always have to time to make such a thing. (b) it's not cost or time effective to produce one.

Personally I've done both sort of user interfaces, the quick crowded ones, and the task Centeric ones, where time is spent on design and redesign.

I don't really necessarily think that programmers are inherently bad screen design, but it requires the time/effort and inclination; where often, one or more of those are in short supply.

<comment><USER18> That was my thought: First, a text box to make entry POSSIBLE, then, if time allows, a select/option list to constrain to meaningful answers, then, the "other-with-text" option to allow the long tail of never before seen but valid answers.

And so on, as time allows.

</comment></comment><comment><USER19> > "Having a privacy hurdle If the data structure needs personal information or > information that is difficult to obtain (such as having an account, finding > an account name or password), users might be unwilling to provide such > information—especially if such information shouldn't be required to perform > (or at least initiate) the task. Warning sign: Users are required to have > accounts, log on, or provide personal information unnecessarily."

Just about every app, service, and site does this now. It's ridiculous that this is the case, but everyone wants a slice of the pie so to speak.

</comment><comment><USER20> I disagree with the date comment "it's easier to expose a raw date than a properly formatted date for the user's locale". I'm constantly fighting software that tries to localize dates. Paypal thinks I live in country A and uses that format. My web host is in country B and uses their own format. I actually live in country C which has yet another format. It's no fun reconciling this in a spreadsheet which I have to keep changing the date formats in. Just use YYYY-MM-DD everywhere. Everyone knows what it means.

</comment><comment><USER21> Not a other critizism of the wget UI! We know it's bad, but it's simply because it's a 1:1 mapping to the command-line options. And command-line options is a flattened representation of all options. They are quite often not even combineable. A well-designed options class hierarchy with all constraints in place would map quite nicely to a GUI wizard.

</comment><comment><USER22> This reminds me how much I miss those old clusterfuck interfaces of early windows day.

I find them much more easy to play with and learn them in details.

<comment><USER23> Windows 95 to 2000 and the respective Microsoft software during this time had the best interfaces, hands down. Minimalistic, grey, fast.

<comment><USER10> I miss those and their Unix counterparts, the old Motif apps. We should bring Motif back.

Heck, with flat design being a thing, we should bring Xaw back.

</comment></comment></comment><comment><USER24> Another take on the same theme: [http://www.howtospotapsychopath.com/2009/05/21/a-surprisingl...](http://www.howtospotapsychopath.com/2009/05/21/a-surprisingly- un-awful-interface/)</comment>


Show HN: A simple python API for doing face recognition in 5 lines of code - <AUTHOR> https://github.com/ageitgey/face_recognition <comment><USER2> I am so excited to tinker with this. Looks awesome.</comment>


Gambit in Emacs in the browser - <AUTHOR> https://feeley.github.io/gambit-in-emacs-in-the-browser/ <comment><USER2> The best thing about this is that there is emulated thread management via setTimeout going on underneath to allow for Ctrl-C and other interactivity while things are running. No background workers, no running things on the server, this is some single-page engineering goodness.

It implements proper tail calls, too, so it looks like you can run `(define (f) (f)) (f)` forever.

Cool stuff, this is one of the most robust single-page REPL experiences I've seen.

</comment><comment><USER3> This is hilariously awesome! Aparently it's using ymacs (which I didn't know about until now) [http://www.ymacs.org/](http://www.ymacs.org/) for the 'emacs'and an emscripten compiled version of Gambit (scheme).

So wrong, yet so right. I would hire this guy.

(working swell in Chrome)

</comment><comment><USER4> I'm glad you guys appreciate the hack value! I'm personally amazed at Mihai Bazon's work on ymacs. By the way, if you want to drive the DOM from Scheme, there's the builtin function jseval:

(jseval "alert('hello')")

</comment><comment><USER5> Interacting with the REPL directly works for a few forms, and then it seems to get confused, interpreting output as part of the next form (I'm on Chrome for Linux version 38.0.2125.104 (64-bit)). However, when I open a buffer and write the Scheme over there, sending expressions as I usually do when developing, it works beautifully. An example:

        > (display "help!")
        help!>(newline)
        *** ERROR IN (console)@2.1 -- Unbound variable: help!>

That problem aside, this is really, really cool, and I enjoyed playing with it using a separate buffer.

</comment><comment><USER6> The code is here: [https://github.com/feeley/gambit-in-emacs-in-the- browser/tre...](https://github.com/feeley/gambit-in-emacs-in-the- browser/tree/gh-pages)

</comment><comment><USER7> This thing just has killed my browser.

<comment><USER8> Can confirm it on Firefox 33.1 in Linux. The memory usage shot upto 98% and swap got filled by 50%. Had to kill the process.

</comment></comment><comment><USER9> What's Gambit? Links like this without context aren't useful.

<comment><USER10> It's an excellent Scheme implementation: [http://dynamo.iro.umontreal.ca/wiki/index.php/Main_Page](http://dynamo.iro.umontreal.ca/wiki/index.php/Main_Page)

I agree with your sentiment in general, although I feel Gambit is pretty well- known, especially because of its merits for application delivery.

</comment></comment><comment><USER11> Does it have tail calls, dynamic-wind, and hygienic macros? If so, I have a very interesting use for this in creating visualizations of microkanren execution traces.

</comment><comment><USER12> Why did you create it?</comment>


GnuPG 2.1.2 released - <AUTHOR> https://lists.gnupg.org/pipermail/gnupg-announce/2015q1/000361.html

<comment><USER2> What is different from the previous 2.1 is not clear.

The most interesting part is:

"Since the start of the funding campaign in December several thousand people have been kind enough to donate a total of 250000 Euro to support this project. In addition the Linux Foundation gave a grant of $ 60000 for 2015, Stripe.com and Facebook.com each pledged $ 50000 per year.

I am amazed by this superb and unexpected support for the GnuPG project. This will not only allow us to continue the project and hire at least a second full time developer but gives us also the resources to improve things which have been delayed for too long."

I think everybody agrees that the most of the success of the campaign is due to this single article:

[http://www.propublica.org/article/the-worlds-email- encryptio...](http://www.propublica.org/article/the-worlds-email-encryption- software-relies-on-one-guy-who-is-going-broke)

<comment><USER3> And to my delight, the article was written by Julia Angwin whose book

[1] J. Angwin. Dragnet Nation: A Quest for Privacy, Security, and Freedom in a World of Relentless Surveillance. Times Books, 2014.

I recently finished and recommend. It's one of those few "books" written these days (quotation marks because they`re really more like stretched-out magazine articles) that actually make sense, come from the right place in terms of sentiment and tone, and most important are logical or coherent. I was curious about the author because I was so surprised and saw in her bio that she studied mathematics in college. Go figure ;-)

<comment><USER4> Yeah, without taking any credit away from the donors (or from the GPG authors, of course), we should be grateful to her and ProPublica for bringing this to light.

It kinda bothers me that, while I can make a donation to ProPublica, there's no way to express _why_ I'm doing so. On the other hand, maybe there's a danger in becoming too donation-focused.

EDIT: After donating, they say: _" Please consider sending a note to firstname.lastname@example.com or tweeting @ProPublica sharing your reason for donating. We'll use some of these messages to encourage others to donate."_

Which I think is fair enough.

<comment><USER5> Why cryptocurrency/cryptoempathy needs fields for attaching messages for meaning to value and account why we give.

Giving sharing significance and expressivity by allowing message passing for basic help as a fundamental part of currency.

</comment></comment></comment></comment><comment><USER6> A quick PSA for those using Mac OSX with homebrew or a Linux variant with apt- get:

There are two versions of gpg on homebrew. If you're like me, you installed gpg with:

        brew install gpg

or

        sudo apt-get install gpg

However this gives you:

        $ gpg --version
        gpg (GnuPG) 1.4.18
        ...

What you probably want is:

        brew install gpg2

or

        sudo apt-get install gnupg2

This gives you:

        $ gpg2 --version
        gpg (GnuPG) 2.0.26
        libgcrypt 1.6.2
        ...

This is probably what you want, as it's the stable version.

There's no harm in using gpg 1.4.x; the docs indicate that this is maintained because of its use on older and embedded systems, and AFAIK there's no reason to suspect gpg 1.4.x's security if you trust 2.0.x. It's just that the 1.4.x version has fewer features.

I haven't tried other package managers, but I suspect they also have this minor pitfall.

<comment><USER7> It's worth noting that Homebrew has "gpg" and "gpg2" as aliases for the "gnupg" and "gnupg2" formulae, respectively. This might be confusing to some people wondering why there are two versions of the same thing.

</comment></comment><comment><USER8> If anyone here is a `pass` [1] user, and you're using 2.1.1, or 2.1.0, I encourage you to upgrade. These two older versions of GnuPG had some nasty bugs, fixes for which the pass community sent upstream where they were accepted.

The result is that pass 1.6.5 and GnuPG 2.1.2 work nicely together.

[1] [http://www.passwordstore.org/](http://www.passwordstore.org/)

<comment><USER9> Interested on those nasty bugs too, as I am on pass mailing list and have not read anything about them. This sounds more like a plug to pass, from a throwaway account.

<comment><USER8> Um, no. This isn't a throwaway account. This is my account - the username I use for everything. Type it into google. It's been my handle since I was super young. Whois the .com of it - I registered it in 2000.

IN FACT the mailing list archives of the pass mailing list happen to be at that domain. Here's the relevant post: [http://lists.zx2c4.com/pipermail/password- store/2015-January...](http://lists.zx2c4.com/pipermail/password- store/2015-January/001336.html) This is from a CC that went to the GnuPG mailing list.

<comment><USER9> I said "it sounds...", not that it was from a throwaway account. The "nasty" bugs you referred were never a problem for me, and the way you wrote sounded sensationalist.

<comment><USER8> I'm glad to hear you didn't have trouble with those bugs. They actually prevented new packages from building in Debian and Fedora, due to our unit tests catching the bugs, and we had to do a double-release on one day to fix them -- quite stressful for us. Anyway, it's a nice thing you weren't directly affected.

</comment></comment></comment></comment></comment><comment><USER10> I've been wondering if there was anything like "proper" forwarding in gnupg- agent, and thus at least signature support over SSH connections for remote mail clients.

My use-case: I have a shell server which receives my emails. I use a local client (mutt) to read my mails, but I do _not_ want to save my private key on the server, because it is not 100% under my control. The idea is that my physically local box would hold my private keys, and the agent would simply forward the to-be-signed data from the remote host to my local system, and transmit the signed data back.

When I send encrypted emails, at least those I can easily do on my physical box and send the file over first. It's a bit of an inconvenience, but I can live with that. Being able to sign my mails on that remote shell box without actually putting my keys there is the one thing I'm looking for.

I found some kind of hack for this a year or so back, but now I can't even remember what google-fu I had to employ.

</comment><comment><USER11> Is there any work being done to formally prove GnuPG' algorithms correctness? Just curious since there are other topic around Coq in the front page and I couldn't find much by searching.

<comment><USER12> Algorithmic correctness is probably less interesting than the presence of side-channels nowadays, and that's far, far, far harder to model.

<comment><USER13> I do know of one researcher working on this: Julian Bangert at MIT out of Dartmouth has modeled x86 family processors in SMT, and is able to prove that all paths through a particular piece of code compile so that all cache hits are identical, CPU ticks are identical, etc. Brilliant young man - I expect we'll be seeing a lot more from him.

</comment></comment></comment><comment><USER14> Does anyone know when Curve25519 (encryption, not signing, I know Ed25519 is already there) will be implemented? I'm waiting until that happens to generate my new long-term keypair.

</comment><comment><USER15> It's very generous of them to do this work and give it away, but consider paying for what you are getting.</comment>


New Google Layout - <AUTHOR> http://www.davidnaylor.co.uk/new-google-layout.html <comment><USER2> They've been testing it for a while although, its starting to rollout more recently. I've been noticing it mainly when I've been in Gmail and the new design is reportedly linked to the "+1" which is also rumoured/expected to launch soon.</comment>


Leo editor – "a revolutionary step in the right direction for programming" - <AUTHOR> http://leoeditor.com/ <comment><USER2> I use Leo every day. It's a great little editor, very stable and with a ton of handy features.

Outlines are a fantastic tool to keep track of the state of a complex project. My .leo files are what keeps me sane when I'm in the middle of a project with a few hundred documents and need to compile a report. I like it that it's local instead of some online service (there are plenty of those, but I would feel remiss if I used them for something that is supposed to be confidential).

Leo is not very polished, it's a programmers tool and I'm ok with that. I tried getting the hang of emacs 'org' mode for a while but the learning curve was just too steep for me. I want my tools to get out of the way rather than that they dictate my whole workflow and Leo is light enough to do that. It's written in Python, you can modifiy it fairly easily. There are some warts (for instance, the export tree to html function is next to useless) but on the whole I'm very grateful that it exists and I would not know of a replacement other than emacs org mode.

If you're into literal programming Leo is very nice for that as well.

True, it's no IDE. It's not in the same space as Eclipse, Intellij and so on. But if you don't actually need a full IDE and you make your outlines in some non-structured editor then give Leo a try, you probably won't regret it.

<comment><USER3> > emacs 'org' mode for a while but the learning curve was just too steep for > me.

What's to learn? Pretty sure the core of org mode including tables, todos, times etc can be picked up in 15 mins

<comment><USER2> Wish the same was true for emacs. Using a new editor is like learning to walk all over again.

</comment></comment></comment><comment><USER4> The superlatives are being laid on a little thick and likely to invite criticism. There are lots of similar features here that have been introduced in other IDEs and environments before.

<comment><USER5> I hope that sample screen shot is not supposed to suck me in, is it? Honestly I took one look at that screen and closed the tab.

Is this particular editor suited to a specific language or set of languages?

<comment><USER6> Arthur here. We designed that screen shot to be as bad as possible! Our goal was to get people to leave the site as quickly as we could and then to ask random strangers about our IDE rather than clicking around for a few minutes and seeing what it offered. Mission accomplished!

<comment><USER7> Can you explain your reasoning behind that goal? It doesn't seem like something I'd want, but I'm not a great marketer so I'm curious what you'll gain from it.

<comment><USER8> Exhibit A on why sarcasm detection is difficult. ([https://news.ycombinator.com/item?id=7845193](https://news.ycombinator.com/item?id=7845193))

</comment></comment></comment></comment></comment><comment><USER9> I'm having a hard time figuring out what it is that differentiates Leo from everything else. There's a link "Learn why Leo is so special" that goes to a bunch of links, each filled with quotes from random people gushing with praise.

I'm sure it's fine to have so many people liking your product, but I still don't get what Leo actually DOES, or how it's going to help me write better code / organize my time better / make me the most popular guy in the room.

I went to the screencasts page but came away more confused than ever. It looked like he was just cutting/pasting nodes in a tree for arbitrary reasons.

<comment><USER10> I would like to echo these comments. "Learn why Leo is so special" was especially misleading.

How about a 4-paragraph summary of what Leo actually is, because I poked around the website for several minutes and still have no idea... other than that it's written in Python, which is a plus for me. :-)

The "Learn about Leo in two hours" just links to the documentation, which is conspicuously missing a "What is Leo?" introduction and goes right into commands.

I'm very intrigued by what this project promises, but I'm intrigued by a lot of projects. If I can't get a good idea of what the software even does in 5 minutes, someone has failed. Maybe it's me, but I don't think so.

</comment></comment><comment><USER11> I love (don't use, but love) editors like this -- that push at some boundaries and rather than having that push be some "side feature" or "addon" \-- is the core part of the editor.

An older editor that I thought was amazing was Acme ([http://en.wikipedia.org/wiki/Acme_%28text_editor%29](http://en.wikipedia.org/wiki/Acme_%28text_editor%29)) -- the famous plan 9 editor. [http://youtu.be/dP1xVpMPn8M](http://youtu.be/dP1xVpMPn8M) is a Tour of Acme, some really cool features are built into it. Any text is a command.

<comment><USER12> > editors like this -- that push at some boundaries and rather than having > that push be some "side feature" or "addon"

The problem with this is that it's really, really hard to build a good editor, and when you concentrate on some novel idea the "basic" functionality suffers.

For example, both Vim and Emacs have easily accessed commands for moving cursor by words, sentences, paragraphs and moving to the nearest given character (on top of traditional beginning and end of line, of course). Also for selecting them, copying and pasting, indenting, surrounding and so on, all in 2 to 3 keystrokes. Another example is the multilevel, tree-like, infinite undo of both editors. And many more: mark ring, multilevel "clipboard", registers, very easy to access macros, narrow to region (this one _could_ be Emacs specific, but I doubt there's no Vim plugin for this), search (both literal and regex) and replace (both literal and with _expression_ , incredibly powerful) and so on and on, and that's just built-in purely editing features, not going into integration with other tools or plugins.

I would have migrated to LightTable long ago if it provided at least some of a more advanced, but purely editing oriented features. It doesn't yet, and it will take it long years to get there because of sheer size of a required featureset. And that's still not counting my 3k lines of personal Elisp and some 20k lines of Elisp in installed plugins.

I'm going to give Leo a try, but even if it really is as good as it's web page says I can't see myself migrating to it in the near future. I may use it from time to time for specific tasks, but my daily editing will be done in Emacs; in the worst case I will have to copy&paste bits of text between Emacs and Leo windows if Leo's editing model/core editing functionality sucks enough (I did this with some other editors and I despite copy&paste overhead I was still faster than I'd be staying in the single editor).

<comment><USER2> We grow attached to our tools. I built a full screen editor somewhere in the 80's, used it until 2005 or so, when I really could no longer justify (to myself) the effort required of keeping it going. I _still_ type 'e filename' or will use its weird set of control keys when I'm not paying attention. And it's almost 9 years now since I stopped using it. So I suspect you'll be back with emacs before the first day is out. But please do look at leo, it has some interesting ideas embedded in it and there may be ways to get emacs to adopt some of those.

Leo is the best implementation of the literate programming method that I've come across to date, and the first time when it all clicked for me.

</comment></comment></comment><comment><USER12> Look at [http://leoeditor.com/screencasts.html](http://leoeditor.com/screencasts.html) if you want to quickly see what this is about.

My impression is that it's a neat idea, with some features borrowed from Emacs and others borrowed from Smalltalk and Morphic, but packaged in a single easy to use editor. Which is nice. I'll definitely try it out when I get some free time.

<comment><USER13> Sadly the author does not explain what he does when manipulating the leo nodes, and it makes it difficult to follow what haopens, which made me abandon the screencasts

</comment></comment><comment><USER14> I'm watching a few of these screencasts, and I'm really not understanding this editor. Could anyone enlighten me or point me in the right direction?

Is this editor only for python? What is an outline? I'm pretty lost on what this editor is all about.

</comment><comment><USER15> I wrote Python in Leo for a few months 7 (?) years ago. The best use case I found was for bug tracking, you could make a URL node to the ticket and then clone all the relevant pieces of code under it to have everything in the same place. The problem is that outlines required sigils in the source code at the time and that makes sharing a repo with somebody else difficult. The other problem is that it's not Vim.

I expect some version of Leo's outline concept to be a banner feature in some IDE in the future. The idea is good but it's execution in Leo is workable but clunky.

<comment><USER16>
      The other problem is that it's not Vim.

I also have this problem when trying out any other text editor. It's actually quite annoying because every other text editor I try is often much better than Vim at everything it does except editing text.

I'm holding out for someone with more time and patience than me to write Vimim.

<comment><USER17> There's a line on the home page that says "Leo works well with emacs, xemacs, vim, and ipython." And they have a page about using Leo with other programs, which does have some details about Vim and Emacs:

[http://leoeditor.com/leoandotherprograms.html](http://leoeditor.com/leoandotherprograms.html)

But I haven't tried any of it yet.

<comment><USER16> Thanks, I did see that actually. Obviously I'll have to try before having an opinion on how well it works. I did try sublime (an editor I really like) with the vi extension and found that, although support was comprehensive, there were enough differences for me to ultimately give up.

It would be great if you could integrate Vim itself (or any text editor) with Leo, similar to how Vim integrates with Mutt.

In any case, it's worthy of at least a play because it has some very interesting functionality.

</comment></comment></comment></comment><comment><USER18> The time is ripe for a tool that integrates knowledge management and development with this "linked outline" structure; it's an upgrade over the classical folder-based storage of files.

There have been previous attempts to create integrated editors for semi- structured information both from the software (Code Bubbles) and documentation (Haystack project, Chandler) perspectives. Machines are now powerful enough for them to become useful and responsive (as these environments consume huge amounts of memory).

For such a tool to become widespread though, it will need to be based on user- friendly design, and be able to share content on the web. Evernote and Workflowy come to mind as tools with those properties, but they don't handle code execution; and reactive, live programming tools like jsfiddle don't allow for outline-like structuring.

It would be great if such a tool was also Open Source, but that's unfortunately not a pre-requisite for success; I imagine Google or Apple will be the first to build such development environment, or to buy a start-up that build it and turn it into an standard.

</comment><comment><USER19> I broadly speaking applaud the idea of breaking code into code-units which are independent of a specific file hierarchy and tagged with meta data; then reassembling them as a pre-compile / pre-run step.

As far as I can understand this is broadly speaking what leo 'nodes' and 'outlines' are.

It could really do with some better examples of usage though.

I'm also a little skeptical in some regards though; I don't see any tests, and 'mixing in' code units like this (using outline scripts) in various places surely means you have tonnes are unexpected side effects when you change the code units?

Still, interesting stuff.

<comment><USER2> The way it does that is by making 'clones'. So for example, if you have a long program that you find a bug in then you take the node with the buggy code (usually a few lines, maybe 10's of lines) and you 'clone' that node to the spot where you have your bug report or list of tickets or whatever. You then describe the bug, the fix and modify the code (which will modify the original code as well), or you modify the original code leaving the clone as a 'living pointer' to the code to document the location where the bug was found. You add the unit test as another node right where the bug report lives.

The 'thin' model (where you import external files without actually having them present in your outline) is a variation on this theme, another way of doing it is to have the code present in the outline itself and then to export the code whenever the document gets saved.

Both methods have their pros and cons. The biggest con in my opinion is that leo use is not widespread enough that you can expect to collaborate with someone else like this. That means that if you post your project on github or if you share the code with a fellow coder that all that Leo mark-up is wasted. This also likely does not play nicely with version control but I haven't tried that particular scenario myself (I just dump the whole tree in git or subversion and I work on most of this stuff on my own anyway).

</comment></comment><comment><USER20> On GitHub here: [https://github.com/leo-editor/leo- editor](https://github.com/leo-editor/leo-editor)

</comment><comment><USER21> [https://en.wikipedia.org/wiki/Folding_editor#Editors_with_fo...](https://en.wikipedia.org/wiki/Folding_editor#Editors_with_folding_capability)

Voilà.

Most likely you have an "outliner" already at your fingertips. ;-)

</comment><comment><USER22> I spent a fair bit of time on the site and I'm convinced the author is actively trying to prevent people from using this product.

</comment><comment><USER23> Roger me sidewise, that editor screenshot looks like a Kindernet Exploder window from 1995.

I'm still waiting for the code editor that takes mouse input exclusively (via gazillions of icons, etc) and refuses any keyboard input. Is Leo on the way there?

</comment><comment><USER24> This is a troll, right? It looks like a windows 3.1 editor.

</comment><comment><USER25> You had me until "Qt".

<comment><USER26> what?

</comment></comment><comment><USER27> As far as I can tell, it only runs on Windows.

In my opinion, any general programming tool, however revolutionary, is at a significant disadvantage if it lacks multiplatform support.

<comment><USER28> I haven't tried it, but at least they claim that it runs on all three major platforms:

"Leo a PyQt app that runs on Linux, Windows and MacOS."</comment></comment>


Steve Jobs: Thank you for disobeying me - <AUTHOR> http://www.folklore.org/StoryView.py?project=Macintosh&story=Hide_Under_This_Desk.txt

<comment><USER2> Skunkworks, underground, and off-the-radar initiatives are a lot more common in large technology organizations than people might imagine. While you may have "Product Action Committee's" and "Executive Commits" and "Roadmaps" that theoretically set the mandate/budget and resourcing for the company, many of the best projects occur because a few (sometimes only one) smart engineer or manager gets it in their mind to do their own thing.

I will admit, it takes a little gall to hide an entire T&E budget to Japan on a skunkworks project though. Great Story.

<comment><USER3> It kind of reminds me of the story of the development of the Graphic Calculator application. <http://www.pacifict.com/Story/>

</comment></comment><comment><USER4> Hee. We used to find Twiggy drives in various places (typically in the back of labs when we moved, or when some older cow-orker quit and left boxes of crap behind). We'd send the drives via inter-office mail to one of the original hardware guys responsible for the Twiggy.

We were still finding the damned things and shipping them off to him in 1993. The H/W guy had a pretty good sense of humor about it.

What future albatrosses are /you/ working on? :-)

</comment><comment><USER5> This is a great story. Not only were the engineers brave enough to defy Jobs, but also Jobs being able to admit his own mistake and not fire the entire team upon finding out. Signs of true leadership all round.

<comment><USER6> You know what would have been better leadership? Not making the wrong decision in the first place. Yeah, obviously, backing down gracefully when proved wrong is quite important, but so is not betting the product on being able to make a new component...

<comment><USER7> "Never make mistakes" isn't exactly a workable business plan.

<comment><USER6> I never claimed it was. But all other things being equal, it's better to be wrong less often, to make fewer mistakes. And when you do make mistakes, it's better to gracefully correct. But in any given case, it really is better to not make the mistake at all.

<comment><USER8> Of course it's better to be perfect than not. You've stated a tautology.

<comment><USER6> A tautology means it's always true. That a curious criterion to use as a criticism. All statements are either tautologies, or sometimes false (or are not meant to have truth values, of course).

It is what should be an obvious tautology. Criticizing it for its obviousness makes sense -- if it were that obvious it would add nothing to the discussion. Yet for some reason I did feel the need to remind people of this.

Acknowledging that one was wrong when at last forced to by circumstances is not a particular trait of great leaders. It is a bare minimum. Acknowledging and correcting as circumstances suggest better options, but before circumstances thus force one is a sign of great leadership. Having backup plans ready in case your first choice doesn't work out is another. (It's true that many leaders do none of these. Leaders that do only the first are better than leaders that do none. But it's still not a sign of great leadership.)

<comment><USER8> > Criticizing it for its obviousness makes sense

That was my intention. I was perhaps unclear.

</comment></comment></comment></comment></comment></comment></comment><comment><USER9> Anyone who wants to read a fascinating insight into the early days of personal computer development should read this entire site - it evokes wonderful memories for me of how exciting it all was in those days, and how a few guys can develop a historical product all on their own.

<comment><USER10> Alternatively, it exists in a nice, convenient and well-illustrated dead tree version, "Revolution in the Valley".

<comment><USER11> Do you mean "Fire in the Valley", or is it a different book?

<comment><USER10> No the title is "Revolution in the valley", subtitled "the insanely great story of how the Mac was made": [http://www.amazon.com/Revolution-Valley- Insanely-Great-Story...](http://www.amazon.com/Revolution-Valley-Insanely- Great-Story/dp/0596007191)

</comment></comment></comment></comment><comment><USER12> The poor guy's name couldn't really be Hide Kamoto?

<comment><USER13> Why not? Although Hide was asked to hide in a closet, those words are not pronounced the same way and Hide is a common Japanese given name.

</comment></comment><comment><USER14> Brave enough to defy Steve Jobs! And y'all said it was suicide. Refreshing!

</comment><comment><USER15> So, Steve Jobs wasn't always above secretly copying another company's product the way he claims to be now.

"Cupertino, start your photocopiers." ([http://www.macnews.com/content/wwdc- apple-blasts-microsofts-...](http://www.macnews.com/content/wwdc-apple-blasts- microsofts-longhorn))</comment>


Ask HN: How do your team write documentation? - <AUTHOR> Hello,
I have got questions regarding documentation writing within a team of software engineers.
Some context:
I am a new project leader in the computing department of a medium-sized company. My team is 6 software engineers. We are building a new product, and I cannot get the team to write documentation - we need both an user guide and API documentation too. It is seriously lagging behind, for 3 main reasons I guess:
* the product is a moving target, API is not 100% stable, no one wants to start something and have to redo it later
* everybody in the team prefers to write new features and code than documentation
* the release date seems "far away" (in 18 months)
We follow a kind of scrum methodology, but it is not enough to have documentation stories to get nice documentation ;)
So here are my questions:
* how do you deal with developers writing documentation ?
* are there some techniques that help ?
* which tools do you use ? I am thinking of gitbook for user guide, and Sphynx for API documentation
* is it a good idea to hire a "technical writer" to have a specialist writing documentation instead of software engineers ?
* how does it work for open-source projects ? Documentation is sometimes really great, I would like to know the recipe :)
Any help would be appreciated !
Thanks in advance. <comment><USER2> > * how do you deal with developers writing documentation ?

I do like writing documentation that I will use later myself. I dislike having to dig through the code of every tool I wrote just to check how it behaves, so I constantly reach for man pages and, in their absence, for HTML.

I'm in a happy position that all of my tools are later consumed by either programmers or sysadmins, so I'm certainly my own audience.

> * are there some techniques that help ?

A two-part strategy works for me for API docs: (a) use documentation generator (a JavaDoc descendant, like Doxygen, EDoc, or Sphinx), and (b) add docucomments to every function before even writing function's body.

After the (b) part is in place, once the programmer sees that everything else around has the documentation, he wouldn't want his part to stand out by looking raw and plain; or at least that's how it works with me.

For documentation about the tool usage, I use anything that fits with the project's workflow that generates TROFF (man pages). Sometimes it's Perl's POD, sometimes it's Sphinx.

> * which tools do you use ? I am thinking of gitbook for user guide, and > Sphynx for API documentation

Sphinx, EDoc, or whatever I have at hand in the language of the project.

Sphinx can also be used for user guide. I don't know much about gitbook, but given how similar in their essence are all documentation generators (JavaDoc- like ones), it shouldn't matter that much if you went with gitbook, Sphinx, raw LaTeX, or a set of Markdown compiled to HTML. Choosing the tool is not the difficult part.

</comment><comment><USER3> This seems like a workflow issue to me.

Writing documentation should come BEFORE writing code. The developer should know the feature that they are going to implement and how they plan to implement it before they open the IDE. I've found this keeps me from coding myself into a corner.

Obviously some things might change, and the documentation will need to be updated while implementing the feature, but then it is just incremental changes to already existing documentation.

It shouldn't be a matter of "getting" developers to write documentation.. its part of the job.

As for a User Guide, I would think this would be the responsibility of the product owner, or whoever is driving the requirements around what the application is supposed to do. The developers shouldn't be deciding what the application does.. just how it does it.

<comment><AUTHOR> Thanks for your reply. I have another question though : how would you do if there were already a lot of undocumented code ? Would you organise a "documentation retreat" to catch up then doing as you say, documenting before coding ?

<comment><USER3> Paying down the technical debt is a tricky one, especially if your application is large. If you have the cash, it might be worth it to bite the bullet and hire a consultant/freelance technical writer. Get them to build up the documentation that you are missing, and then make sure your workflow enforces that it stays up to date.

</comment></comment></comment><comment><USER4> My team at work (Microsoft) uses a custom document renderer based on markdown. So to enforce good documentation we have two main safeguards. The first is that we have a static analysis tool that forces everyone to have properly formatted comments on functions/classes/fields or else the code won't even build. The second is more informal but we have a requirement on our team that every new feature requires a corresponding design doc and that when the feature is checked in the document gets checked in with it. Subsequent changes that change the API are also required to change the design doc in order to pass code review.

<comment><AUTHOR> Thanks for your reply. So even at Microsoft developers have to write documentation themselves ? There are no technical writers to help ? I do not know how to catch up with the documentation now there are tons of undocumented code - your ideas are great for ensuring the situation won't reproduce but I need to catch up too. Thanks.

<comment><USER4> So for public facing documentation on an API there are technical writers (like for support docs) but for internal documentation we are expected to keep code well documented. Yeah it might be good to just start having these safeguards and as the code base churns it will gradually get fixed.</comment></comment></comment>


Elon Musk gets personal about his 'terrible lows' and 'unrelenting stress' - <AUTHOR> https://www.cnbc.com/2017/07/31/elon-musk-is-bipolar-has-terrible-lows-and-unrelenting-stress.html <comment><USER2> Poor man. It's almost a fraction as bad as the stress people face working for him!</comment>


Here's the Vision of iPad Gaming Greatness Apple Overlooked - <AUTHOR> http://www.fastcompany.com/blog/kevin-ohannessian/not-quite-conversation/can-ipad-achieve-gaming-greatness-or-did-apple-miss-bo

<comment><USER2> The problem with these kinds of scenarios is that they rarely pan out. Most of the time you just don't have access to 3 iPhones/iPod Touches and an iPad. It's too rare and usually not well-known enough to be worth implementing as a major feature: just a gimmick.</comment>


A No-Loss Lottery - <AUTHOR> https://www.pooltogether.us/how-it-works <comment><USER2> So, much like a Premium Bond, then. These have been issued by the UK government since 1956, using an "electronic random number generator" that originally used vacuum tubes as a source of randomness...

[https://en.m.wikipedia.org/wiki/Premium_Bond](https://en.m.wikipedia.org/wiki/Premium_Bond)

</comment><comment><USER3> If lending money on compound.finance is a 100% safe and guaranteed way to make money, why bother joining the pool? Why not just directly lend your own money yourself and get 100% of the profits 100% of the time?

</comment><comment><USER4> This is very interesting. Seems like a good way to get people to save.

<comment><USER5> Some banks even offer savings account that do the same thing - [https://en.wikipedia.org/wiki/Prize- linked_savings_account](https://en.wikipedia.org/wiki/Prize- linked_savings_account)

</comment></comment><comment><USER6> The crypto articles are coming in fast.</comment>


Any other HNers in Tokyo this week? - <AUTHOR>

A bunch of techies, investors and analysts are going on a tour of East Asia this week (http://www.geeksonaplane.com/) and they're having quite a few events in Tokyo early this week. Just wanted to see if there are any other HN folks that are attending and have a chance to meet / grab a drink if you are.<p>Leave a note here if you are and we can figure out a way to meet up. <comment><USER2> I was curious and tried browsing geeksonaplane.com, but couldn't really figure out what this is about. Does it cost money, if yes how much? Do I need to be awesome/famous/rich or can anyone join? There isn't enough information to actually join this.

<comment><AUTHOR> They have an event during the day tomorrow that is free for anyone to join. It's 7000 yen for the day. I'm not rich, famous or awesome, but I'm going.

It's basically a bunch of well-known geeks and investors from the US and other parts of Asia. I saw Dave McClure speak here a few months back and he had a lot of great information for young startups.

</comment></comment><comment><AUTHOR> [a whippoorwill sounds, a tumbleweed rolls by]</comment>