I am conducting usability tests internally and consistently a design choice keeps coming up as confusing. The tripping point is toggleable tags and whether they should look disabled when not toggled(current design) vs tags only present when applied (perhaps more clear but tags do not align.

I care about alignment in the tag structure to make scanning easier, in addition to distinctive colors.

Simply put, should I change this design choice now in favor of preventing remaining testers from getting tripped up on something we will likely address?

  • Earlier today I was sharing my screen for a UI demo and noticed a bug so I talked about other stuff while I changed the code on my other monitor and then refreshed the page :)
    – DaveAlger
    Dec 10 '14 at 1:28

Although I wouldn't change a prototype based purely on the feedback from the first tester, I would definitely and fearlessly make such a change if I got a majority of the first 4-5 testers revealed a problem. ...and I would stop wasting testers on the old design until that change was complete. Here is why...

In UX, Ignorance is precious. Every user only has one first impression for a given prototype. Once they have used it up, they will never again approach that prototype as a complete innocent. Informed feedback gathered from experienced users who have had time to digest and consider their responses to your design are practically worthless compared to the in-the-moment response of a new user, seeing it for the first time.

If you have an infinite testing budget, then go ahead and waste your tester's ignorance; you can always buy more testers. If, on the other hand, you live in the real world with the rest of us, then horde your first impressions like a miser hording gold coins. Spend them as slowly as possible because you probably won't get any more, once they are gone.

There is a spectrum to appropriate response to tester feedback. Redesign everything that each tester stumbles over and you will never finish the project. Wait to redesign until all of your testers have gotten a look at the old design, and their treasured ignorance will be gone forever.

  • 1
    Agreed. Fixing glaring issues ASAP also means your remaining testers won't get tripped up on the same issue. This means they have more mental energy to focus on other potential issues your first users haven't have the time to express.
    – nightning
    Dec 9 '14 at 17:17
  • Thanks for the insights - the perspective of seeing first impressions as a resource was useful! Dec 9 '14 at 17:31

It depends on how much evidence you need to change the design decision. Gather feedback until you have enough evidence. In this case, if it's a pure interaction design decision that you have a lot of control over, yes, change it right away. If it was a decision that has more to do with branding or the product itself, you might feel like you will only get momentum outside your team to change it if you can show a huge percentage people tripping it over it. Showing how and why the users were tripping over it and seeing where their reasoning is might be very useful for improving the product/branding/whatever. It's a judgement call.

Your Answer

