If you want to improve your product’s documentation — or the whole user experience — there’s a tried and true technique: do a survey. At least that’s what we’ve always been told.
Let me tell you a couple of stories.
Earlier this week the British government, in the person of Science Minister Jo Johnson, announced that its new research vessel will not be christened Boaty McBoatface, even though that name won an Internet poll with 4 times as many votes as the runner-up.
Evoking memories of Graham Chapman’s Colonel, Johnson declared that the winning name was simply too silly and that a more “suitable” name will be chosen instead.
Much the same thing happened in 2006 when Stephen Colbert, in his Comedy Central days, encouraged his viewers to vote in an online contest to name a bridge in Hungary.
Stephen Colbert Bridge won, garnering more votes than there are people in Hungary. Things hit a snag when Hungary’s ambassador to the U.S. good-naturedly informed Colbert that in order to be honored, he would need to be (a) fluent in Hungarian and (b) dead.
Today the bridge is known as Megyeri Bridge because it connects two towns whose names end in -megyer. I’m not sure that’s better than Colbert Bridge. But I’m not Hungarian so I guess it’s none of my business.
The moral of both stories? Surveys and polls can be entertaining. But their results aren’t always useful.
Now I know that nobody is going to turn your customer survey into a prank. Still, when you ask your customers what they want, they don’t always know. Their responses likely will be knee-jerk, not reflective of careful thought.
Want a better index? Sure, that sounds good. Bigger icons? Why not? Soon you’ve got a lot of “results” that you can turn into action plans. Yet you’ve missed the issues that truly affect the UX.
The solution? Don’t ask your customers what they want. Instead, ask them how they actually use the product, and ask them what things give them trouble. Do they have difficulty finding the instructions they need? Are the instructions relevant to their work situations? Are there product features that go unused because they’re hard to set up and maintain?
When you ask your customers how they really use your product, then you can use your own know-how to decide how best to make their lives easier.
There’s an even better way, although it’s harder than administering a survey. If you can observe your customers at work, if you can see for yourself where they succeed and where they struggle, then you’ll know exactly where to focus your efforts at improving both the documentation and the rest of the product.
So there you have it.
Surveys that ask customers what they want: too silly.
Surveys that measure the way customers actually use the product: much better.
In-person observation (including usability tests): harder, but best of all.
Tell me about experiences you’ve had improving your products by gathering information from your customers.