Usability and testing is a great way to learn how users are responding to your application, but it’s rarely quick and easy to do. Here are a few examples of testing options that can make a big difference to the user experience:
Testing a Concept
There’s an old Foxtrot comic strip where Jason announces “I’m going to invent shoes with wheels on them.” His sister Paige asks dryly, “will you call them roller skates?” Concept testing gets at far more than just double checking that no one else has come up with the same one. I’ve worked with a hospital where concept testing showed us the logo resembled a cigarette, and with a mobile development company that found through concept testing that their site’s messaging around “creation” made them sound like house painters, rather than developers.
Testing a concept involves first having one or more concepts to test: messages, images, or key words. I like to create a maximum of 8 messages to test, and then put them online, against a white background with no context. I can send people to see the various URLs from social media, and easily get a large number of comments on each concept.
For some projects it’s better to show all of the concepts to each user, though in most situations you want users to only see one concept (to decrease the likelihood that seeing one concept will influence their thoughts on another).
Paper prototypes (i.e. sketching screens onto paper, and allowing a tester to “click” on buttons with a finger, then showing them the next page/screen) get a lot of flack from testers saying that their users couldn’t get a sense for the application’s actual usability. They’re right – paper prototypes aren’t intended for testing ease of use. They are, however, an excellent way of testing messaging and flow.
The key to testing a paper prototype is preparation. If I have created an ecommerce flow, I need to sketch a screen for every page a user might try to get to. I need to make sure my paper screens have clearly identified buttons and links, and I need to get familiar enough with the variations within the flow to be able to easily hand the user his next “screen” based on what button or link he selects.
Most tests work best with one or more observers taking notes, so that the moderator can be free to ask questions without worrying about writing down answers. For a paper prototype, this is even more important. The moderator will be responsible for “changing” screens, meaning he or she will certainly not have time to make notes on when the user was surprised or frustrated or confused.
Tree tests are quick to put together and valuable for getting insight into both a user’s vocabulary and the viability of the IA.
In a tree test, the user is asked to find information on a topic, and then is shown the main navigation of a site (as they would when they saw the top nav bar). This is typically done with software such as User Zoom or Treejack, which allows the user to then click on a navigation item and see what pages or subnav appears. When the user is satisfied that he or she has found the section or page that would have information on the requested topic, they click “submit.”
For observers, this is a great way to identify user expectations.
We all know that words have power, but some have different connotations to different groups of people. With this in mind, picking key words from the taxonomy and asking target audience members to define or react to them is a great quick way to explore the audience’s vocabulary.
Ideally, a word association test should not ask about more than 10 words. More than that will begin to influence the way the user is hearing the terms.
One piece of advice: given the variety of connotations for any given word, word associations are only useful if they’re coming from the target audience.
Where most user testing focuses on exploring the user’s mindset, surveys are often less open ended, simply because they are not in real-time, and don’t provide opportunities to ask follow up questions. However, surveys are a great way to identify goals, expectations, frustrations, needs, and fears.
For example, if a project has a discovery period, and we want to identify the three biggest problems to address in a software system, we can accomplish that by asking all 300 employees to respond to an online survey much faster than if we were to interview even 30 of them.