Resources
Articles
- "Why You Only Need to Test with 5 Users" (Jakob Nielsen, NN/g)
- "Quantitative Studies: How Many Users to Test?" (Jakob Nielsen, NN/g)
Why five is the right number of participants for most diagnostic usability tests. Perhaps the single most valuable article ever written on usability testing.
The case for 20 participants for comparison tests.
Frequently Asked Questions
What is remote usability testing?
In remote moderated testing, participants are at a different location from the moderator and observers. By using a high-speed Internet connection and an online conferencing service, they can participate from anywhere.
For remote moderated tests, observers can view the live sessions anywhere using the same the online conferencing service. This makes remote testing particularly helpful for distributed teams. During test sessions, observers can communicate with the test moderator and other observers via instant messaging. In addition, time can be reserved at the end of each test session for observers to ask the participant additional questions.
The main difference between moderated remote tests and in-person tests is that we don't see the participant's face or gestures during the test session. In most cases, this isn’t a drawback: with skillful moderation, we are almost always able to get the necessary data by hearing the participant’s voice and viewing what the participant is doing.
Remote testing eliminates the need for participants to come to a central location for tests, which can be especially valuable for harder-to-find participants and in areas with congested highways. It also makes it possible to recruit from a larger pool of potential test participants, including participants outside areas that are unusually sophisticated about technology and the Web, such as the San Francisco Bay Area.
What are the differences between moderated and unmoderated usability tests?
Unmoderated tests can be faster and less expensive than moderated tests. You can get results for unmoderated tests in as little as a day. Arranging a moderated test and scheduling the participants typically takes at least 2-3 additional days.
In both moderated and unmoderated tests, participants are normally asked to think aloud about what they're doing and thinking. In doing this, they can provide much valuable information about what is and isn't working.
Moderated tests can provide richer, more detailed data. During moderated test sessions, the moderator can ask test participants questions to get more information. The answers can provide additional insights and help reveal underlying reasons for usability and user acceptance problems we observe.
In tests of prototypes that are not complete or that have bugs, a moderator can steer participants around problems and keep them focused on the aspects of a prototype that we want to learn about.
Why do your usability tests focus on participants' task performance?
In general, the better a test simulates what users will do with a design in real life, the more reliable the results of a test will be. Creating a good simulation starts with identifying tasks that users would do in real life. You then create a prototype with design elements to support those tasks.
When test participants attempt to perform the tasks, they encounter problems that actual users would encounter.