Developing and Testing Content

Usability research helps organizations understand user needs, identify potential issues, and generate ideas for improvement. While usability testing is often used to evaluate a website’s user interface (UI), this method is also invaluable for discovering the best way to present information on your website. By paying attention to how people read, interpret, and access content, you gain a greater understanding of how to communicate, structure, and format information.

It’s great when sites have good navigation. But too often we see the user experience fail at the content level: People can navigate to the content but don’t understand it. Analysis shows that people often use websites to collect, compare, and choose products or services. Have users evaluate your digital copy so that articles and information match their needs and expectations. People read online content differently than printed material.

The usability study methodologies for evaluating UI versus content are fairly similar. However, there are nuances to the methodologies that are worth considering when the primary goal of the usability study is evaluating digital copy.

Below are suggestions for how to get the most out of your research.

Tips for Testing Content on Websites

1. Avoid recruiting proxy users: In every usability study, you should always aim to test your designs with representative users. However, when testing content, your recruiting criteria should be even more stringent. Take extra care to recruit the right participants.

   Those people evaluating the information on your site should truly be representative of your user population: they should have the same mindset, situation, AND user goals. The flexibility you have with recruitment depends on the use case and type of information on your site. You may have some leeway with general e-commerce sites, but for content-rich, research-intensive activities or for B2B websites, you must find people who fit the exact circumstance.

   In other words, the scenario that you give people should match the current problem they need to solve. Unlike regular UI-focused studies, content-focused studies should not ask test participants to “pretend” or “imagine” to be in a situation. The risk of invalidating the study is much higher for content because the participants’ motivation is much more important for obtaining accurate insights.

   It is impossible for proxy users to instantly acquire knowledge or know the situation well enough to assess the value of the content. For example, people who have just been diagnosed with a serious medical condition are more likely to relate to the content accurately than someone who is asked to pretend to be interested about a disease.

   It’s not good enough to recruit participants who generally fit the demographic profile, such as by age, gender, income level, and location. Such criteria are too broad to give you deep insight. General recruitment criteria won’t cut it. You must find people who are actually in the process of researching the information you are evaluating.

2. Be aware of the limitations of unmoderated studies: Unmoderated studies are done without the facilitator present: Participants work on their own. This method can be useful for getting user feedback on narrow parts of the site such as workflow or snippets of information. However, when trying to discover how people conduct research, compare offerings, and make decisions, the best approach is to conduct a moderated study, where the facilitator is present.

   Content studies tend to have long stretches of time when the user is simply scanning page after page—in silence. When left alone (such as in an online unmoderated situation) users may feel awkward and wonder whether they’re being helpful. Without proper feedback and reassurance, participants often alter their behavior by approaching the task in a more superficial manner. Task times are often shorter for online studies than in traditional test settings. When on their own, participants assume that the goal is to work quickly, not realistically.

   Also, the facilitator can ask the user for clarifications. With unmoderated studies, you miss opportunities to ask personalized, user-tailored follow-up questions. Even though participants are instructed to think out loud, they often forget to explain their actions and thoughts.

3. Give tasks that are tailored for each individual: In most traditional usability studies, researchers follow a prepared script and give study participants prescripted tasks to perform. For content testing minimize your reliance on a script. Spend time at the beginning of each session to discuss the participant’s situation and make sure the task scenario matches their exact circumstance. It’s OK to prepare some more generic tasks prior to the study, but be willing to modify or craft new ones on the spot as you learn more about the participant’s situation, and as the session unfolds. You want to give participants the freedom to research a topic as they please, so you uncover what’s important and what’s not. Don’t rigidly control the activities or force an unrealistic task. The more pertinent the tasks, the more vested people are at completing them.

   The best results occur when study participants forget about the testing environment and immerse themselves in the activity rather than merely going through the motions. Participants can sometimes “fake” their way through simple pass or fail activities (e.g. Find the contact name for Press Relations), but such is not the case for exploratory tasks where having a scenario that precisely matches the person’s current situation and emotional state is critical.

4. Remember, there is no right answer: Unlike well-specified tasks (e.g., “Find the opening hours for the Fremont public library”), open-ended tasks don’t have a definitive answer. Open-ended tasks are meant to assess content quality and relevance. Use this time to learn how people explore and research, what questions they have, how they expect information to be communicated, and whether your site meets their needs.

   Consider competitive testing: Sometimes you can get insights into your users’ needs by allowing them to search freely on the web or by letting them visit competitors’ sites rather than restricting them to your own site. Don’t worry that you’re wasting precious testing time: if users are truly representative, the insights will often be revelatory. And you can always limit the free exploration to a small part of your session.

5. Set expectations for time allocations: Open-ended tasks have vague end points, often leaving participants wondering how to best spend their time. At the beginning of reach session, tell people to work at their own pace and not to worry about the time.
6. Get comfortable with silence: Expect long stretches of quiet time while the participant focuses on processing the information. Don’t appear impatient. Avoid being interruptive or fidgety. Injecting too many questions while users work breaks their concentration and alters their behavior. If you need to ask a question mid-task, keep it neutral, such as “What are you thinking?” or “What are you looking for?” Once users answer, let them continue. Resist the temptation to blast questions. Save questions for the end. When user testing is conducted well, users behave authentically and the study generates realistic findings.