UI Design Newsletter – November, 2007

In This Issue:

Meta-Usability:
When the method is not the message

HFI's Chief Scientist, Kath Straub, PhD, CUA, looks at the gap between what researchers study and what practitioners want – and the gap between what practitioners present and what clients want to hear.

The Pragmatic Ergonomist

Dr. Eric Schaffer, Ph.D., CPE, founder and CEO of HFI offers practical advice.

Into the abyss...

Every year about this time we update HFI's Putting Research Into Practice course. And every year about this time we find ourselves faced with the same challenge – how can we effectively bridge the gap between what researchers study and what practitioners want to know.

Actually it's not just a gap that separates what (most) researchers study at and what most practitioners want...it's more like an abyss.

Practitioners, like users, want it to be easy.

Practitioners want research to solve a problem, or justify a specific design or business decision.

• Where should this button go?
• What research method should I use?
• What color should this box be?
• What is the best label for...?
• How many participants do I need to test... really?
• What should I read / study / learn to become a better practitioner?

The most ruthless practitioners think of research as useful when it is essentially a set of bake-offs.

Researchers tend to think more broadly, focusing on abstract questions that seek to describe the system and the variables that work within it. The answers to questions that researchers explore is often neither immediate nor concrete:

• How can I extend the GOMS model to accommodate... X?
• What is the role of personality in risk assessment that influences likelihood to convert?

There is a necessary connection between theory and practice. But there is also a difference between the two. And that difference, as van de Snepscheut said, is larger in practice than it is in theory.

Details, details

To be fair, research papers are not really written for practitioners. They are peer-to-peer conversations amongst researchers. Researchers and practitioners have different information needs and goals. And this mismatch makes the literature essentially unusable for practitioners. Details that are keenly relevant to researchers are just details to practitioners. There is too much work involved to make the leap from research to reality. Even industry leaders indicate that research is often unengaging – "I would probably rather curl up with a good detective story than with a bad research paper. OK, I would rather read any detective story than nearly any research paper." (Jarrett, Journal of Usability Studies, 2007)

It seems fair to say that researchers and practitioners are different user groups with different tasks and different objectives. Researchers want to know (and, as such, their papers convey) the logic driving to the method and the analysis driving to the conclusion. Practitioners just want to know the conclusion and how it solves a problem in their world. Today.

Take a step back.
Have you ever sat through a usability testing presentation where the first third of the presentation is spent (re)selling usability and setting up the project, covering the details of the methods, data collection approaches? Where the first actual findings come 20 minutes into the presentation? Where the design recommendations come later? And where the prioritized solutions are on the last slide? Kinda ironic, don'tcha think...?

Practitioner, hear thyself

Cooke and Mings (2005) observed this gap between the approach and focus that practitioners have, and the answers that the industry wants and needs. To gain an understanding of how to close the gap, they conducted in-depth interviews with (non-usability) professionals involved in product design, development and documentation lifecycles at a major software organization.

As should be expected, they found that individuals who focus on usability tend to focus on... details of usability. While the details are important, and knowledge of (benefits of) methods are core skills, conveying the findings – better yet, the solution – rather than the analysis leading to the selection of a method is really what clients are looking for. Based on their research, the authors identified key skills that drive UX success in the "real world."

• Effective practitioners need to be able to identify, diagnose and execute a correct solution quickly.
• Practitioners need to know which methods are out there and when to apply which, and just do it.
• Practitioners should be able to articulate the tradeoffs for favoring a specific method based on the business perspective. This means balancing the quality of the data against constraints like:
  • How much will the various approaches cost to deploy?
  • Which is fastest given our current state?
  • Which is the most powerful but least expensive data we can get to make the case (e.g., quantitative, qualitative)
• Practitioners need to provide concrete recommendations, not problems and findings. And the earlier the recommendations come, the better.
• Conversely, practitioners need to set up and provide longitudinal metrics on how a product or interface fares over subsequent updates and releases.

What your listeners want to hear...?

Finally, Ming and Cooke note that practitioners need to sell in business terms. They suggest that practitioners (and training programs) should hone negotiation and audience monitoring skills, and the ability to recognize when there is a gap between what is being presented and what the audience is seeking. After all, the business wants answers: What do we do? How to do it? What is the projected business impact (ROI) of making such changes? They aren't always as interested in the method and data that lead up to that solution.

Such a cut-past-the-chase approach can be difficult for practitioners. The chase is the fun part, right? And, the answers the business guys want ARE in the presentations.

But from business guys' perspective, there are all those other details to wade (wait?) through... Why that method?... What are the logical steps for driving from data to recommendation... Presentations seem designed for like-minded peers, not the intended end users.

Sound familiar? Back to reading research papers, so we can give you what you want...

Our details are dangerous. They can lead us to focus on design issues, while missing the large strategic issues where user-experience analysis can make the greatest contribution. Our details can also make our presentations lose the interest of our executive sponsors and clients.

But as professionals we MUST keep up on the literature. We need the research to guide us. We also need to provide the citations that quickly and cheaply prove our point. So Kath, back to your work on the 2008 Putting Research into Practice course.

References

Cooke, L. and Mings, S., (2005). Connecting Usability Education and Research With Industry Needs and Practices, IEEE Transactions on Professional Communications, Vol. 48, No. 3.