About HFI   Certification   Tools   Services   Training   Free Resources   Media Room  
 Site MapWe help make companies user-centric   
Human Factors International Home Page Human Factors International Home Page
Free Resources

UI Design Newsletter – August, 2002

In This Issue

Making Research-Based Design Decisions

Bob Bailey, Ph.D.,
Chief Scientist for HFI — What is the best way to get user-centered research results to practitioners?

The Pragmatic Ergonomist

Dr. Eric Schaffer, Ph.D., CUA, CPE, Founder and CEO of HFI offers practical advice.

Making Research-Based Decisions


I have reviewed the research literature in professional journals, conference proceedings, technical reports, usability test reports and books for most of my career. Each year I repeat the review process, and try to organize the relevant research into a meaningful 3-day presentation for practitioners (the Annual User Interface Update).

There are about 1,000 usability-related articles published each year. My guess is that less than 5% ever have any practical, long-term value to most usability practitioners. In some cases, the topics being studied are of little interest to practitioners. In many cases the research results are simply too hard for practitioners to find.

I have two questions to ask, one to researchers and one to practitioners. I do not think the answer to either question is obvious.

To researchers: If you conducted a series of studies that helped to answer a critical, important usability question, and you wanted to publish so that the largest number of practitioners (not just other researchers) would read your article, where would you publish?

To usability practitioners: If you seriously needed to find a recent research article on a particular topic, where would you look?

To demonstrate how difficult it can be for practitioners to find recent research results, assume that you know of an article, where it was published, and you wanted to read it. This is a far easier task than trying to find any article, or all articles, on a particular topic.

To be more concrete, assume that you want to quickly find and read an article that you know was authored by Melody Ivory and Marti Hearst and was published in the most current CHI Conference Proceedings. According to the CHI Web site, "The annual CHI conference is the leading international forum for the exchange of ideas and information about human-computer interaction (HCI)."

To make matters even easier, assume that you already have an ACM identification number, a Web account number, and a password. Many practitioners may not have these numbers. The steps for accessing the CHI article are as follows:

  • Enter the URL: acm.org (shows the home page)
  • Click: Special Interest Groups (shows a new page)
  • Click: Conference Proceedings (shows a new page)
  • Click: List of Proceedings available on ACM's Digital Library (shows a new page)
  • Click: Inside the search box (shows a dialog box that says "Search is a Controlled Feature," and requests your Web account number and password)
  • Click: "Cancel" (the dialog box disappears)
  • Click: Proceedings (shows a new page)
  • Click: CHI-Conference on Human Factors and Computing Systems (from a list of 166 items spread over 6 screenfuls)
  • Click: 2002
  • Click: Inside the search box to set the cursor, and enter the term: "Ivory" (shows the title: Website analysis: Statistical profiles of highly-rated Web sites)
  • Click: On the title (shows the abstract and references, plus a small inactive pdf icon)
  • Click: On "Click here to gain access to the Full Text" (shows a dialog box that says "Full Text is a Controlled Feature," and requests your Web account number and password)
  • Enter your Web account number and password (shows the abstract and references again, plus a small active pdf icon)
  • Click: On the pdf icon (shows: The full paper in pdf format)

To access this paper, you needed to accurately find and click on 12 links, enter the correct URL, your Web account number, your password and a search term (a total of 31 keystrokes), and do considerable scrolling on certain pages. Will most practitioners take the time to do this?

Barbara Chaparro is the editor of Usability News. This is a free Web newsletter that is produced by the Software Usability Research Laboratory (SURL) at Wichita State University. Her group uses the newsletter to report on their research. To access one of their articles, you must:

  • Enter the URL
  • Click on the desired article title

Of course, she only has to deal with 10 full articles each quarter (about 40 per year), whereas the people at CHI have to deal with about 60 usability-related articles each year.

Finding research-based usability information can be difficult for all of us. Recently, I contacted two major local universities to see if they would provide me access to their digital libraries. They both agreed to do so as long as I came to the university and used their computers and printers. Earlier this week, I found the abstract for an article that interested me in the journal, Behaviour and Technology. I had no other choice, but to purchase it online for $22. The article may or may not contain the information for which I am looking. They are sending me a paper copy by snail mail.

If practitioners are to make the best research-based decisions, they must have timely access to the research. The research that addresses and answers the most difficult design questions should be the research that has the most impact on those design decisions. However, I suspect that the research findings that are the easiest to find, eventually will have the greatest impact on design decisions. Maybe usability researchers and their publishers need to make some serious changes in how they report their findings, so that they are more readily available to practitioners

The Pragmatic Ergonomist, Dr. Eric Schaffer

The top firms and experts specializing in ergonomics spend significant time keeping track of the literature. My office has a book shelf that is nearly 20 feet high. But for practitioners working in companies it is nearly impossible to keep up to date. Attending the conferences helps (CHI, UPA, and HFES). The people there tend to be talking about the interesting results from the last year. But this method of obtaining information can be pretty unreliable. It depends too much on who you know and where you happen to be standing at the cocktail party. That is why HFI works hard to keep our courses up-to-date (so new people get the latest) and why we have our Annual Update course to consolidate the new material. Without these types of resources I think practitioners sink into using common sense and out-of-date research.


Leave a comment here

© 1996-2014 Human Factors International, Inc. All rights reserved  |  Privacy Policy  |   Follow us: