|
A Pair of Techniques for Effective Interface Evaluation:
Cognitive Walkthroughs and Think-Aloud Evaluations, Beer, T., Anodenko,
T. and Sears, A., Proceedings of the Human Factors and Ergonomics Society
41st Annual Meeting, 380-384, (1997).
|
What is a good combination of usability testing methods?
Usability testing early on in the development life cycle attempts to
find severe problems that can be fixed as soon as possible. To be most
successful, it is important to apply the most effective evaluation techniques
at each stage. No single usability test seems to be universally applicable.
There is little published advice on when to use which tests, and which
techniques work best together. Beer, Anodenko and Sears suggested the
effective use of two usability testing methods: cognitive walkthroughs
and think-aloud evaluations.
Cognitive walkthroughs require evaluators to step through the actions
users would perform to accomplish tasks while looking for usability problems.
The technique originally focused on learnability issues and was considered
difficult to apply. The technique has been modified over the years to
be more usable by individuals with limited formal training in user interface
evaluation.
Think-aloud evaluations involve having users attempt to perform specific
tasks while thinking out loud. The primary goal of these evaluations is
to discover what usability problems exist. Preparing for think-aloud evaluations
involves recruiting and scheduling representative users, and developing
a set of tasks for the users to perform.
This article suggests the combining of the two techniques. Designers
should do cognitive walkthroughs followed by think-aloud evaluations.
These work well together because task descriptions developed for the cognitive
walkthroughs can be converted directly into the task descriptions needed
for think-aloud evaluations.
A study was conducted to evaluate the effectiveness of using these two
techniques. They found that using cognitive walkthroughs early-on allowed
the most serious problems to be identified and corrected. This was done
without the need to use representative users. Later, think-aloud evaluations
were used to find the remaining problems.
|
|
Cognitive Modeling Reveals Menu Search is Both
Random and Systematic, Hornof, A.J. and Kieras, D.E., Proceedings
of CHI ‘97, 107-114, (1997).
|
How can we substantially improve the user interface design process?
One of the most exciting advances in the usability field is the creation
of software that will simulate the behavior of typical users. Eventually,
this will allow designers to conduct initial, and very effective usability
tests, without requiring real users to perform the tasks. One of the most
promising new methods is a system named Executive Process Interactive
Control (EPIC).
EPIC
- has a programmable production-rule cognitive processor,
- incorporates constraints from human performance research, and
- generates a specific sequence of perceptual, cognitive and motor
activities required to perform each specific instance of the task.
To use the software, designers must specify a cognitive strategy and
perceptual-motor processing parameters.
To ensure that the system behaves in a manner similar to actual users,
many studies are being conducted to determine exactly how users respond
in specific situations. In this study, Hornof and Kieras attempted to
determine how users search for a known item in an unfamiliar menu. They
made adjustments to their software until the new system responded to a
menu selection task in exactly the same way as typical users.
This and other studies should eventually lead to a general purpose computer-based
tool for evaluating the efficiency of user interfaces. User interface
designers would provide a definition of a screen layout and a task, and
the system would predict the time required for users to execute the task.
The development of formal evaluation methods, such as EPIC, has tremendous
potential for significantly improving the quality of user interfaces.
|
