Testing: "What, Me Worry?" (continued)

GETTING USER FEEDBACK What good is testing? Testing provides you feedback. Feedback lets you know how well you're doing your job. We all ask "How am I doing?" in one way or another; in sports, in relationships, in personal growth. Usability testing tells us how we're doing on our design work. In our seminar Practical Usability Testing, we cover many points where usability testing counts during the development life cycle. And some (not all) of these points need user preference measures. We'll cover them now, starting with the most general observation.

Marketplace Preferences In our examples above, we assumed that performance was primary, and preference a secondary criterion of success. For example, in corporate systems, you have a captive audience--users who must use the software to do their job. Luckily, most users willingly share your design goal of speed, accuracy, and ease of learning. Together, these performance goals contribute to user "performance." Note that users cannot assert "preference" to reject a system that is fast, accurate, and easy to learn. Otherwise they would appear impractical to their managers. Also, you can explain the design tradeoffs to these corporate users and mollify their preference concerns.

But what if your application must be sold to individual users? What is the criterion of satisfaction in the marketplace? It's clearly a different situation than we have with a captive audience. The design must captivate attention and earn user allegiance by any legal means. Clearly, we now must appeal to preference more than performance. As the adage goes, "An unsold application is unusable!"

In such cases, collect both performance data and preference data to uncover any contradictions such as we illustrated above. Our recommendation? Use your data to enhance user preference in areas where it makes a difference in sales. Meanwhile, avoid any "killer problems" by examining the performance data as well. Improve performance while keeping your product at a high level of competitive preference. Remember, most software reviewers look at performance as well as preference qualities.

Function Testing This soul event must occur early in your development life cycle. In fact, most so-called "user testing" really is a mistaken and belated form of function testing. That is, designers often complete their application or prototype, and then see if users will find all the functions they need! This represents the junk food cryptotesting criticized early in the article. However, soul function testing doesn't need a prototype or any detailed design at all. The issues uncovered with function testing are far more broad and comprehensive, as follows:

• Not all functions are desirable or usable. Imagine a drive-up window at a fast food restaurant with featured instructions in Braille. Sound strange? We have see labels for a first aid kit near the elevators of a high-rise office building--in Braille. What did the designer have in mind? In another case, a sales company issued an on-line scheduling calendar for salespeople who were on the road without computers.
• Developers prefer different functions than users. Generally, developers (and user representatives) identify functions that exceed user requirements. Developers see complexity as a "friendly challenge." Thus, functions tend to be esoteric, complex, powerful, error prone, system related, and yes, impressive. However, typical users seek functions that are simple, practical, and common. Granted, they may sometimes be "wild," as well.

Given a list of functions, you can test their suitability with written a questionnaire or survey. Here is a list of four kinds of surveys and their suitability. Make sure you target subjects who truly represent potential users of your application. Match the range of job types. If managers and marketing people are not going to use the application, don't mingle their responses with those of "real users." You can compare data between these groups to detect any mismatch of expectations. In all cases, compare strength of user preference with the costs to develop the function. Insure that you get good bang for the buck, preference-wise. See Figures 3 to 6 for the four types of function tests.

Testing Your Task Design You got the right functions. Then you figured out the task design. Did you create a prototype yet? We hope not! Task design must precede detailed screen design. How do you know you have a suitable UI architecture--the mental model that makes your application easy to understand? How do you know you have a reasonable division into modules? How do you know whether your sequence of task events suits your users? Have you brainstormed about problems and alternatives? Your task design provides another point in the development life cycle where you get your money's worth with usability testing. And your best input comes from your user population!