UI Design Newsletter – May, 2005

Insights from Human Factors International

In This Issue:

When what they see
is what you get –
but satisficing
isn't enough

Kath Straub, Ph.D., CUA, Chief Scientist, looks at the importance of effective detailed design.

The Pragmatic Ergonomist

Dr. Eric Schaffer, Ph.D., CPE, founder and CEO of HFI offers practical advice.

satisfice: verb: decide on and pursue a course of action satisfying the minimum requirements to achieve a goal; "optimization requires processes that are more complex than those needed to merely satisfice"

Benign neglect isn't always benign

We know that good design can change behavior. People impulsively buy more Snickers bars when marketers put them right next to the checkout line.

But is the reverse equally predictive? That is, if we hide the Snickers bars (either intentionally or otherwise) will fewer candy bars get eaten? What happens when designers ignore (or worse, are innocently ignorant of) users tendencies to interact in predictable ways?

When designers are not attentive to subtle design issues, users trip up. Inattentive design feels frustrating because it forces us to try to actively avoid or suppress behaviors that we are hard-wired to do. It's easy to demonstrate that bad visual design undermines task completion. Think about a task that requires you to read and compare data across rows. Now consider what will happen if a well-meaning but inattentive designer uses color to delineate the columns.

Inattentive information design has detrimental effects too. Designers need to consider the differences between viewing content on paper and interacting with content on the Web. Otherwise they force us to wade through and sort out too much information presented too fast in chunks that are too big or organized so that mapping the information to our mental model takes work. This may ultimately affect how deeply users process and encode information.

Why are survey studies always about breakfast food?

Does benign neglect of detailed design go as far as changing behavior? Can design (or inattention to it) influence the decisions users make and the way that they interact within the information space?

Couper, Tourangeau, Conrad and Crawford (2004) propose that it can. Their studies compare the application of what we know about user interactions with paper studies to user interaction with Web surveys.

Web surveys present significant and interesting improvements in survey data collection. We can use automation to present contingent questions only when they are appropriate. We can randomize choices within questions to minimize the primacy effects (found in paper surveys) and recency effects (found on telephone or verbally presented surveys). The Web also presents interactive opportunities that are not available in the paper survey world. But just because we can doesn't always mean we should...

To understand how what we know about paper surveys translates to Web environments, Couper and colleagues have conducted a series of survey studies that demonstrate that choice presentation of interactive surveys can influence the response selection of participants.

In one of their many studies they compared users' response selection when identical forced-choice questions were presented different ways. The different methods of presentation varied both how much information users see from the offset and how much work they need to do to answer the question (i.e. find the snickers bar).

In the Radio-Button condition users were presented all the choices right up front, with a radio button selection interaction, as in Figure 1. In this case, users simply scan the options and indicate the one they select with a single click.

Figure 1. Radio-Button condition: all choices visible

In a Click-to-See-Most condition users were presented the same question, but with a pulldown selection widget. Here respondents need to click to see any of the options. Note that this question style, as is presented in Figure 2, also requires users to also scroll if they are to see the last item in the choice set.

Figure 2: Click-to-See-Most condition: drop box with most
items visible (after clicking), scrolling required to see all options

In the Window-to-Five condition, the selection choices were presented via a bounded window. The window presented only five options at a time, so in this condition respondents were required to scroll to see more than half of the choice set.

Figure 3: Window-to-Five condition: choice window with
5 items visible, scrolling required to see many items

What they saw...

The findings are interesting.

First they note that respondents tended to pick one of the first options in the first set that they saw. So in the Radio-Button condition (Figure 1) and the Click-to-See-Most (Figure 2), they tended to choose items from throughout the list. By contrast, in the Window-to-Five condition (Figure 3) they tended to pick one of the first five options. This is not surprising. It replicates what we know from paper survey research.

And it's because we satisfice, right? That is, we only go as far as necessary to pick a good answer. We go with that one and move on. Right?

Not really. Couper and colleagues' evidence suggests that we tend to pick one of the first items we see. However, their study is designed well enough to explore why that is happening. This team tucked the "None of the above" option at the bottom of the scroll in both hidden choice conditions. This clever experimental design provided a route to determine whether respondents were satisficing. If they were, we should see more "None of the above" responses for the radio button presentation (Condition 1), since it takes the least work to see that response choice in this condition..

Interestingly, Couper and colleagues' results show clearly that participants did not satisfice. Specifically, the distribution of "None of the above" responses did not differ significantly across the question presentation types.

When satisficing isn't enough

Couper, et. al. have seemingly conflicting results. On the one hand, respondents tended to pick one of the first choices they saw. On the other hand, they are not satisficing.

This team uses a cognitive load explanation to describe the participant behavior. When respondents evaluate a sequence of choices in a selection environment, a few things happen:

1. Respondents spend more time thinking about the first options than they spend thinking about later ones
2. Their thinking about the first options is clearer. After all, they are juggling less comparisons early in the selection process
3. Initial options accumulate more and different evidence with each subsequent comparison. This makes an initial option harder to dislodge against later comparisons.

Put simply, users tend to spend more effort thinking about the first options so they tend to pick them more. The items presented first get a jump start.

Note that this explanation is very different from satisficing. Satisficing suggests that respondents pick early choices because they simply lack the motivation to examine the whole choice set.

Couper and team find that this effect is exacerbated by presentation. Users picked items from the first visible set most often in the Window-to-Five condition.

Do no harm

This study (and others in the works) clearly demonstrate that presentation design not only can, but does influence respondents' choice behavior. The choice of response format in Web surveys can influence the response distribution.

If a respondent is picking a known response from a long list (e.g., their state or salutation title), dropdowns may be fine. However, when the respondent is comparing selection options, hiding data options can shift response patterns. In this case, the behavioral tendency of designers to use dropdowns to save space can be problematic.

We have solid evidence that we need to think hard about the interactions we design... RIGHT DOWN TO THE DETAILS OF THE WIDGET LEVEL! Our often casual selection of a control can influence decision making. So we can never again make that selection casual. Sure, the study shows surveys can be biased. But what about ballots... or medical diagnostics?

I have often seen professionals design the high-level navigation for a site, and then leave the detailed page designs to designers without real usability engineering capabilities. If you are one of those... you need to rethink that.

References

Couper, M.P., Tourangeau, R., Conrad, F.G. (2004). What They See is What We Get - Response Objects for Web Surveys, Social Science Computer Review 22 (1) pp. 111-127.