UI Design Newsletter – December, 2003

Insights from Human Factors International

In This Issue:

Key Research Findings Related to User-Centered Design (2002–2003)

The Pragmatic Ergonomist

Dr. Eric Schaffer, Ph.D., CPE, founder and CEO of HFI offers practical advice.

HFI's December newsletter reviews the findings of the research presented in our Putting Research into Practice course. In preparing this course, recent research from various disciplines (including Human Computer Interaction / Ergonomics, Cognitive & Social Psychology, Computer Science, Marketing, Economics...) that might have implications for usability professionals is systematically reviewed. The most interesting, important, and applicable papers are summarized for presentation in our 3 day seminar – essentially a "Cliff Notes" course for usability research, updated annually.

The list below differs slightly from that of previous years. Rather than presenting design "dos" and "don'ts", this year we present key findings of many of the papers presented in the 2003 PRP course. As such, in addition to providing design guidance, this list provides you recent research references to directly justify your analysis, design, and testing decisions.

By the way, if you only read one or two papers about usability this year, read these:

1. How Do People Evaluate a Web Site's Credibility? Results from a Large Study
2. Experts vs. Online Consumers: A Comparative Credibility Study of Health and Finance Web Sites

Index of Key Findings

Interface Definition: Methods
Interface Definition: Use Models
Interface Design: Branding / Credibility
Design: Navigation
Design: Presentation
Design: Interaction
Design: Content
Interface Assessment: Methods
Special Environments: Cell Phones
Special Environments: IVR
Special Environments: Globalization / Localization
Special Populations: Aging Users
Finally (!)... a few General Observations about Design Guidelines

Interface Definition: Methods

Behavior reported in focus groups do not reflect behaviors observed during usability testing. (Eysenbach & Kohler, 2002)

When you are interviewing users to determine critical and desired product attributes, asking participants to generate rank-ordered lists yields a group of attributes that varies widely. If, instead, you ask participants to describe the ideal version of the product, you will end up with fewer attributes that will highlight attributes that participants use to evaluate the product. (Breivik & Suphellen, 2002)

Interface Definition: Use Models

Users demonstrate three primary interacting approaches on the Web:

• Browsing (non-specific surfing),
• Information Gathering (reconnaissance for future transactions / information finding), and
• Information Finding (seeking specific answers).

Under a task analysis, these three approaches to interacting with the Web have different flows, behavior characteristics, and determinants of success. Therefore, it is important to know how users approach your site. For example, it seems unlikely that Intranet users will do much true browsing – if they are really browsing, they are more likely to be doing it on external sites, instead. (Sellen, Murphy & Shaw, 2002)

Experienced users access the Web less often and more sporadically. (Cothey, 2002)

More experienced users access fewer sites. (Cothey, 2002)

More experienced users tend to browse to sites (either directly or via other trusted sites) rather than getting there via search. (Cothey, 2002)

Users take advantage of visual details (white space, text saliency) to plan their scan paths. (Hornoff and Halverson, 2003)

Interface Design: Branding/Credibility

Users who do not have in-depth knowledge of the content domain evaluate a site's credibility based on its:

1. design look
2. information design / structure
3. information focus.
   (Fogg, Soohoo, Danielsen, Marable, Stanford and Tauber, 2002)

Domain experts evaluate the credibility of a site based on domain specific criteria.
Medical experts consider:

1. name / reputation / affiliation
2. information source
3. company motive.

Financial experts consider:

1. information focus
2. company motive
3. information bias.
   (Stanford, Tauber, Fogg & Marable, 2002)
   For discussion see: Web Credibility: What makes a Web site credible?

Design: Navigation

Structures that have multiple levels should concentrate site navigation information at the first level and at the level closest to the ultimate content pages (or terminal nodes). (Bernard, 2002)
For discussion see: Depth vs. Breadth

Providing a visible sitemap facilitates site learning and encourages comprehensive exploration of a site. (Danielson, 2002)

Less than half of users take advantage of breadcrumbs (even when most report having noticed them). (Lida, Hull and Pilcher, 2002)

Under click-stream analysis, breadcrumbs are not more efficient than other approaches to navigation. (Lida, Hull and Pilcher, 2002)

Expandable menus are slower to navigate than sequential menus. This is particularly true when users are deeper in a site. (Zaphiris, Schneiderman & Norman, 2002)

Sequential menus helps users develop a better sense of orientation within a site. (Zaphiris, Schneiderman & Norman, 2002)

Users scan group labels within indexed content. (Hornoff and Halverson, 2003)

When users have expectations about where desired information will appear on the next screen, they move their eyes there in anticipation while the screen paints. (Hornoff and Halverson, 2003)

Indexed menus and vertical menus are preferred over horizontal menus (in absolute, not statistically significant, terms). (Bernard and Habmlin, 2002)

Users preference tends toward indexed menus over vertical or horizontal menus. (Bernard and Habmlin, 2002)

For searching tasks, pull-down menus provide fastest performance. For browsing tasks, combined global / local navigation provides fastest performance. (Yu and Roth, 2002)

Presenting short summaries with text links helps users understand and predict the content associated with the link. (Baker, Bernard & Riley, 2002)

Users only perceive / encode (change in) elements of the display that they are directly focused on. (Simon & Chabris, 1999)

Design: Presentation

Presenting 10 to 50 search returns per results page optimizes both performance and preference. (Bernard, Baker and Fernandez, 2002)

Users find it difficult to ignore gestalt grouping principles of color similarity and common ground. (Beck and Palmer, 2002)

Color similarity has a stronger perceptual influence than common region, proximity, or grouping. (Beck and Palmer, 2002)

Proximity has a stronger perceptual influence than common ground for grouping. (Beck and Palmer, 2002)

Users expect a back-to-home link in the upper left corner of the page. Bernard (2002)

Expectations of footer links are fading. This reflects a change from Bernard (2001). Bernard (2002)

Users expect Help / Service to appear in the upper right corner of a page. Bernard (2002)

Highlighting with surrounding silence directs attention and enhances recall in short lists. Extrapolation: white space serves to direct attention and enhances recall. (Olsen, 2002)

Using animation in banners helps to capture attention but does not necessarily increase recall. (Bayles 2002)

Chart junk (additional visual information, often included with good intentions) gets in the way. (Blasio and Bisant, 2002)

Users can compare and evaluate absolute information (tables) more efficiently than relative information (graphs). Users prefer relative presentation. (Blasio and Bisant, 2002)

Banner animation does not significantly undermine speed of performance. It does, however, significantly increase perceived workload and frustration of users. (Burke and Hornoff, 2001)

Users favor fluid page presentations over centered or left-justified presentations. There is no performance difference across the three presentation styles. (Bernard and Larsen, 2001)

Users prefer sans-serif fonts to serif fonts on the Web though there is no significant performance change. (Bernard and multiple other authors in multiple papers)

Users read 12 point fonts faster than 10 point fonts. (Bernard and multiple other authors in multiple papers)

Users read Times and Arial faster than Courier, Schoolbook, and Georgia. (Bernard and multiple other authors in multiple papers)

At font size 10, users prefer Arial, Courier, Comic, Georgia, and Verdana to Times. At 12 point, Arial is reliably preferred to Times. (Bernard and multiple other authors in multiple papers)

Design: Interaction

Poorly titled pages make using the Back Button / Browser History harder. (Cockburn, McKenzie and Jason-Smith, 2002)

Users report that scrolling on content pages is not worse than paging. (Baker, 2002)

Users typically peruse the first page of search results (93% of links visited were within first 10 results). (Eysenbach & Kohler, 2002)

The specific navigation approach that users demonstrate (click-navigation vs. search behavior) tends to depend on the task at hand. (Spool, 2001)

The more times users try to search, the less likely it is that they will find what they are seeking (if users search 3 or more times, they won't find it). (Spool, 2001)

Providing category information helps users group and effectively evaluate search returns. (Dumais, Cutrell and Chen, 2001)

Delays greater than 100ms effectively "hold the perceptual system hostage". (Hornoff and Halverson, 2003)
For discussion see: Are We There Yet? Effects of delays on users' perception of Web sites

Design: Content

Presentation style influences how users encode and interact with content. (Jedetski, Adelman & Yeo, 2002)
For discussion see: Decisions, Decisions. What's a poor designer to do?

Presentation can influence user decision-making strategies (compensatory vs. non-compensatory). (Jedetski, Adelman & Yeo, 2002)

Content RE-written for presentation on the Web reduces time to task (80%), improves memory for content (100%), and increases satisfaction-with-use (37%). (Morkes & Nielsen, 1998)

Users' eyes are drawn to pictures of human faces the first time they see a page. (Riegelsberger, Sasse & McCarthy, 2002)

Photographs do not increase the trustworthiness of already credible sites. They do, however, improve the credibility of sites that are not generally perceived as trustworthy. (Riegelsberger , Sasse & McCarthy, 2003)
For discussion see: From Bricks to Clicks: Building customer trust in the on-line environment

Interface Assessment: Methods

If you conduct both usability testing and expert / heuristic review on the same site, there will be some overlap in the usability problems that are identified. However, there will also be a specific set of problems that is uniquely identified by expert / heuristic review, and an orthogonal set of problems that is uniquely identified by usability testing. (Fu, Salvendy and Turley, 2002)

Heuristic review tends to uncover usability issues related to presentation (skills- and rules-based user performance). (Fu, Salvendy and Turley, 2002)

Usability testing tends to uncover issues related to domain-specific knowledge and interaction (knowledge-based user performance). (Fu, Salvendy and Turley, 2002)

For discussion see: Pitting Usability Testing against Expert Review

Special Environments: Cell Phones

Talking on a cell phone undermines attention to driving. (Trbovich & Harbluk, 2003)

Special Environments: IVR

IVRs with 3 or fewer levels, and 4 or fewer choices per level, work best. (Dulude, 2002)
For discussion see: Press 8 for Natural Language

Special Environments: Globalization / Localization

Comprehension and recall is enhanced when information / arguments presentation (inductive vs. deductive) is culturally appropriate. (Spyridakis & Fukuoka, 2002)

Special Populations: Aging Users

IVR Study: Older users have a harder time recovering from errors. (Dulude, 2002)

Older users have a harder time understanding / adjusting to internal jargon. (Dulude, 2002)

Finally (!)... a few General Observations about Design Guidelines

Design guidelines can be helpful, but they should not be construed as rules. Users can (and do) learn and adapt to new environments – even when things are not quite as expected. (Spool, 2002)

Design guidelines should be specific. Their effectiveness should be measurable and testable. That is, they should be "operationalized". (Spool, 2002)

Know HOW and WHY the guidelines you are applying were created. Were they created in the same task-design context that you want to apply them in (e.g., If you are designing a data-mining interface, look for guidelines that speak to designing for that specific task environment.) How were they / will they be tested? Are they derived from sound research? (Bosley & Straub, 2002)

This gives some idea of how much NEW information there is to absorb, not to mention the massive foundation of previous research. This is why it is NOT enough to just design from intuition and good intentions. We need the benefit of the scientific insights. As we move toward mature usability engineering, institutionalized in organizations, we must have resources to constantly incorporate these insights into our practice.

If you had sent in an "Ask Eric" submission wondering if a graph was better than a chart, I would have recommended the graph for any presentation of a trend or relationship. Now, having read this newsletter, Blasio and Bisant have got my attention. I'll now need to review their paper in detail and decide if I must change my position. I've changed hundreds of such positions based on new research. Constant growth in understanding is a requirement of this field.

References

Baber, C. and Baumann, K. (2002) Embedded human computer interaction, Applied Ergonomics 33, 273-287.

Baker, J. Ryan, (2003), The Impact of Paging vs. Scrolling on Reading Online Text Passages, Usability News 5.1.

Baker, Bernard and Riley, (2002). Reading Online News: A Comparison of Three Presentation Formats, Usability News 4.2.

Bayles, M. (2002) Designing Online Banner Advertisements: Should we animate?, In: Proceedings of CHI, April 20-25, 2002, pp. 363-368.

Bernard M. (2002), Examining User Expectations for the Location of Common E-Commerce Web Objects, Usability News 4.1.

Bernard, M.L. (2002) Examining the effects of hypertext shape on User Performance, Usability News, 4.2.

Bernard, M (2001), Developing Schemas for the location of Web objects, Usability News 3.1.

Bernard, M., Baker, R., & Fernandez, M., (2002) Paging vs. Scrolling: Looking for the Best Way to Present Search Results, Usability News 4.1.

Bernard, Fernandez & Hull, (2002), The Effects of Line Length on Children and Adults' Online Reading Performance, Usability News 4.2.

Bernard, M., and Hamblin, C., (2003), Cascading versus Indexed Menu design, Usability News 5.1.

Bernard and Hull, (2002), Where Should You Put the Links? Comparing Embedded and Framed/Non-Framed Links, Usability News 4.1.

Bernard, M. and Larsen, L. (2001), What is the best layout for multiple-column web pages?, Usability News 3.2.

Bernard, M., Lida, B., Riley, S., Hackler, T. and Janzen, K.A., (2000), comparison of popular online fonts, Usability News.

Bernard and Mills (2000), So, what size of type and font should I use on my website?

Bernard, Mills, Peterson, and Storrer, (2001), A comparison of popular online fonts: Which is best and when?

Beynon-Davies and Holmes, Integrating rapid application development and participatory design, IEEE Proceedings-Software, V145 (4) 105-112.

Blasio, A.J and Bisantz, A.M. (2002), A comparison of the effects of data–ink ratio on performance with dynamic displays in a monitoring task, International Journal of Industrial Ergonomics 30, 89–101.

Bolger, N. Davis, A and Rafaeli, E. (2003), Diary Methods: Capturing Life as it is Lived, Annual Review of Psychology, 54, 579-616.

Bosley, J., and Straub, K., (2002), Data exploration interfaces Meaningful web database mining by non-statisticians, IBM Make IT Easy Conference 2002.

Breivik, E. and Supphellen, M., (2002), Elicitation of product attributes in an evaluation context: A comparison of three elicitation techniques, Journal of Economic Psychology, 24, 77-98.

Brewster, Lumsden, Bell, Hall and Tasker, (2003), Multimodal "Eyes-Free" Interaction Techniques for Wearable Devices, In: Proceedings of CHI, April 5-10, 2003, pp. 473-479.

Bruseberg A and McDonagh D (2002), "Product Handling and Visual Evaluation Supporting New Product Development." In: McCabe P T (ed.) Contemporary Ergonomics 2002. Taylor & Francis, London, pp 303-308, ISBN 0-415-27734-5.

Bruseberg, A. and McDonagh-Philip. D. (2002), Focus groups to support the industrial/product designer: a review based on current literature and designers' feedback. Applied Ergonomics, 33,27-38.

Cockburn, A., McKenzie B.and Jason-Smith, M., (2002), Pushing Back: browsers, International Journal of Human-Computer Studies 57, 397-414.

Constantine, L., (2002), Devilish Details: Best practices in Web design, In: For USE 2002 Proceedings. Rowley, MA: Ampersand Press.

Copas, G., (2003), Can Internet Shoppers Be Described by Personality Traits? Usability News 5.1.

Cothey, V., (2002), A longitudinal study of World Wide Web Users' Information-Searching Behavior, Journal of the American Society for Information Science and Technology, 53(2): 67-78.

Coyne, K.P., (2002), Testing more than Alt Text: Techniques for testing Accessibility and Usability, Neilsen Norman Group.

Danielson, D., (2002), Web navigation and the behavioral effects of constantly visible site maps, Interacting with Computers 14 pp. 601-618.

Ekman and Lankoski, (2002), What should it do? Key Issues in navigation interface design for small screen devices, In: Proceedings of CHI, April 20-25, pp 622-6223.

Eysenbach, G., and Kohler, C., (2002), How do consumers search for and appraise health information on the World Wide Web? Qualitative study using focus groups, usability tests, and in-depth interviews, BMJ 324, 9.

Fogg, B.J., ( 2002), "Stanford Guidelines for Web Credibility." A Research Summary from the Stanford Persuasive Technology Lab. Stanford University.

Fogg, B.J., Soohoo, C., Marable, L., Stanford, J., and Tauber, E., (2002), How Do People Evaluate a Web Site's Credibility? Results from a Large Study, Report released: October 29, 2002, Report updated: November 11, 2002.

Fox, S. and Rainie, L., (2002), Vital Decisions, PEW INTERNET & AMERICAN LIFE PROJECT.

Fu, L., Salvendy, G., and Turley, L. (2002) Effectiveness of user testing and heuristic evaluation as a function of performance classification, Behaviour & Information Technology, vol. 21, no.2 , pp. 137-143.

Gershman, A. Ubiquitous Commerce - Always On, Always Aware, Always Pro-active, In: Proceedings of the 2002 Symposium on Applications and the Internet (SAINT'02).

Hearst, M., Elliott, A., English, J., Sinha, R., Swearingen, K. and Lee, K., (2002), Finding the flow in Website search, Communications of the ACM September 45(9), pp. 42-49.

Hornoff, A.J. and Halverson T., (2003), Cognitive Strategies and Eye Movements for Searching Hierarchical Computer Displays, Proceedings of the ACM Computer Human Interaction Conference. Ft. Lauderdale, FL.

Horrigan, J., and Rainie, L., (2002), Counting on the Internet, PEW INTERNET & AMERICAN LIFE PROJECT.

Ivory, M., Sinha, R., and Hearst., M., (2001), Empirically validated web page design metrics, In: Proceedings of the CHI, volume 1, 53-60.

Jacko, J.A. and Slavendy, G., (1996), Hierarchical Menu Design: breadth, depth and task complexity, Perceptual and Motor Skills, 82, 1187-1201.

Jedetski, J., Adelman, L. and Yeo C., (2002), How Website Decision Technology Affects Consumers, IEEE Internet Computing, March-April, 72- 79.

Josephson, S. and Holmes, M.E., (2002), Visual Attention to Repeated Internet Images: Testing the Scanpath Threory on the World Wide Web, In: Proceedings of the ACM Eye Tracking Research & Applications Symposium, New Orleans, LA.

Kiger, J.I., (1984), The depth/breadth tradeoff in the design of menu-driven interfaces, International Journal of Man-Machine Studies, 20, 201-213.

Kim, J. and Lee, J. (2002), Critical design factors for successful e-commerce systems, Behavior and Information Technology, 21(3). 185-199.

Larson, K and Czerwinski, M., (1998), Web page design: Implications of memory, structure and scent from information retrieval, Proceedings of the Association for Computing Machinery's Computer Human Interaction Conference, 18-23.

Lida, B., Hull, K., and Pilcher, K., (2003), Breadcrumb Navigation: An exploratory study of usage, Usability News 5.1.

Matson, (2001), Re-Forming Information: A Case Study In teaching Content Encapsulation.

McDonagh, D., Bruseberg, A, and Haslam, C., (2002), Visual product evaluation: exploring users' emotional relationships with products, Applied Ergonomics 33, 231-240.

McInerney, P., (2002), How Teams manage Usability defect information: some case studies, IBM Make IT Easy 2002.

Morkes and Nielsen, J., (1998), Applying Writing Guidelines to Web Pages, In: Proceedings of ACM SIGCHI, April 18-23.

Olsen, G.D., (2002), Salient Stimuli in Advertising: The effect of Contrast Interval Length and Type on Recall, Journal of Experimental Psychology: Applied 8(3), 168-179.

Pirhonen, Brewster and Holguin, (2002), Gestural and Audio Metaphors as a Means of Control for Mobile Devices, In: Proceedings of CHI, 291-298.

Purchase and Worrill, (2002), An empirical study of on-line help design: Features and principles, International Journal of Human-Computer Studies 56, 539-567.

Russell, M.C., James, M. and Cohimia, A., (2002), Reading from a PalmPilot using RSVP, Usability News 4.1.

Silvers, V.L. and Kreiner, D.S., (1997), The Effects of Pre-Existing Inappropriate Highlighting on Reading Comprehension, Reading Research and Instruction, vol. 36, no. 3, 217-23.

Simon, D. and Chabris, C.F., (1999), Gorrillas in our Midst: sustained inattentional blindness for dynamic events, Perception 28, 1059-1074.

Snowberry, K., Parkinson, S., and Sisson, N., (1983), Computer Display Menus, Ergonomics , 26, 699-712.

Spool, J., Evolution trumps guidelines, UIE Articles

Spool, J., Searching Behaviors, User Interface Engineering Newsletter, May, 2001.

Spool, J., Users don't learn to search better, User Interface Engineering Newsletter, November 27, 2001

Spool, J., People search once, maybe twice, User Interface Engineering Newsletter, November 20, 2001

Spool, J., Searching vs. Linking, User Interface Engineering Newsletter, December 4, 2001

Spyridakis, J.H., and Fukuoka, W., (2002), The Effect of Inductively Versus Deductively Organized Text on American and Japanese Readers, IEEE TRANSACTIONS ON PROFESSIONAL COMMUNICATION, VOL. 45, NO. 2.

Stary, C., (2002), Shifting knowledge from analysis to design: Requirements from contextual user interface development, Behaviour and Information Technology, 21(6), 425-440.

Strahan, E., Spencer, S., and Zanna, M., (2002), Subliminal priming and persuasion: Striking while the iron is hot, Journal of Experimental Social Psychology 38 p.556-568.

Teagyem R.C., and Whitney, H.X., (2002), What's love got to do with it? Why emotions and aspirations matter in person centered design, User Experience(Winter), 6-13.

Uther, M., (2002), Mobile Internet Usability: What can 'Mobile Learning' learn from the past? Proceedings of the IEEE International workshop on wireless and mobile technologies in education.

Vrendenburg, K., Mao, J.Y., Smith, P., and Carey, T., (2002), A Survey of User-Centered Design Practice, In: Proceedings of CHI, 4(1) pp. 471-478, ACM press.

Weenig, M.W.H., and Maarleveld, M., (2002), The impact of time constraint on information search strategies in complex choice tasks, Journal of Economic Psychology 23, 689-702.

Wobbrock, Forlizzi, Hudson and Meyers, (2002), WebThumb: Interaction Techniques For Small-Screen Browsers, Proceedings of CHI, 205-208.

Wogalter, M., Conzola, V., and Smith-Jackson, T., (2002), Research-based guidelines for warning design and evaluation, Applied Ergonomics 33 219-230.

Woodling, D.S., (2002), Fixation Maps: Quantifiyng Eye-movement Traces, Paper presented at ACM - ETRA, New Orleans, LA.

Yu, B., and Roh, S., (2002), The effect of Menu Design on Information Seeking Performance and User's Attitude on the World Wide Web, Journal of the American Society for Information Science and Technology, 5(11): 923-933.

Zaphiris, P., and Kurniawan, Sri H., (2001), Effects of Information Layout on Reading Speed: Differences between paper and monitor presentation, Proceedings of the Human Factors and Ergonomics Society 45th Annual Meeting.

Zaphiris, P. and Mtei, l., (1997), Depth v. Breadth in the Arrangement of Web Links.

Zaphiris, P., Schneiderman, B., and Norman, K., (2002), Expandable indexes vs. sequential menus for searching hierarchies on the World Wide Web, Behavior and Information Technology, 21(3), 201-207.

Zhou, W., Farrington, S., and Hutcheson, T., (2002), Enlightened Tree View for Search and Navigation, IBM EOU Make IT Easy Conference, San Jose, CA.

Past Issues