Ask Eric: Questions & Answers

November 13, 2001 – submitted by Anonymous in the US

Question: We are thinking of adding a new feature to our Intranet that is present on every page and will allow our employees to rate each page.

We are skeptical of doing this because we already have channels for associates to offer feedback and we rarely receive any.

What is your opinion on this type of feature? Are there any studies out there that can help us in making this decision? Thanks.

Eric's response: It is a good idea to have an e-mail address available for associates to use if they have a problem with a page. But the idea of a feedback link seems questionable. It will not be used often. Those few who will use it will be a very poorly selected sample. Only a fraction of a percent will use it and these will be users that feel very good or very bad about the page. Given this, what will you do with the data? There are many effective usability testing and survey methods. These give valuable insight into the user experience and performance. I suggest you use those.

August 22, 2001 – submitted by Lisa Fahy of Rochester, New York

Question: After the completion of a usability test, the participant is asked to complete a questionnaire. In your opinion, is it best to read the questions to the participant and write their responses, or is the preferred method giving the participants the questionnaire and having them complete it on their own?

Eric's response: If you want to be able to probe about responses consider doing the questionnaire verbally. But if the questionnaire has scale choices this is VERY awkward (e.g., "So on a scale of 1-7 how important...." YECH).

In general, let them fill out final debrief questionnaires themselves.

April 1, 2001 – submitted by Harish Bhardwaj of India

Question: What exactly is "scenario based designing"?

Eric's response: "Scenario" refers to a story of how the site or application will be used. We used to say "Task Design." Then we said "Use Case." Now we may use "Scenario." There is no real difference. But the idea is crucial. You design the user experience first and then make the technology support that experience. That is user-centered design. That is the ONLY way you are likely to get a successful design from the user's viewpoint.

March 15, 2001 – submitted by Farida Hassanali of West Yorkshire, UK

Question: 1) Why would you consider evaluation to be important in user interface design?

Eric's response: 1) How often can a developer code a substantial program and get it right the first time without debugging? Never happens. So what would you call a developer who codes an application and sends it out for release without debugging? Unprofessional (or probably, fired).

It's the same with usability. There is no chance of getting it right without testing.

When I was working on the Dell.com site, we put in a tab called "STORE" – meaning, "You can shop here." When we tested users, they thought it meant something else – they told us, "Gee I didn't know you could save data on the Dell site!" I am pretty good at UI design, but I never thought that anyone would read it that way. No matter how much experience you have in design, you need to test.

Question: 2) How should an evaluation exercise be carried out – what criteria should we evaluate against and what evaluation techniques should we use?

Eric's response: 2) The first rule is: test early and often. Generally, it makes most sense to start with cheap and fast methods like expert reviews and quick tests. Later you can do more expensive high-fidelity testing procedures.

We always test to measure speed, accuracy, self-evidency (or training requirements), and satisfaction. But we prioritize these factors depending on the application. For example, in a Web site used by the public to shop, self-evidency and satisfaction are critical. But for clerks processing insurance forms, speed and accuracy are what matter.

There are hundreds of different testing methods to choose from. It is a good idea to get some training and/or professional help on which methods to choose and how to use them properly. But be aware that not all consultants understand these things. I met with a client who had used a major system integrator to do usability testing, at a cost of $100,000. The test protocol involved showing the users the process for setting up a portfolio, and then asking "Was it easy?" Sad. In usability testing, you don't ask users if things are easy; instead, you watch to see if users can actually complete tasks easily.

Question: 3) Why is the adoption of a disciplined design approach to the design of user interfaces seen as important?

Eric's response: 3) We all know that designing software is hard – yet software is written for computers, which are a whole lot simpler and more reliable than people. That's why it's harder to do good interface design. To do really good UI design you need to be a professional with a deep understanding of perception, cognition, learning theory, motivation theory, biomechanics, and social psychology. And that's just they start. You also need to know the specific literature and methods of the field of software ergonomics.

On top of that, you must follow a strong methodology that integrates well with business and technical staff. Otherwise, the application is likely to fail.

Question: 4) What means of carrying out a design exercise (e.g. design techniques and guidelines) should be adopted?

Eric's response: 4) To create an effective user interface design you need a systematic process. At HFI we start by creating a high-level user-interface structure and style. These must fit with the user's needs, taskflows, and deep mental models. We then test to insure that the functions, navigation, and appearance meet the business objectives. Next, there must be standards to ensure page consistency. With these in place it is relatively easy to proliferate the detailed page designs. There must also be more advanced usability testing toward the end, to ensure that it really works.