Ask Eric: Questions & Answers

December 16, 2003 – submitted by Karl of USA

Question: Do you know of any guidelines or suggestions that would be useful in creating an online survey?

Eric's response: Sure, happy to help. First be very clear the question you are trying to answer. Surveys can only measure what people report. So a question like "Is this efficient?" is out. Consider if you are just getting opinions ("I like it"), or gathering facts ("Select the reason you came to this site"). Are you measuring against something like a past version of your site or a competitor's site.

Consider carefully WHO you will include and how you will get enough people to respond. As the response rate goes down you start measuring from a highly selected group (generally only people with a lot to say either good or bad) and you start missing the true mainstream viewpoint.

Design the questions so they are not leading. If you strongly believe something ask it in the opposite way. For example if you think a cartoon site is just for kids ask "How much of this site usage is by the adults in the family?". Also be sure the questions are very specific. Avoid general questions like "Did you like it?". The answers won't mean much. Finally, be aware that a survey can only serve as a small part of an overall data gathering process.

November 25 , 2003 – submitted by Janis McDonald of Pittsburgh, PA

Question: Has there been any usability research done that shows that online support group users are less likely to continue communicating online after having the opportunity to meet in person? I am currently involved in a schizophrenia Web site support group study that includes one face-to-face psychoeducational workshop where all participants will, in fact, meet face-to-face, losing their on-line anonymity.

Eric's response: Janis, I am not aware of any research on this topic. However, I expect that meeting people in person will change the character of the interaction. This may not at all reduce the amount of online interaction. Just look at the number of classmates and coworkers who interact online regularly. In a given instance the relationship could be enhanced or diminished based on the character of the physical meeting. However, there may be a therapeutic value to the anonymous support group which could be lost. I would be very much interested to hear your experience.

September 16, 2003 – submitted by David Angaran of Florida, USA

Question: RE: Patient evaluation instruments for individualizing e-health technologies and techniques.

What evaluation tools exist for a health care practitioner to assess a patients competency, capability and preferences for the use of the various e-health technologies eg. phone, e-mail, video, etc? The goal being to meet the IOM patient centered criteria and improve the efficiency, cost: effectiveness and safety of the system.

Eric's response: FANTASTIC question! The answer: None. I recall Arni Lund trying to create a team to just DEFINE dimensions of limitations in using technology. I don't think it ever happened. So I believe there is not even a standard description of the types of limitations. Sure, usability practitioners have some general ideas. We know about various levels and types of limitations in sight, hearing, motor control, cognitive capabilty, etc. We do try to design to avoid gratuitiously adding barriers (like the incredibly stupid Control/Alt/Delete required to get into certain windowing environmrnts). But there is no specific instrument for ability to use various technologies. I guess clinicians will have to do it based on thier common sense and adjust it based on their patient's expeieince.
But it's a DAMN good idea to have a formal assesment instrument!

July 11, 2003 – submitted by Mary Dvorak of Humble, TX

Question: What is the standard for using wildcard functionality? – *, ?

Eric's response: Unfortunately, there is no standard and most people don't understand wildcards. Only a TINY fraction of searches use wildcards. Those that understand the idea of wildcards will usually use them infrequently and therefore not remember much about the syntax. There are wildly different wildcard syntaxes used.

This means that you must have an "advanced search" facility that helps people with advanced searches. It is actually better to have the advanced search provide the wild card and other capabilities not as a syntax but as a form-filling interface. We have very good data that Boolean logic confuses most users. So hide this and make it as simple as possible.

For very advanced users you may want a wildcard functionality. The * is reasonable for any number of characters. The $ is sometimes used for only a single character. You also need a "NOT" capability in most cases (generally a minus sign "-" or the word "NOT").

May 13, 2003 – submitted by Prema Subramaniam of Kuala Lumpur, Malaysia

Question: Why is it important to test usability on e-commerce?

Eric's response: Prema, here is a related question: Is it important to debug the programs used to create an e-commerce site? Certainly you COULD code the site and just put it on the Web. You might hope that it would work well. But you would be assured of a site that has points of failure. Are these points of failure acceptable? They frustrate users. They reduce sales. They hurt your company reputation. With that it is silly and unprofessional to skip debugging and people rarely do it.

It is actually WORSE to skip usability testing then skipping debugging. If you skip usability testing you are completely assured that you will have a site with points of failure. The USER will be unable to complete activities. The user will be frustrated. Sales will be roughly cut in half. Your reputation will be damaged.

But what makes it worse is this – with code problems kind users are likely to let you know there was this error message or that broken link, but with problems of usability users are less likely to help. They will just feel frustrated and feel that it is unpleasant to do business with your company. A few will complain. But the complaints will be nonspecific ("this site sucks") and if they do make recommendations they will rarely recommend the correct solution (e.g., If users ask for 'Search' it is almost always a symptom of poor navigational structure).

If you have EVER created an e-commerce site without testing, it was surely a big mistake. Don't do that again.

March 20, 2003 – submitted by Debra Smith of United States

Question: I would someone to explain what is meant by 'User Characteristics' in relation to Human Computer Interactions.

Eric's response: To do a good design you must know the user characteristics, taskflow, and environment. The user characteristics are critical because they indicate the limit and nature of the interface designs you can use. For example a very novice user will have more trouble with an image menu then a simple button menu. They also indicate the level of reading skill, general intelligence, and expertise. These all make a difference in the design decisions. In addition to capabilities we must also study the user's motivation. We design interfaces that eliminate things users hate to do, and increase the things they like to do. We make designs that attract users because they resonate with the brand characteristics they care about.

Be careful when you consider expertise. This actually can mean three different things. There is expertise with computers. There is expertise with a specific application (for example a user who operates an interface full time will soon become an expert). Then there is also expertise with the domain (for example a user may be terrible with computers – they may use an interface once a year for 1 hour – but they may be very knowledgeable about insurance).

January 29, 2003 – submitted by Pinky Thakkar of Bloomington, IL

Question: Could you give me some tips how to conduct an informal usability test. Is there a checklist that I would need to follow. Could you also suggest me some sites where informal usability is conducted for a online billing system etc...

Eric's response: I am not familiar with a site that conducts informal usability testing for online billing. But if you want to try it yourself I can give you enough to get some interesting results. Be aware that a professional study will offer more then you can get with this approach.

1. Schedule about 12 people for testing. Make sure they are representative of typical users. Do not even THINK about using other company employees. Tell them it will take an hour and that you will pay them $75 for their time.

2. Make up a list of tasks that users would typically perform. Make up enough tasks so that they will take about 45 minutes to complete. Make sure that the wording of the task does not give away how to use the interface. So if the dropdown is labeled "Edit", the task wording should say "what if you want to change this word". Do NOT say "What if you want to edit this memo?"

3. When people come in, have them sit in front of the computer and you sit next to them. Tell them you will have them try out several tasks on the system. Tell them "Read out loud, talk out loud, and tell me what you are thinking".

4. Have the user go through each task. You hold a set of screen printouts and when people get stuck draw a red circle around the part of the screen that is having the problems.

5. When you are done, if you have three or four red circles around a part of a screen printout, you have identified a serious problem.