| |
Question: Hi, there seems to be a trend towards using rapid prototyping to drive out requirements. At one employer we used Dreamweaver; at another, we used iRise. I am wondering what tool(s), if any, you recommend for high fidelity prototyping? |
Eric's response: First I have to say that rapid prototyping is hardly an optimal way to drive out user requirements. I think rapid prototyping is pretty much only used for very small and non-critical applications.
In user-centered design we do prototypes to support various data gathering and usability testing programs. We have a neat little tool that goes along with PowerPoint™ or Visio™ and we use that for basic wireframe mockups. If we need more interactivity we mock it up in HTML.
But the key thing is definitely NOT the tool that you use to prototype. It is the process that you are using. Rapid prototyping has been around for ages and I don't think I see an uptrend in its use. In fact there have been several short term fads since rapid prototyping was in vogue (e.g., Extream Programming). |
Top |
| |
Question: I know this isn't a question with an easy answer, but I was wondering if you could provide any insight as to which usability methods (CW, Heuristics, Questionnaires) are useful in certain circumstances. I realize different methods are applicable at different stages of the design cycle, some are better at providing design recommendations, others at capturing the subjective opinions of the user, but in reality it seems the method used relies heavily on the evaluator's preference / familiarity / time. Is this true? I guess what I was looking for, and have yet to find, is in a "perfect world" given this circumstance, you should use this method. Thank you for your time. |
Eric's response: Actually we do know the correct process and methods that work in different situations. When Susan and I wrote the Schaffer Weinschenk Method™ we not only defined a series of methods in depth, but we also defined the criteria for selecting different methods. So a good methodology will answer much of this type of question.
Beyond selecting the basic methods needed, there are many decisions related to cost (e.g., with a geographically dispersed population, the remote testing methods are often indicated). There are also decisions related to culture (e.g., many of the basic American usability methods have to be modified to work in Asia). Of course you also must decide on many issues of detailed methodological design, such as locations and numbers of participants to test.
So first you need a good methodology, which answers most questions. Then you need an experienced expert to refine the details of the methods. |
Top |
| |
Question: When presenting a message to the user indicating that the system is currently unavailable, is it best to tell the user to retry in a specified period (say 15 minutes) when we are unsure if it will be available then? Or should the message indicate that they try later and the user determines when they want to do so? We are trying to determine which message would cause the user less frustration. |
Eric's response: It depends if you have any information to share. It is difficult to estimate how often the user would WANT to try again. If it is very urgent, the user might just keep clicking. If it is not urgent, the user might try tomorrow. If you have a fault where you know the recovery time even roughly, you can telegraph this with a phrase like "Try again in a few minutes", or "try again later in the day". Otherwise, it would be a bad idea to tell them to try again in 15 minutes, when it might be ready in a moment. |
Top |
| |
Question: Well, here's my problem: I just don't know how to approach this. The thing is that I've coded a Web portal along with a back office tool to upload all the content you want, both from scratch. But now I need to test my back office's usability. But the main problem is that most of the tasks that I should ask to the user require some previous knowledge of the back office functioning. So I don't think I can ask the user to do all the tasks I want without getting him to understand how the back office works. It would be a waste of time and the test results would be terrible. But on the other hand if I told the user how the back office works then it wouldn't make much sense to do a usability test, right?
Well, thanks for your patience and for your time! |
Eric's response: First of all let me voice a major concern about process. It sounds very much like you are about to start doing "usability" with the system coded. This is really, really, really, really, really, really bad.
First of all usability is NOT just usability testing. You should be following a user-centered process from the beginning. You need to understand the user, their needs, their taskflow, their environment (etc) before you conceptualize the system and certainly also when structuring the interface. You should have standards and follow usability principles in the detailed design of the interface.
In terms of testing, you should be testing early and often. The kind of final testing you are describing is almost certain to show lots of problems that you will not have time or budget to fix.
Finally, when you look at the test process, you will need some training in how to do it successfully. Usability testing primarily is useful for measurement of self-evidency. This is important. But is NOT an indicator of efficiency if you have long-term, high-volume users. In this case there are modeling techniques that let us tell if the interface will work efficiently for test users. |
Top |
| |
Question: Do you know of any single question that can be asked as part of a larger questionnaire that measures overall usability? I am familiar with SUS, WAMMI, QUIS, PSUQ, and others. But, I want to capture the usability of a Web site using a single question without having to make users go through an entire questionnaire? If such a "silver bullet" exists, has it been validated? Thanks! |
Eric's response: The best you can do is to identify the business-based human performance or experience objective and ask that. For example, an eCommerce site might have a question like "Did you buy something?" A game might have a question, "Was it enjoyable enough to recommend to a friend?" This gives an indication of the extent to which it is working. But it provides almost no diagnostic input. So it's a bit limited. |
Top |
| |
Question: How do you actually test an interface? Is it enough if we do schema validation alone? |
Eric's response: By "schema evaluation" I assume you mean testing the structure to ensure that people can understand the high level organization of the software. This is VERY important. But there is more needed. One key part is the testing of the detailed pages. It is easy to have a nice structure that is useless because of confusion with wording, layout, color, highlighting, control selection, and other detailed design issues. There are lots of other issues to test of course. Is the software useful? How is the brand manifest?
So if I only had one test to run, I would run a test of the high level structure. But that is like saying if I could only debug one page of a website I would debug the home page. Not exactly a professional viewpoint. |
Top |
| |
Question: If you do not have a test lab, what would be the best environment to conduct your usability testing? Should the user's desk be ruled out given the potential for disruption? Or would the user feel more at ease there? Appreciate your thoughts. |
Eric's response: The desk is probably the worst place to do controlled usability tests. The interruptions are a real problem. Other than that, a quiet and protected space will work fine. A conference room is quite satisfactory. If you want data from the user's desk you should use other methods, like observation, culture probes, or other unobtrusive measures. Even remote testing is not great at a person's desk because of the interruptions that are inevitable. The formal test lab is most critical for showing other people the test in progress. Otherwise, you can run it without fancy cameras and software. The key thing is knowing HOW to test properly. I see SO many companies that spend big on a lab, without anyone there to run it properly. |
Top |
| |
Question: Recently I got into this debate of how many users are good enough for usability testing. What I know from recent research is that this is purely dependent on the requirement but in any case we just can't stick to the number 5 as said by Jacob N. This also came in a recent paper from HFI Also I read that 3 to 4 are good enough to get results, but is this actually based on research or just a thought by someone? |
Eric's response: You can certainly get results with 3-4 users being tested. You can get results with ONE user. The question is if those results are sufficient for to assess usability or guide design.
As you reduce the number of users tested, you increase the chance you will miss problems. You also increase the chance you will falsely identify problems due to the random nature of your selected participant(s).
You will have an even harder time making decisions. Let's say that 3 of your 5 participant prefer one design over another. You might decide to go with that design. But you might as well just flip a coin, because the "3 of 5" is pretty close to a random guess.
It is even WORSE if you want to estimate things like the percentage of people who would use a feature, buy a product, or follow a pathway in the interface. If you have 3 out of 5 participants say that they value a feature, does that really mean that 60% would value it? Actually, you have almost no idea.
You may want to look at some of our newsletters on this topic:
Why "how many users" is just the wrong question
Enough is enough... but five probably isn't
How reliable is usability performance testing? |
Top |
| |
Question: There are some people in industry who think that Cognitive Walkthroughs can be used as one of the methodologies for a Usability Test. I feel that analysis coming from a CW can be an input to design test scenarios. CW as a tool / technique should not be done by users. Please give your inputs. |
Eric's response: A "cognitive walkthrough" is a term that refers to a systematic consideration of the thinking process necessary to complete a task. It is done by professional usability staff and is often a part of an overall expert evaluation. It is not done by users. During usability testing there is a common method call a "talk aloud test." In this very common procedure you ask the user to "read out loud and say what you are thinking." Here the user describes what they are thinking while doing a task. I suspect / hope that this is what you meant and it is a VERY effective method during testing. The only real drawback is that the talking aloud can get in the way of complex tasks. In this case a "retrospective" method can be used where users view video of their work and describe what was happening mentally. |
Top |
| |
Question: Hi Eric, What a great resource, wish I'd found it sooner. If I'm running a focus group (in my case on research hospital intranet site) should I have a laptop set up for each member of the focus group? Do you have any suggestions on questions for an intranet focus group study?
Would you advise giving participants something to do prior to the meeting (a survey, a task to measure functionality)? Any advice is appreciated |
Eric's response: The critical question here is "Why are you running a focus group?" There is almost no place in the usability field for focus groups. A focus group is good only for generating lots of ideas, with most of them bad. So we use them for new product ideation (and then work hard to sift out the few good ideas). But when focus groups are used for evaluation of a site they provide VERY poor data. There are "group think" effects where basically the loudest person's opinion dominates. They also don't reveal at all what the real behavior would be. You are far better off with a talk-aloud usability test. This will show what a person will really encounter and give you very useful insights. |
Top |
| |
Question: Hi, as an IT developer who is following your guidelines of usability, I will be glad if you help you on this questions. From where should we start the process of Usability Testing on a Web site? How can I estimate duration of that test to take it in to account. |
Eric's response: Usability testing is one small part of the overall usability assurance process. There are lots of types of tests and they can be used in several places. For example you might do the following...
Before starting on a development project you might complete a summative test of the old version to learn what needs to be improved and convince executives that usability work is needed.
In developing the structure of the interface you might complete multiple cycles of testing to evaluate concepts and ensure that users can navigate effectively.
Throughout the process of detailed design you might run tests to fine tune the design and also to objectively compare design ideas.
Then when the design is complete you might run a summative test to validate that your usability objectives have been met.
But this said, don't overestimate the power of a usability test. For example, if your main concern is long term use by an expert user, the usability test is of almost NO use. The test only measures performance in a one hour window. This tells you very little about what an expert would want (something we do with modeling not usability testing), |
Top |
| |
Question: I am working in a consulting firm doing IT services. Once I got one requirement from a client in usability testing with psychology background and I am unable to get profiles for this requirement. Can you help me out in this by giving some keywords to find profiles. |
Eric's response: You will find almost everyone in India by using "Usability" as a keyword. You might find that there are not a lot of people with that background available. We are working on it. But they are scarce (and of the ones claiming that expertise, only a small fraction actually have it). |
Top |
| |
Question: We're in the process of choosing a portable usability testing software. I know Morae is a good tool but need to have other alternatives to compare with and submit the proposal to management. Do you know of other software that's comparable to Morae or any open source software? |
Eric's response: Actually the best cheap alternative is to just run the test with video and not use specialized software at all. Even the video is really just for record in case there is a question about what happened during the test. Other than that you can test with nothing but paper notes and get fine results. The main practical advantage of software is easy editing of video clips. If you need this you can also get Camtasia, or use a scan converter and mixer direct into a video recorder. Then you can use commercial editing software to pull together clips. |
Top |
| |
Question: I am a fellow hci/usability professional and came across a truly unique information design decision on the Web site of a highly recognizable brand. I thought you might appreciate it – details are at: pwdoncaster.blogspot.com |
Eric's response: This is interesting if true. A Web site with a printable user's guide! We always said that the documentation is basically a compilation of the failures of the interface designers. So I expect this one is an extreme disaster. The movement in the information age is toward information appliances that have obvious operating models. The Web is even more extreme in the requirement for self-evidency. So a printable user guide is indeed comical.
In a way the most interesting aspect is how very out of perspective development teams can be. This site is for consumer electronics. The users probably care very little about the site. They would never spend the effort or printer time to get the manual: because they DON'T CARE enough. But the developers who work every day on the site care a lot, and somehow get the idea their visitors care as much. I've seen that a lot. |
Top |
| |
Question: Good morning Eric.
What's your opinion about think aloud? We hear and read more and more critical notes related to the once much appraised think aloud protocol. According to critics, it does not only disturb time measurements (which is of course logical and not new to all of us) but also would disturb the clickflow. In other words: according to critics, think aloud leads to other - not the right - results compared to just issuing tasks, observing and not applying think aloud.
When to use think aloud and when not? Would you use think aloud in eye tracking research? Would you use eye tracking time measurement and think aloud separated in one research? What's your opinion? |
Eric's response: Think aloud is wonderful. We use think aloud regularly for formative usability testing. It allows us to quickly understand where and why people get stuck.
It is not useful for summative testing where you can not interfere with the task performance and experience with such an obtrusive method.
It is also not good when you have very complex tasks (like landing a plane on instruments) or where absorption in the behavior is essential (like some testing of computer games). In these cases we use retrospective testing, where we show the participant a tape of their behavior and they narrate what is happening. But retrospective testing takes TWICE as long as normal talk aloud. So it is only used when we have no other choice. |
Top |
| |
Question: We are eagerly looking for USABILITY TESTING TOOLS for intranet Web interfaces. Please help me finding these. In one of your Answers you have mentioned "The moderated testing is VERY promising as a way to get good data and reduce costs. We run remote tests all the time from our offices in India for example. This works very well on every type of usability test. Moderated tools for support of in-person testing are reasonably useful."
Could you please suggest which are these tools and softwares? A nice simple package for capturing the test and editing highlights is very useful. It would be great if you can highlight specific tools with usability testing features. |
Eric's response: You can do very good usability testing with a simple WebEx-type of connection (supplemented by a phone link). But yes, for recording we use Morae, which gets solid reviews from our staff. |
Top |
| |
Question: How do you measure the emotional response of users? Can you give me an equation? I don't prefer a questionnaire-driven measurement, do you have a stand alone formula? Your comments please. |
Eric's response: Well there are really only three ways to measure the emotional response of a
user.
1. You can ask them
So this is the questionnaire. There are good ones and bad ones. But it is the primary method we use. The Microsoft game testers just have one scale... "How much fun is this?". We have in-person, remote, online survey, popups, etc., etc.
2. You can check their physiological responses
There are some physiological correlates of emotional response. You can look at things like pupil dilation. This does show arousal level. But it gives little indication of what the arousal is really about. So it is used infrequently.
3. You can see what they do
Finally, you can see what they do. Watch if they ACTUALLY buy the product. This gives a nice objective measurement. If you really care if they buy the product that is good. But it does not really tell us exactly what the dynamics are that make someone want to buy it. |
Top |
| |
Question: What would you say are the principal differences between usability testing and UAT? |
Eric's response: User Acceptance Testing (UAT) is normally a buyer or committee of users inspecting software and agreeing that it meets the requirements. It rarely catches situations where there is poor usability. The committee members are generally concerned only with functionality and correct processing. In fact, they have usually been involved enough in the design process so that the interface looks OK to them no mater how bad it is.
Usability testing has users who have not been involved in the design actually try to use the new software. It measures where they get stuck, how well they perform, and how they experience the interface. This can be used first to guide refinements to the design (formative testing) and later can be used to measure the success of the design (summative testing). The testing process is organized to get valid and reliable data which takes a moderately high level of expertise. |
Top |
| |
Question: What is your take on current trends using "eye tracking" tools and software-based (MORAE) usability testing? I understand they are only a part of the equation, but do you think companies will rely more and more on these type of new tools to bypass traditional usability testing (to save money)? |
Eric's response: Well let me answer in terms of the different methods. First, eye tracking is impressive. But it has very little applicability to practical design. It is more a research tool. If someone recommends eye tracking for normal design work it usually means they are trying to impress, but don't know what they are doing. Actually, in China I see this all the time where labs are set up with eye trackers just to make it seem impressive. I wish they spent as much on staff that knew how to test.
There are two types of remote testing: moderated and unmoderated. The moderated testing is VERY promising as a way to get good data and reduce costs. We run remote tests all the time from our offices in India for example. This works very well on every type of usability test. The unmoderated testing is less valuable I think. It is OK as a summative test, to measure if people can do a task. But it does not tell you much about WHY a problem is happening. The unmoderated testing is more like clickstream analysis.
Finally, tools for support of in-person testing are reasonably useful. There are some packages that are total overkill for practical design. They are mostly for research labs, or perhaps some military grade applications. But a nice simple package for capturing the test and editing highlights is very useful. |
Top |
| |
Question: Hi Eric, I understand that we need a full user-centered design process for developing any "Usable" software application. In my case we already have a developed application (tool), and would like to make it more usable. Would that mean conduct "Usability test" and then work on the results / evaluations? Or what would be the best way to make the existing software (more!) usable? Thanks. |
Eric's response: The question is what is the real need. We will often do an expert review at that point. From this we can tell what the problem / opportunities are.
If it just needs nicer graphics there is a process for that.
If it needs structural redesign then you really have to do a whole user-centered design process.
If there are detailed design problems then you can take a test-and-refine approach. |
Top |
| |
|
Question: What saves you the most time keying a document:
A. wysiwyg
B. hitting the delete key
C. thesaurus
D. search-and-replace |
Eric's response: Hmmmm., Are you cross checking your professor's grading with me?
Ok, so which saves more time...
The correct answer would be 'B' if there are no other ways of deleting. In some VERY early systems there was no delete capability and a miskeyed entry meant retyping at least the whole line (or punch card). But I don't think this is what you meant.
Given SOME delete capability, WYSIWYG will provide the most time savings. This is because it will avoid having to print out the document, review it, and then make adjustments which are again printed and reviewed. It is time consuming not only while online, but also even longer for 'running-to-printer time'. It also wastes paper. |
Top |
| |
|
Question: Sorry if this isn't the correct place to post this
question. I would like to know the future strategies
for remote usability evaluation. I'm interested in knowing
if new technologies or methodologies are going to be introduced that are
different from the ones which are being used now (surveys, video
conferencing, automatic data gathering, user reported remote
evaluation...). Thanks!!! |
Eric's response: Well I can't share all the details. But let me say we have a team that
is actively working on remote testing methods. I think there are some
exciting innovations in the future. Right now we are still often
struggling with firewalls and slow setup times. But I predict we will
add some technologies that may make it not only closer to being there,
but BETTER then just being there in a simple test. |
Top |
| |
|
Question: When is fault tolerance enough? |
Eric's response: Many people think of fault tolerance as simply a hardware and software
issue. A typical example is the ability of a system to manage a power
outage or management of a system crash. Yet there is a whole usability
side of fault tolerance.
We must consider how the system manages potentially catastrophic user behavior. Many designers carefully work through system crash issues and fail to do even basic due diligence on user risks. The result is often injury or death. I could easily make my whole living just testifying about such poor design practice in liability cases (and have done some of that in the past).
A couple of years ago I was working on a system for monitoring blood glucose levels in diabetics. I noticed a couple of points where the user could make mistakes that would lead them to read the sugar levels incorrectly. They could take too much insulin and have a hypoglycemic attack. That can be fatal. So even in fairly innocuous applications we MUST look at the handling of such potentially catastrophic behaviors. |
Top |
| |
|
Question: Hi Dr.Eric: I want to ask: do you know from
which site I can find the evaluation questionnaire for the information
system usability ISO 9241-10, or ISOmetrics questionnaire.
Thank you so much. |
Eric's response: The questionnaire appears to be free
for student/research purposes. For industry there is a nominal license
fee.
www.isometrics.uni-osnabrueck.de/index.htm |
Top |
| |
|
Question: How can we pick out flaws in our documentation
using Usability Testing? |
Eric's response: You can certainly include the documentation
in your usability test. Simply make the materials available and allow
users to access them as needed. I think you may find that they are rarely
used. But if they are used there should be plenty of insights.
Alternatively you can specifically test the documentation by giving the
user tasks such as "Find the installation procedure in the manual." |
Top |
| |
|
Question: I would like to know more about usability
testing, especially how to conduct a usability test for a Web site. Thank
you. |
Eric's response: User-centered design takes much more
then just a usability test. Usability testing is the way that we 1) fine-tune
our design and 2) verify that the design is working. We have two types
of testing that correspond to these two objectives. Formative testing
helps us fine tune designs. It's a dentist having you bite down on carbon
paper and grind your teeth to see where the teeth must be smoothed out.
Like this, we have users try to use the site and see where they get stuck.
In summative testing we have criteria for success and we test the user
to see if these benchmarks can be reached. For example: we know a competitive
site allows users to sign up for an account in 3 minutes. We test to ensure
that our new site is more efficient. |
Top |
| |
|
Question: I am part of a team developing a new Web site
and application. The marketing manager would like to "test"
many versions of the home page in a live environment – meaning once
the site has been built. The home page is not just a static page, but
incorporates many programmed elements that, with each change, will require
substantial programming rework. The testing aspect will be critical to
determine what layouts, colors, and ads/offers work best to boost sales
and the marketing manager feels that testing it with "real"
users is critical. Are there any testing methods that could be used to
yield valuable results and eliminate the need for these real-time changes
to the live site? |
Eric's response: The live site testing idea is very
useful. There is no way to get data as good as live site data; particularly
if you have a split site model. By running two versions of your site (and
randomly assigning people to each version by cookie) and comparing results,
you will really have unbeatable data.
However, before getting to the live site work you SHOULD apply the developmental
methods. This includes a user-centered design process. It includes in-depth
data gathering and application of research-based principles. It also includes
formative usability testing. In this way the first version of the site
will be excellent and you can use the live site process to continuously
improve it. If you do not do this you will find you are trying to evolve
the site from a weak start. This will be long and painful. |
Top |
| |
|
Question: What factors influence how quickly Web pages
are retrieved from the Internet? |
Eric's response: There are a whole set of factors that
affect the bandwidth. This includes the type of connection (dial-up, various
broadband choices, etc.) There are also factors related to the positioning
of the server and its level of connection to the Internet backbone. Then
there is a major factor of the page weight (number of bits in the page
design).
Interestingly, the actually page display time is not as important as
we initially thought. The way that the page displays is important (if
you can get the user working, say by displaying test first, then the fact
that the download is continuing is not that important). In addition, a
useful and engaging page will be SEEN as faster, even if the actual download
is slow. |
Top |
| |
|
Question: How do I define and relate usability to a
Web site for kids which are 4 to 6 years old? |
Eric's response: It does not really matter the nature
of the users. The process for kids is the same as any other group. You
complete a user-centered design process. Identify their needs and structure
the site to fulfill these needs. Focus your efforts on the important user
performance and experience objectives. That means speed, accuracy, self-evidency,
satisfaction, and/or safety. In the case of a kids site I would focus
on safety, self-evidency, and satisfaction in most cases.
Designing for kids in this age range certainly has specific challenges.
Consider their reading capability, attention span, domain of interests,
etc. |
Top |
| |
|
Question: We would like to measure how much time savings
there is before our portal implementation and after. Is there any industry
average on how long one mouse click takes? We thought if we could measure
how many clicks were saved with the portal we could put a time/dollar
figure to that. |
Eric's response: Indeed you can measure the average
time for a mouse lick or keystroke. However this is a small fraction of
the time spent at a portal. The primary time is spent scanning and reading.
So measuring the physical movements won't be very interesting. If you
want to make a comparison to show the change, develop a 'basket' of representative
tasks (e.g., "find new employee's phone number", "find
HR site"). Then run a usability test contrasting the original situation
with the portal. I think this will be most meaningful. |
Top |
| |
|
Question: I'm a college student in an internship for
New Media. I have been given the task to write my *own* user testing method
for a small scale software. I'm not very familiar with in-depth testing
methods – are there some guidelines that I can follow to write my
own usability test?
Thank you. |
Eric's response: Well if you want to develop your *OWN*
method don't ask Eric. I only know methods I learned from other people...
And maybe a couple I developed myself.
If you want a book for "inspiration" I can heartily recommend
Handbook of Usability Testing: How to Plan, Design, and Conduct Effective
Tests by my old friend Jeff Rubin.
If you want to just read one sentence...
Make up a set of representative tasks and tell a sample
of representative users to each complete the task without help, while
you watch, and then see where they get stuck. |
Top |
| |
|
Question: I am a Corporate Data Manager and would welcome
your views on the following. I recently requested that a supplier currently
developing a COTS prototype for the organization provide feedback on the
level of compliance against the corporate policy and standards. One of
the policies to be looked at was the Usability Policy. In the comments,
I stated (perhaps unadvisedly) that implementing a COTS solution puts
the organization at a disadvantage regarding usability as a truly usable
system can only be achieved by designing it to meet the needs of the specific
user group(s) involved, i.e. bespoke. And that using COTS means that a
compromise must be reached between the solution offered and the users'
perception of how information should be presented. This was stated in
the context that the design of the COTS product could not be customized
to any large extent. I would welcome your views on this as the project
manager has taken umbrage and said that COTS are "more usable than
bespoke". |
Eric's response: Audrey, I am sorry that your supplier
took offense. However, in general you ARE correct. Without question the
wider the target population and taskflow, the harder it is to make a solution
that is ideal for all users. Since I think it is safe to say that COTS
packages must accommodate a greater diversity of users, taskflows, and
environments, they have a much harder time being usable. On the other
hand, if a COTS development company invests enough in usability work their
result can be BETTER then a company doing a customized job without the
benefit of a mature usability operation. So rather then depend on generalizations
it is best to evaluate the supplier based on the quality of their process
and product. Check carefully how well the product meets your needs. |
Top |
| |
|
Question: One of the fundamental principles of usability
is to "know your user". Giving specific examples, what is the
importance of user profiling as part of interface design. |
Eric's response: Very fundamental question! Let's take
some examples...
A medical application for doctors can have more complex language then
for patients. It can also include more graphic images and blunt assessments.
A site selling banking services for high net-worth customers will be
different then a college credit union. It will have to have more sophisticated
investments available. It will be able to use more complex terms. The
font must be larger (as the users will be older and therefore generally
have worse vision). The style must be different (due to the required difference
in branding and also due to differences in population preferences).
An online marketplace might work for customers in the USA. But in Japan
it fails because they do not use credit cards. Also they make decisions
in a group process that makes the timed auction difficult. Of course translation
to Japanese and Yen is important. But color preferences are also different.
In user-centered design we design for specific ranges of users. The wider
the range, the harder the job. Without knowing the user, the design has
little hope of success. |
Top |
| |
|
Question: My company is finally redesigning their corporate
Web site. They are striving to be more "customer-centric". Toward
this end, they plan to hold focus groups, and focus groups alone to determine
our user needs, and from that determine feature sets and functionality
requirements for the site. The plan is to hold the focus groups and then
let internal line of business reps determine the feature set for the new
site.
I am a big proponent of User-Centered Design. I am thrilled that our
executives are finally striving to be customer focused, but I am petrified
of relying of focus groups alone, and then letting internal staff run
with the data.
I know focus groups often produce misleading, biased results and I foresee
them causing some big problems in the long run unless we adopt a more
robust UCD process that includes other research methods such as contextual
inquiry, task analysis, user profiles, etc.
What advice can you give me, or what resources could you point me to
that will help me convince my bosses that we need a larger variety of
research techniques to accurately get a picture of our target audiences? |
Eric's response: Well it is common today and a good
thing that executives are understanding that usability is mission critical.
However, as your case illustrates beautifully; this understanding of criticality
does NOT translate to an immediate understanding of the HOW of user-centered
design. Their focus group approach is virtually certain to fail. It is
pathetic. What is WORSE, you may find that they decide that user-centric
design is of limited value when they find that a focus group does not
translate to a great Web site. (Sigh).
The very best thing you can do at this moment is get the executives educated.
Don't expect them to listen to you from within the organization. They
will only listen to ultra-credible outsiders. My book "Institutionalization
of Usability: A Step-by-Step Guide" is a good bet. It gives a clear
outline of the requirements for mature usability work. It also specifically
addresses the focus group problem. Also consider getting them in touch
with a usability consultant who can guide them to a proper understanding
and plan. |
Top |
| |
|
Question: What are the key aspects of usability and
how might they apply to a test plan for a customer booking database? |
Eric's response: The key MEASURES of usability are speed,
accuracy, training requirements (or self-evidency, satisfaction, and safety.
These then become the things you may want to test in your session. Specifically
test how long it takes to do the major booking tasks. Can people in fact
accomplish them? Check the error rate. See what training they need to
operate the system. Also, you might have participants rate their feelings
about the interface. |
Top |
| |
|
Question: Are there any standardized tests/research
that suggests at what point that a college or university student's handwriting
speed will start to adversely effect their exam performance?
I am asking this from an angle of testing for exam concessions e.g. extra
time. |
Eric's response: In the USA students with significant
handwriting impairment get extra time in some tests. You might check the
rules used. However, I think they are quite rough and probably have little
scientific basis. I suspect they amount to a 'note from a doctor' approach. |
Top |
| |
|
Question: I'm currently developing an administration
solution for a small gardener's company. Is it a good strategy to actively
involve my customer in the development process, meaning to turn him into
a usability tester? I was considering to provide him with the functional
specification at first and give him frequent access to the evolving versions
of the software in terms of testing and evaluating. Do you think that
it would be appreciated from the customer, or would it leave an impression
of nonprofessional acting. |
Eric's response: Involving your customer is certainly
a good thing. Research shows that more involvement is good. Also in Germany,
the DIN standards actually require involvement of users. However, the
question is HOW to involve the clients and users. The method you described
may not be very effective. I recall acceptance testing which involved
mailing a 4-inch-thick requirements document to the managers of the users.
They could not really understand the document and just ignored it.
Your client should be involved in briefing you on the direction and vision
of the interface. Then they should help work out the strategy for tuning
that design based on the data gathering you do. That is about it.
Users should indeed be sources of data and should be used as test subjects.
It is however VERY important not to have a single user involved repeatedly.
Use a set of users to gather data. Then use a DIFFERENT set for usability
testing. Once a user has participated they become 'polluted'. Their opinion
and understandings are changed by their experience with your project.
So involving the users is VERY professional. But you have to involve
them properly. Always remember... USERS ARE NOT DESIGNERS. |
Top |
| |
|
Question: Hello to all at HFI. I am searching for research
on what, if any, effect privacy policy statements have on users of online
account management sites. Do people avoid using sites that don't have
them? Do they consider sites that do have them more worthy of trust? I
would appreciate any insight you can offer. Thanks. |
Eric's response: This is an interesting case where many
people SAY that it is important, but almost NO ONE actually looks at it. |
Top |
| |
|
Question: I am working on developing a usability test
plan for our product's UI. I have a list of heuristic evaluations but
do not really know how it's done. Do we take a checklist and go screen
by screen and put down our evaluations? I would like to know if there
is any source that would give me some practical knowledge.
Thanks a lot! |
Eric's response: The idea of a heuristic evaluation
is to go through the screens considering each point in the heuristics.
This is not very effective (catching perhaps 30% of issues). I suggest
at least getting yourself some training in UI design. Then you can review
the screens based on the multitude of far more specific principles and
research findings. This will certainly be better.
I also wonder, based on your question, if you have a full user-centered
development program. Reviewing screens is a hopeless way to compensate
for lack of systematic usability engineering practices. I would hope to
see definition of target users, data gathering with actual users, task
analysis, task design, etc. I would hope that about 10% of the development
community is comprised of usability practitioners. If this is not the
case you can pretty much rest assured that the design is poor. |
Top |
| |
|
Question: I would like to find out what the latest research
has to say about the number of users, and type of users necessary for
testing products for older users. Do we include users with all types of
ailments e.g. visual, tactile, cognitive and try to optimize the product
for all types? If the effects of aging are gradual, then the users chosen
never remain the same, so how can we be sure that the population of older
users is being represented accurately in our sample? |
Eric's response: Well actually you are raising two separate
issues. In testing for aging populations it makes sense to test five to
ten older users. There is MUCH controversy over exactly what is considered
'old'. This is best determined by your actual expected population. If
you will indeed HAVE a significant user set over 80, then test users over
80.
Specific ailments are different. They are not entirely age related (though
there is a correlation). You can certainly find young users with almost
every possible impairment you might find in older users. In general we
manage this not so much by testing, but by designing with those limitations
in mind. |
Top |
| |
|
Question: Do you know of any guidelines or suggestions
that would be useful in creating an online survey? |
Eric's response: Sure, happy to help. First be very
clear the question you are trying to answer. Surveys can only measure
what people report. So a question like "Is this efficient?"
is out. Consider if you are just getting opinions ("I like it"),
or gathering facts ("Select the reason you came to this site").
Are you measuring against something like a past version of your site or
a competitor's site.
Consider carefully WHO you will include and how you will get enough people
to respond. As the response rate goes down you start measuring from a
highly selected group (generally only people with a lot to say either
good or bad) and you start missing the true mainstream viewpoint.
Design the questions so they are not leading. If you strongly believe
something ask it in the opposite way. For example if you think a cartoon
site is just for kids ask "How much of this site usage is by the
adults in the family?". Also be sure the questions are very specific.
Avoid general questions like "Did you like it?". The answers
won't mean much. Finally, be aware that a survey can only serve as a small
part of an overall data gathering process. |
Top |
| |
|
Question: I would want to get more information about
typing proficiency? What standards are there, or what are required? As
I understand, some employers require 60-70 wpm, but how to understand
the practice 30-40 words per minute and the given information in this
Web page related with the fact of 150 wpm.? Thank you in advance. |
Eric's response: There is plenty of data on typing speed.
150 wpm is probably near record breaking and therefore of little interest
to interface designers. A typing test showing about 60 wpm is often used
to demonstrate typing proficiency. But forget having people type that
fast for 8 hours a day! That is simply a test to show people have touch
typing skill. The actual long term productivity is very much a function
of taskflow and interface design. |
Top |
| |
|
Question: Has there been any usability research done
that shows that online support group users are less likely to continue
communicating online after having the opportunity to meet in person? I
am currently involved in a schizophrenia Web site support group study
that includes one face-to-face psychoeducational workshop where all participants
will, in fact, meet face-to-face, losing their on-line anonymity. |
Eric's response: Janis, I am not aware of any research
on this topic. However, I expect that meeting people in person will change
the character of the interaction. This may not at all reduce the amount
of online interaction. Just look at the number of classmates and coworkers
who interact online regularly. In a given instance the relationship could
be enhanced or diminished based on the character of the physical meeting.
However, there may be a therapeutic value to the anonymous support group
which could be lost. I would be very much interested to hear your experience. |
Top |
| |
|
Question: RE: Quantitative evaluation of interface effectiveness.
Currently, I am working on a project related to the evaluation of risk
to the operation of the International Space Station for NASA. I would
be interested if in the process of your usability analyses you ever came
up with relative quantification of the solutions, i.e. this particular
arrangement is ten times better than that one? Most Human Factors processes
seem to recommend changes that should be better, but no estimate of their
effectiveness. Is HFI any different in their evaluations? It would very
interesting to hear from you.
Thank you |
Eric's response: Tony, quantification of risk has a
long and hallowed role in usability work. We can often take alternative
taskflows and estimate the risk numerically. We use error probabilities
(like ones in the AIR database which lists various actions and the probability
of errors in each). It is true that many times the best choice is pretty
obvious. But when there is a close choice, or when you need to strut your
value for management, the numbers help. |
Top |
| |
|
Question: RE: Patient evaluation instruments for individualizing
e-health technologies and techniques.
What evaluation tools exist for a health care practitioner to assess
a patients competency, capability and preferences for the use of the various
e-health technologies eg. phone, e-mail, video, etc? The goal being to
meet the IOM patient centered criteria and improve the efficiency, cost:
effectiveness and safety of the system. |
Eric's response: FANTASTIC question! The answer: None.
I recall Arni Lund trying to create a team to just DEFINE dimensions of
limitations in using technology. I don't think it ever happened. So I
believe there is not even a standard description of the types of limitations.
Sure, usability practitioners have some general ideas. We know about various
levels and types of limitations in sight, hearing, motor control, cognitive
capabilty, etc. We do try to design to avoid gratuitiously adding barriers
(like the incredibly stupid Control/Alt/Delete required to get into certain
windowing environmrnts). But there is no specific instrument for ability
to use various technologies. I guess clinicians will have to do it based
on thier common sense and adjust it based on their patient's expeieince.
But it's a DAMN good idea to have a formal assesment instrument! |
Top |
| |
|
Question: I am designing an Intranet 'portal' for a
relatively small group of users that will serve a very defined task set.
My questions is, do I really have to go through all the hurdles of User
Centered Design (contextual inquiry, focus groups, card sorting, personas,
affinity diagrams, etc., etc.) to achieve my goal, or can I simply informally
interview my user base to get the info I need and design from there?
The site will be a link between common tasks for contract managers in
healthcare, and outside services that provide information necessary for
completing the tasks. For example, one known common task would be "Determination
of efficacy for products x, y and z". Conceptually, there will be
some sort of link for that task, and under that links to the specific
services provided by the outside research agencies that my company partners
with to provide that information.
To me, the two biggest issues here are task analysis – determine
which tasks are consistently undertaken across lines of business, and
information architecture – finding a common name for the various
tasks. Then, I would simply determine the outside service that best meets
the needs of each task and provide links via a Web site.
I am just wondering if oversimplification of the ideal "UCD process"
will result in inaccurate information ultimately leading to an ineffective
site design. |
Eric's response: Jeff, you raise two interesting issues.
The first is what to do with projects that have low potential value from
a usability viewpoint. With a small number of users and little of a really
mission critical nature in their work, there may be little you can do
to really make the company more money by working on this sort of project.
In fact, you may be better off spending more time on the public sites,
HR, and Extranet, than on this type of very small and insignificant site.
Let the Web designer go ahead and do the best he / she can, based on whatever
usability training and standards they have. MAYBE quickly review their
work. Other then that... FORGET IT.
The other issue is the onerous nature of task analysis. One of the most
common mistakes I have seen in a determined but less experienced usability
practitioner's work is too much detail in task analysis. I have seen shelves
of task analysis; without a single useful insight gained and with too
much stuff to EVER really use. My general secret is "Just have enough
detail in your task analysis so that you could do the work yourself."
It's a good rule of thumb. |
Top |
| |
|
Question: What is the standard for using wildcard functionality?
– *, ? |
Eric's response: Unfortunately, there is no standard
and most people don't understand wildcards. Only a TINY fraction of searches
use wildcards. Those that understand the idea of wildcards will usually
use them infrequently and therefore not remember much about the syntax.
There are wildly different wildcard syntaxes used.
This means that you must have an '"advanced search" facility
that helps people with advanced searches. It is actually better to have
the advanced search provide the wild card and other capabilities not as
a syntax but as a form-filling interface. We have very good data that
Boolean logic confuses most users. So hide this and make it as simple
as possible.
For very advanced users you may want a wildcard functionality. The *
is reasonable for any number of characters. The $ is sometimes used for
only a single character. You also need a "NOT" capability in
most cases (generally a minus sign "-" or the word "NOT"). |
Top |
| |
|
Question: Hi Eric. Our design office is facing a controversy
that I hope you can help us with. What would be the best way for our company
to test the effectiveness of Flash-enhanced Web pages? Some of our ad
response pages (that are accessed from a URL that we place in our magazine
ads) will incorporate Flash animations. We want to measure customer response
to these pages, to see if the animation helps to reinforce our brand/visual
identity.
Do you have any advice for the best method to solicit and measure customer
feedback on Flash-enhanced Web pages? |
Eric's response: There are several approaches to measuring
advertisement effectiveness. If there is a direct response involved, you
can create alternate versions of the ad and measure the differences in
response rate. For example, make a given ad in both a Flash version and
a straight HTML verson. Then check the difference in response. This is
probably the best method.
If a differential in response can not be measured (as in advertisements
oriented toward brand recognition only), then you will need to make less
direct measurements. You can have people rate a series of ads and compare
their preference for ads with Flash. You can show them a whole set of
ads for various companies and then see if they remember yours (doing this
with Flash ads and then with others). These less direct measures can still
give you useful indications. |
Top |
| |
|
Question: Why is it important to test usability on e-commerce? |
Eric's response: Prema, here is a related question:
Is it important to debug the programs used to create an e-commerce site?
Certainly you COULD code the site and just put it on the Web. You might
hope that it would work well. But you would be assured of a site that
has points of failure. Are these points of failure acceptable? They frustrate
users. They reduce sales. They hurt your company reputation. With that
it is silly and unprofessional to skip debugging and people rarely do
it.
It is actually WORSE to skip usability testing then skipping debugging.
If you skip usability testing you are completely assured that you will
have a site with points of failure. The USER will be unable to complete
activities. The user will be frustrated. Sales will be roughly cut in
half. Your reputation will be damaged.
But what makes it worse is this – with code problems kind users
are likely to let you know there was this error message or that broken
link, but with problems of usability users are less likely to help. They
will just feel frustrated and feel that it is unpleasant to do business
with your company. A few will complain. But the complaints will be nonspecific
("this site sucks") and if they do make recommendations they
will rarely recommend the correct solution (e.g., If users ask for 'Search'
it is almost always a symptom of poor navigational structure).
If you have EVER created an e-commerce site without testing, it was surely
a big mistake. Don't do that again. |
Top |
| |
|
Question: Is there any empirical data which suggests
the ideal number of panelists needed to make results from an average Web
site usability study most relevant?
At SportsLine.com we gather the following data in our studies through
automated means:
1. Click Stream Data
2. Qualitative Data
3. Quantitative Data
We are interested in the most effective yet economical sample size. We
would like to do studies more frequently throughout the year on a quite
limited budget.
Any feedback would be appreciated. |
Eric's response: This is a very hot topic and it is
the subject of a number of our newsletters. But let me give you a summary.
First, there have been recommendations for usability tests of 5 people.
Other experts recommend up to several hundred. So practitioners have a
right to feel confused. But remember that the usability test is not attempting
to create a statistical proof of a small difference. It is rather trying
to identify places where people get stuck in trying to use your design.
For this goal we need perhaps 12-30 participants. You need more if there
is a lot of diversity in user population. You need less if the users are
very homogeneous. But that range will work most of the time.
Your click stream analysis and survey results all serve as useful information
to supplement your evaluation. But your usability testing will give the
most powerful results. |
Top |
| |
|
Question: I would someone to explain what is meant by
'User Characteristics' in relation to Human Computer Interactions. |
Eric's response: To do a good design you must know the
user characteristics, taskflow, and environment. The user characteristics
are critical because they indicate the limit and nature of the interface
designs you can use. For example a very novice user will have more trouble
with an image menu then a simple button menu. They also indicate the level
of reading skill, general intelligence, and expertise. These all make
a difference in the design decisions. In addition to capabilities we must
also study the user's motivation. We design interfaces that eliminate
things users hate to do, and increase the things they like to do. We make
designs that attract users because they resonate with the brand characteristics
they care about.
Be careful when you consider expertise. This actually can mean three
different things. There is expertise with computers. There is expertise
with a specific application (for example a user who operates an interface
full time will soon become an expert). Then there is also expertise with
the domain (for example a user may be terrible with computers –
they may use an interface once a year for 1 hour – but they may
be very knowledgeable about insurance). |
Top |
| |
|
Question: I'd like to find out more information about:
"The Bollywood Technique: an innovative approach to usability testing
with subjects who don't like to criticize."
Do you have any visual information that shows how users are different
between regions and/or countries (if they are)?
We are testing internationally with users in Australia, NZ, Singapore,
Thailand, and India and are wondering about using this method with all
the users, or should we modify the script based on country.
What are the pros and cons with the data from this technique?
Looking for more details.... |
Eric's response: Thanks Pamela. The Bollywood
Technique was developed by Apala Chavan, HFI's Managing director in
India.
The method is appropriate for countries with strong tendencies to avoid
criticism. I would suggest it for Singapore, Thailand, and India. Australia
and New Zealand will do fine with typical test protocols. However, you
may wish to use the same Bollywood process for all to reduce costs and
make collation of data easier. I do not think the method will prove troublesome
in Australia or NZ. |
Top |
| |
|
Question: Could you give me some tips how to conduct
an informal usability test. Is there a checklist that I would need to
follow. Could you also suggest me some sites where informal usability
is conducted for a online billing system etc... |
Eric's response: I am not familiar with a site that
conducts informal usability testing for online billing. But if you want
to try it yourself I can give you enough to get some interesting results.
Be aware that a professional study will offer more then you can get with
this approach.
1. Schedule about 12 people for testing. Make sure they are representative
of typical users. Do not even THINK about using other company employees.
Tell them it will take an hour and that you will pay them $75 for their
time.
2. Make up a list of tasks that users would typically perform. Make up
enough tasks so that they will take about 45 minutes to complete. Make
sure that the wording of the task does not give away how to use the interface.
So if the dropdown is labeled "Edit", the task wording should
say "what if you want to change this word". Do NOT say "What
if you want to edit this memo?"
3. When people come in, have them sit in front of the computer and you
sit next to them. Tell them you will have them try out several tasks on
the system. Tell them "Read out loud, talk out loud, and tell me
what you are thinking".
4. Have the user go through each task. You hold a set of screen printouts
and when people get stuck draw a red circle around the part of the screen
that is having the problems.
5. When you are done, if you have three or four red circles around a
part of a screen printout, you have identified a serious problem. |
Top |
| |
|
Question: I am interested in tips on how to perform
usability testing on users using a tutorial, when the goal is to evaluate
the tutorial. Is it recommended that the user complete the actual and
complete tutorial as you observe? Or, should they try segments of the
tutorial and then follow-up questions? I need direction in this area because
I see it as different than simply testing the ability of the user to complete
a task while using a finished product. Similarly, recommendations for
testing a help section of a product? |
Eric's response: Testing help and tutorials has two
very different components. One aspect of testing is evaluation of the
usability of the facility. You must test the user's ability to navigate
and complete interactive operations. The second issue is the help or tutorials
content. Does the facility convey the learning objectives.
To test usability have the participant complete a series of sample tasks
and monitor their performance and subjective impression. Make sure that
these task are representative and include significant navigational capabilities
(like leaving and returning to the tutorial).
To test the learning objectives, participants must complete a full cycle
of usage and then complete a test. For help this may just mean getting
an answer from the help system. For tutorials you might want people to
complete the whole program and then get tested. |
Top |
| |
|
Question: I'm doing some research for my degree in ergonomics
and I'm searching for expert systems that have been developed to help
usability professionals in the evaluation of interfaces. Do you know of
any? Where can I find this kind of information?
Thanks for the help! |
Eric's response: There is a long history of work in
this area. There are indeed a number of facilities available. A good example
is "Bobby" which is a good facility that checks Web pages for
compliance with accessibility issues. The ability to analyze a page directly
for ergonomic issues has never been terribly successful. There are specialty
tools that can help check various issues. For example AnyBrower.com will
check for browser compatibility. Colorfield Insight will check for usability
by color blind users. But currently, there is no facility that comes close
to the quality of a trained reviewer following a systematic process. |
Top |
| |
|
Question: I have to write test standards and test methodologies
for my new company. My company buys off-the-shelf software and makes in-house
modifications to meet company specific requirements.
I wonder how I can introduce usability testing here since the screens
have already been designed by the vendor. |
Eric's response: Good question! The most value you can
provide is in helping to SELECT the software being purchased. If it starts
with poor usability, then you are unlikely to be able to fix it in a modest
customization. Then make sure that the customizers have at least some
training in screen design. THEN, you can do a final summative test (to
check that it really works and is meeting the objectives). But this last
test is unlikely to help much. |
Top |
| |
|
Question: The project I am working on is requiring an
in-house validation for a new mechanical enhancement. I need to write
up a test scenario, then a satisfaction questionnaire. How many questions
do I really need because of the few users that will be present (Users:
4) to justify my data? |
Eric's response: There has been much discussion of the
number of subjects needed for usability testing. Various positions have
been taken. But 4 is too few. You cannot get a range of users. You cannot
get reliable results. I would think 12 would be the minimum. Without enough
subjects, no number of questions will 'justify' your data.
With at least a minimum number of subjects of the right type you can
begin to look seriously at the results. With subjective rating questionnaires
you do not need more questions to 'justify' the results. You can have
a single question. The value of additional questions is primarily to dissect
out the ASPECTS of the subjective experience. For example, I might ask
overall satisfaction. But if it is poor, what does this mean? So to know
more I might ask about perceived speed, aesthetic appeal, and simplicity.
Now if they do not like it I could find that the main problem is that
the design is ugly. I can then use graphic arts staff to improve it. The
other reason for asking more questions is to make sure that your results
are not an artifact of wording confusion. So you might ask if it is aesthetically
pleasing in several different ways. You can then combine the aesthetic
items to make a single assessment. This will tend to improve the reliability
of the questionnaire. Generally, I would provide about 20 subjective rating
questions to get a good understanding. |
Top |
| |
|
Question: How many users know how to cut and paste?
How many of these cut-and past-users would think to cut and paste sections
of their resume on a Word document into an online CV form? |
Eric's response: I have not seen any data on the general
rate of cut-and-paste knowledge. Cut and paste is NOT one of the core
skills you can assume. Something like basic mouse operation can pretty
much be assumed based on the fact that people got to your site. But cut
and paste is not like that. You might consider the specifics of the population.
That might suggest the percentage that will know about cut and paste.
There is a second issue. Even those people knowing about cut and paste
my not think to use it to fill in a form. It might seem like an obvious
solution to you. But people will often attack tasks with mathematically
sub optimal procedures simply because they do not think of the better
way. So be sure to suggest the cut-and-paste method to customers, and
perhaps provide a link with instructions on how to do it. Let me know
what percentage of people use that link. That way I won't get stuck without
a full answer again. :) |
Top |
| |
|
Question: Which usability testing software do you recommend? |
Eric's response: In fact HFI does not use any event
monitoring package. We find such packages of little use for practical
testing used in design work. They indeed can capture every keystroke and
mouse movement. But then who can pay for the detailed analysis of all
that data? We do simple video records and have the human factors specialist
keep a log of issues and insights.
In the area of remote testing we have simply used WebEx or the Question
Mark exam software for simple tests and surveys. We have also used NetRaker
though as yet I remain unconvinced that this and other remote testing
packages are fully ready to integrate into the design process. Right now
they seem appropriate for specialized situations (as when the users are
in many different countries and therefore hard to test). In any case we
are tracking the remote testing software closely and hope to see it designed
and practical for mainstream usability work. |
Top |
| |
|
Question: Can you tell me which ear (right & left)
goes to which side of the brain. Also what is the function of that side
of the brain (logic or emotion). |
Eric's response: Rick, the left ear goes to the left
side of the brain, and right to the right. It is the eyes that are a bit
odd. The optic chiasma shunts the left side of both eyes to the right
side of the brain and the right side of the retina to the left side of
the brain.
However, you may be thinking of processing of auditory information and
speech. This is handled in the left hemisphere.
While there is a great deal of research and subtlety, you can think of
the left hemisphere as science and the right as art. |
Top |
| |
|
Question: Eric, with software development teams moving
towards agile development, how do usability testers keep up with this?
Are we moving towards agile design and usability testing? It seems that
usability testing is getting overlooked in agile development method. What
is the best way of usability testing for agile development projects? |
Eric's response: Nishant, usability testing is essential
to agile development. But we need to move toward more dynamic testing
methods. The long formal simulation testing at the end of the process
is less our focus. Instead, use more dynamic methods. Do quick studies
that take a couple of days. Use remote testing. Change the design after
every few tests if appropriate.
The one danger of the agile manifesto is that it tends to rely on collaboration
with customers. This sounds wonderful, but does not work. Customers are
not designers. The do a terrible job of creating user interface designs.
In addition, the customers selected for collaboration are almost never
representative; and even if they are representative they quickly learn
too much. They learn the design team's viewpoint and models. They will
understand and like a design that will not actually work well for the
users.
So make usability testing a routine practice. Do repeated quick tests.
Test early in the design process. Use current methods that are fast and
efficient. Highlight for the developers each time the test reveals a problem
that would have been missed. |
Top |
| |
|
Question: How many users |
