|
Choose the number of subjects to match the expected probability of finding
a given problem. Big problems need fewer subjects. Subtle problems need
more subjects. Experience indicates 10-20 subjects would provide insight
into the problems that we anticipated. Our intern tested 16 subjects with
telephone experience and varied educational background and gender. He
used 2 of the sessions to learn to write the subject's comments rapidly
and concisely. We used data from the following 14 subjects. Our intern
videotaped five of the interviews in case we wanted to demo the process.
If needed, provide training to give your subjects the same expertise
your actual users have. (If you expect a specific background, then recruit
– and pay – subjects from your user population.) In our case,
we only needed experience using a telephone and age enough to qualify
for a telephone card. Here's a subject selection summary:
Subjects
- 14 subjects from a university setting.
- Represents cross-section of US population
- 4 Female (29%)
- 4 English as a second language (ESL)
- Ages 41-50:4; 31-40:1; 21-31:7; 17-20:2
- PhD:1; MA:2; BA:4; HS:7
- Homogeneous: ESL subjects had similar satisfaction ratings as English
(5 NSDs)
Comment During data analysis (see below) we wanted to
see if ESL made a difference in how subjects felt about the IVR menu.
Therefore, we used a statistical test to check for differences between
the average scores on each of the 5 satisfaction ratings (given below).
"NSD" means No Significant Difference would be found 19 times
out of 20 similar tests (the so-called "95% confidence" rating).
We used the t-test for unequal variances found in Microsoft Excel. You
don't need such confirmation if your own group of test subjects has no
particular differentiating characteristic.
|
