There is a considerable amount of information published on the Web that is intended to be read by someone. There is evidence that much of the information may be too hard to read and understand for typical readers.
Baker, Wilson and Kars (1997) reported that the readability scores of most articles in the 'Health Reference Center' ranged from 10th to 14th grade levels. Another study (Graber, Roller and Kaeble, 1999) included text-based information from commercial, academic and government sites. They found that the reading material averaged the 10th grade level. In a more recent study, a group of researchers (D'Alessandro, et.al., 2001) conducted readability analyses of pediatric patient education materials on the Web, and concluded that the information was not written at an appropriate reading level for typical users.
Readability formulas have been developed to assist writers in preparing information. These formulas provide a means for estimating the difficulty a reader may have reading and understanding a paragraph, section or entire document on the Web.
The first readability formula was developed over 80 years ago, and a number of formulas have been developed since that time. These formulas originally were designed to help classroom teachers choose textbooks for their students. Currently available, computer-based readability formulas include:
Readability results will vary depending on which formula is used. For example, the Flesch-Kincaid tool often returns a score two to three grades lower than other formulas. Osborne (2000) proposed that grade level equivalent scores tend to be accurate only by plus or minus 1.5 grade levels.
There are numerous factors that affect how easy, or how hard, a given document is to read and understand, including sentence length, word choice, layout and formatting, overall organization of the content, use of illustrations, etc. However, most readability formulas consider only two factors:
(a) the number of syllables (or letters) in a word, and
(b) the number of words in a sentence.
Because most readability formulas consider only these two factors, these formulas do not reveal why written material is difficult to read and comprehend. Most of the important attributes of writing that contribute to reading difficulty have not yet been quantified. Fortunately, many of the difficult-to-measure attributes are highly correlated with the two factors that can be easily measured.
Readability formulas are most useful as predictors of reading difficulty. Klare (1975), in a review of readability formulas, concluded that "as long as predictions are all that is needed, the evidence that simple word and sentence counts can provide satisfactory predictions for most purposes is now quite conclusive."
A document classified as highly readable solely on the basis of a readability formula could be a disorganized disaster—or contain no content at all. The following paragraph has a calculated readability of the 12th grade:
Qwerty uiopas dfg hjkl zxcvb nmqw ertyuio pas dfghj klzxcvb nmq werty ui opas dfgh jklzxc vbnm. Qwertyuiop as dfgh jklz xcvbn mqwe rtyui opas dfghjk lzx cv bn m. Qw ertyu iopas dfghj klzxcvb nmqwert yuiopasdf ghjk lzxcv b nmqw ert yuiop asdf gh jk lzxcvbn m. Qwerty uiop asdfg hjklz xcvbn mqwe rtyuiop asdfgh jklzxcv bnmq wert yui opa sdfgh jklzxc vbnm qwerty uiopas dfghj klzx cvbnm.
Obviously, readability scores depend on the writing style rather than the content of written material. These stylistic features are under the control of the writer.
As general rule, it is better to write a document at a readability level that is below the reading skill level of the intended audience. Ideally, the reading skill level of intended readers would be based on the results of a standardized reading test (e.g., the Nelson-Denny Reading Test). This is usually reported as a grade level, i.e., "95% of the users in the target audience read at an 8th grade level or higher."
Usually it is not practical to administer a reading test to all potential users. An estimate of the reading grade level of the intended audience can be obtained by considering the users' education level. An average eighth grader is assumed to read at an 8th grade reading level, and a twelfth grader at a 12th grade level. People who have completed college are assumed to read at the 16th grade level.
In general, people with more education have better reading skills than people with less education. However, the actual reading ability of a person does not always match his or her educational level. Coke and Koether (1979) collected reading scores for over 200 company employees. The group averaged a 12th grade education, and 95% had reading test scores above the 10th grade reading level. Hilts and Krilyk (1991) reported that adults read at least one or two grade levels below their last school grade completed.
Summarizing several studies done in the United States and Canada, the average reading skill level was estimated to be at around the 8th to 9th grade (University of Utah Health Sciences Center). However, this same study found that about one in five adults had a reading skill level at the 5th grade level or below.
By comparing the calculated readability of a document to the reading skill level of typical users, a writer can estimate whether a document has a good chance of being read and understood. The readability formula can be used as a predictor of difficulty, but should not be used as a diagnostic tool. Readability formulas do not provide information about how to make instructions more comprehensible. For example, a document with a high readability level might be made more readable by changing its format rather than its writing style.
To make written texts truly readable, Website designers should apply all the principles of clear and simple writing. Even though using short words and short sentences will result in lower readability scores, this does not guarantee that a document will be easier to read.
Finally, there are times when document readability issues are not as important as other issues. Klare (1975) found that in circumstances where time is not crucial and readers are highly motivated, the readability of a document was of less importance. Coke (1976) provided evidence that readability was not as important when readers were looking for specific information as it was when users had to remember that information.
Incidentally, the Flesch-Kincaid readability level for this article is 12th grade.
Apala Chavan is the managing director of our office in Mumbai India. She presented her fascinating new testing method at the CHI convention this year. She called it "The Bollywood Technique" and I'd like to share it because I think we can all benefit.
What is the main challenge when you are usability testing in Asia?
In Asia it is impolite to tell someone they have a bad design. It is embarrassing to admit that you cannot find something. So it is very hard to get feedback.
Apala tested a site that offered railroad tickets for sale. She used the conventional simulation method and got little feedback. She could see that users were not succeeding. But they would not willingly discus the problems.
Apala then tried the Bollywood method. Now Bollywood is the Hollywood of India. They make more movies than Hollywood. They are famous for movies that have long and emotionally involved plots. The movies have great pathos and excitement. In the Bollywood method Apala described a dire fantasy situation. The participant’s beautiful, young, and innocent niece is about to be married. But suddenly he gets news that the prospective groom is a member of the underground. He is a hit man! His whole life story is a sham, AND HE IS ALREADY MARRIED! The participant has the evidence and must book an airline ticket for himself and the groom's current wife to Bangalore. Time is of the essence!!!
The participants willingly entered this fantasy and with great excitement began the ticket booking process. Even minor difficulties they encountered resulted in immediate and incisive commentary. The participants complained about the button naming and placement. They pointed out the number of extra steps in booking. The fantasy situation gave them license to communicate in a way that they never would under normal evaluation methods.
I think this is a great method for the Asian markets. But I also expect we might be able to generalize it to special situations in North America and other places where participants may be hesitant to communicate freely.
Baker, L.M., Wilson, F.L. and Kars, M. (1997), The readability of medical information on InfoTrac: Does it meet the needs of people with low literacy skills? Reference & User Services Quarterly, Winter, 37(2), 155-160.
Coke, E.U. (1976), Reading rate, readability and variations in task?induced processing, Journal of Educational Psychology, 68, 167?173.
Coke, E.U. and Koether, M.E. (1979), The reading skills of craft and technical management employees: Estimates from two samples of students, Bell Laboratories Technical Report, May.
D'Alessandro, D., Kingsley, P. and Johnson, J. (2001), The readability of pediatric patient education materials on the Web, Archives of Pediatrics & Adolescent Medicine, July.
Graber, M.A., Roller, C.M. and Kaeble, B. (1999), Readability levels of patient education material on the World Wide Web, The Journal of Family Practice, January, 48(1), 58-61.
Hilts, L. and Krilyk B. J. (1991), Write Readable Information to Educate, Hamilton, Ontario: Chedoke-McMaster Hospitals and Hamilton Civic Hospitals.
Klare, G. R. (1975), Assessing readability, Reading Research Quarterly, 10, 62?102.
Osborne, H. (2000), Assessing readability: Rules for playing the numbers game, On Call, December.
University of Utah Health Sciences Center, Patient Education Materials: An Author's Guide.
Readability Formulas – value of readability formulas in writing for the Web.
Re: The Bollywood Technique – Fascinating. I wonder if this technique could be used successfully in other areas, both outside the Asia theater, but also in other environments.
In the late 1970's I was involved in an examination of maritime safety, specifically ship collisions, rammings and groundings. Preliminary results were obtained by reviewing accident reports and investigations. Later we observed and "dialogued" with Masters and Pilots. While we used "non attribution" to great effect, and follow-on studies used simulations similar to flight simulators, this technique may have also produced excellent results, at a lower overall cost. Something to ponder.
Sign up to get our Newsletter delivered straight to your inbox
HFI may use “cookies” or “web beacons” to track how Users use the Website. A cookie is a piece of software that a web server can store on Users’ PCs and use to identify Users should they visit the Website again. Users may adjust their web browser software if they do not wish to accept cookies. To withdraw your consent after accepting a cookie, delete the cookie from your computer.
HFI believes that every User should know how it utilizes the information collected from Users. The Website is not directed at children under 13 years of age, and HFI does not knowingly collect personally identifiable information from children under 13 years of age online. Please note that the Website may contain links to other websites. These linked sites may not be operated or controlled by HFI. HFI is not responsible for the privacy practices of these or any other websites, and you access these websites entirely at your own risk. HFI recommends that you review the privacy practices of any other websites that you choose to visit.
HFI is based, and this website is hosted, in the United States of America. If User is from the European Union or other regions of the world with laws governing data collection and use that may differ from U.S. law and User is registering an account on the Website, visiting the Website, purchasing products or services from HFI or the Website, or otherwise using the Website, please note that any personally identifiable information that User provides to HFI will be transferred to the United States. Any such personally identifiable information provided will be processed and stored in the United States by HFI or a service provider acting on its behalf. By providing your personally identifiable information, User hereby specifically and expressly consents to such transfer and processing and the uses and disclosures set forth herein.
In the course of its business, HFI may perform expert reviews, usability testing, and other consulting work where personal privacy is a concern. HFI believes in the importance of protecting personal information, and may use measures to provide this protection, including, but not limited to, using consent forms for participants or “dummy” test data.
HFI may use personally identifiable information collected through the Website for the specific purposes for which the information was collected, to process purchases and sales of products or services offered via the Website if any, to contact Users regarding products and services offered by HFI, its parent, subsidiary and other related companies in order to otherwise to enhance Users’ experience with HFI. HFI may also use information collected through the Website for research regarding the effectiveness of the Website and the business planning, marketing, advertising and sales efforts of HFI. HFI does not sell any User information under any circumstances.
HFI may disclose personally identifiable information collected from Users to its parent, subsidiary and other related companies to use the information for the purposes outlined above, as necessary to provide the services offered by HFI and to provide the Website itself, and for the specific purposes for which the information was collected. HFI may disclose personally identifiable information at the request of law enforcement or governmental agencies or in response to subpoenas, court orders or other legal process, to establish, protect or exercise HFI’s legal or other rights or to defend against a legal claim or as otherwise required or allowed by law. HFI may disclose personally identifiable information in order to protect the rights, property or safety of a User or any other person. HFI may disclose personally identifiable information to investigate or prevent a violation by User of any contractual or other relationship with HFI or the perpetration of any illegal or harmful activity. HFI may also disclose aggregate, anonymous data based on information collected from Users to investors and potential partners. Finally, HFI may disclose or transfer personally identifiable information collected from Users in connection with or in contemplation of a sale of its assets or business or a merger, consolidation or other reorganization of its business.
If a User includes such User’s personally identifiable information as part of the User posting to the Website, such information may be made available to any parties using the Website. HFI does not edit or otherwise remove such information from User information before it is posted on the Website. If a User does not wish to have such User’s personally identifiable information made available in this manner, such User must remove any such information before posting. HFI is not liable for any damages caused or incurred due to personally identifiable information made available in the foregoing manners. For example, a User posts on an HFI-administered forum would be considered Personal Information as provided by User and subject to the terms of this section.
Information about Users that is maintained on HFI’s systems or those of its service providers is protected using industry standard security measures. However, no security measures are perfect or impenetrable, and HFI cannot guarantee that the information submitted to, maintained on or transmitted from its systems will be completely secure. HFI is not responsible for the circumvention of any privacy settings or security measures relating to the Website by any Users or third parties.
Human Factors International, Inc.
PO Box 2020
410 W Lowe Ave
Fairfield IA 52556