Entries categorized as ‘Pretesting’

Flesch and SMOG readability tests

November 30, 2007 · 4 Comments

One of the earlier readability tests is the Reading Ease formula developed by Rudolph Flesch. Considered to be almost as accurate a measure as the the Dale-Chall formula that preceded it, the reading ease formula has been noted to be easier to use. To compute it, one only has to count words, syllables, and sentences. Then, sentence length and word length are summed to compute the score. A low score suggests an extremely difficult piece of material while a high score implies a very easy read.

For details on the original Flesch formula, read more …

In 1978, David Foulger presented a simplified reading ease formula which is a modification of the original Flesch procedure. Foulger’s revision aims to cut down on word counting time which should make computations easier.

SMOG Readability Test

SMOG (Simple Measure Of Gobbledygook) is another readability test that’s easy to calculate manually. To calculate
the SMOG reading grade level, begin with the entire written work that is being assessed, and follow these four steps:

1. Count off 10 consecutive sentences near the beginning, in the middle, and near the end of the text.

2. From this sample of 30 sentences, circle all of the words containing three or more syllables (polysyllabic), including repetitions of the same word, and total the number of words circled.3

3. Estimate the square root of the total number of polysyllabic words counted. This is done by finding the nearest perfect square, and taking its square root.

4. Finally, add a constant of three to the square root. This number gives the SMOG grade, or the reading grade level hat a person must have reached if he or she is to fully understand the text being assessed. A few additional guidelines will help to clarify these directions:

* A sentence is defined as a string of words punctuated with a period (.), an exclamation point (!) or a question mark (?).
* Hyphenated words are considered as one word.
* Numbers which are written out should also be considered, and if in numeric form in the text, they should be pronounced to determine if they are polysyllabic.
* Proper nouns, if polysyllabic, should be counted, too.
* Abbreviations should be read as unabbreviated to determine if they are polysyllabic.

Not all pamphlets, fact sheets, or other printed materials contain 30 sentences. To test a text that has fewer than 30 sentences:

1. Count all of the polysyllabic words in the text.
2. Count the number of sentences.
3. Find the average number of polysyllabic words per sentence as follows:
Average= Total # of polysyllabic words/Total # of sentences
4. Multiply that average by the number of sentences short of 30.
5. Add that figure on to the total number of polysyllabic words.
6. Find the square root and add the constant of 3.

Perhaps the quickest way to administer the SMOG grading test is by using the SMOG conversion table.

Read more …

Categories: Pretesting
Tagged: , ,

Readability measures for print materials

November 29, 2007 · 2 Comments

Readability tests like the cloze procedure, Fog index, Flesch formula, and Smog are procedures to determine the reading ease of a piece of written material. It is done before materials are pretested with the target audience. Print materials with lower reading levels are easier to understand and remember even for highly literate audiences.

Readability tests are largely based on number of words, number of sentences or even number of polysyllables. These tests teach us a simple rule in readability — that when writing a piece of material, if we use shorter words (versus words with 3 syllables or more) and shorter sentences, chances are, our writing will be easier to understand by the average reader. The heuristic then is to simplify. See, I just used a 3-syllable word (heuristic) to mean “simple rule”. Strange how people tend to prefer to use complicated words to simpler words although it limits understanding the material they have written.

The Fog index is one method of analyzing written material to see how easy it is to read and understand. The resulting number indicates the number of years of formal education that a person requires in order to easily understand the text on the first reading.

Another psychological tool for measuring the effectiveness of communication is the cloze procedure. When first introduced, it was regarded mainly as a “new and better” way of determining the readability of printed English prose.

Read more …

Categories: Pretesting
Tagged: , , ,

Factors affecting validity of pretests

November 20, 2007 · No Comments

Pretesting a survey questionnaire with a farmer, West Java, Indonesia

When can one say that the pretest results he/she obtained could most likely predict audience reactions to the final version of a communication material? Reed (1974) pointed out these four factors that could affect the validity of pretests - subjects sampled, setting, manipulation, and measurement. Consider these questions that often confront pretesters: Should a random sample of the potential audience be interviewed for a pretest? Or will persons different from the intended audience do? Should we conduct our pretest in the same setting as the one in which the audience will see the final production or in restricted undisturbed settings? Should we present the finished production or will a mock-up do? Do we measure the intended effect of the material or some criterion other than the intended effect?

Read more …

Categories: Evaluation of communication materials · Pretesting
Tagged: , ,