In Jinhua, Zhejiang province last week, 14 student enumerators pretested our draft baseline survey questionnaire with rice farmers. When the college students returned from the field, they spoke about farmers’ difficulty understanding the term “wild flowers” in the questionnaire. Growing wild flowers is an ecological engineering method aimed to restore and enhance important ecosystem services to provide adequate crop health.
The term “wild flowers” was used in a series of attitude/belief statements in our draft questionnaire. Here are some examples:
- Increasing wild flowers on the bunds is a waste of time.
- Increasing wild flowers on bunds is easy to do.
- It is difficult to increase wild flowers on bunds because nearby paddy fields use herbicide.
- We cannot increase the wild flowers on bunds because we burn our rice straw.
- Increasing wild flowers on bunds is additional burden to farmers.
- It is difficult to increase wild flowers on bunds because farmers will step on them.
- Our bunds are narrow so there is no place for wild flowers.
- I am willing to try increasing wild flowers in the bunds to learn more about what they can do
Students reported that farmers thought that wild flowers are useless and they should not waste time growing them and answering questions that had to do with these flowers. This pretest feedback pointed to a lack of understanding between scientists and farmers on what the words meant. As used in the questionnaire, wild flowers are plants which are not cultivated but produce nectar-rich flowers to attract natural enemies. Many are growing wildly around rice paddies, hills and roadsides. On the other hand, farmers thought that wild flowers are utterly useless plants. After a long discussion on what term will best capture uncultivated flowers, it was agreed that the term, “beneficial flowers” be used instead.
What is questionnaire pretesting
The pretest is a try-out of the questionnaire to see how it works and whether changes are necessary before the start of the actual survey. About 15 to 20 respondents, whose characteristics are reasonably similar to the survey population, will be adequate for a pretest. The questionnaire is then revised and finalized on the basis of pretest results. Pretesting guidelines will facilitate the conduct of a pretest.
Why pretest a survey questionnaire
The pretest provides a means of catching and solving unforeseen problems in the use of the questionnaire, such as the phrasing and sequencing of questions. Linguistic and cultural differences also complicate the task of questionnaire development, making pretesting all the more indispensable. The pretest enables one to:
1) improve the wording of the questionnaire;
2) correct and improve translation of technical terms;
3) check the accuracy and adequacy of the questionnaire’s instructions such as “skip” and “go to”;
4) eliminate unnecessary questions and add necessary ones; and
5) estimate the time needed to conduct the interview. In the illustration below, excerpts of pretest results are reported to illustrate how pretesting yielded constructive suggestions which served as the basis for improving a questionnaire.
Years ago, we conducted a rather thorough questionnaire pretest in Iloilo province, Philippines. Here’s the pretest report of Danny N. Valenzuela, our research collaborator for a farmer survey on rice seed health.
Sample pretest results: suggestions from the field
The survey questionnaire for the project “Farmers’ perceptions and attitudes toward seed health for crop management” was pretested in Pandac, Pavia, Iloilo. The goals of the pretest were to determine the reactions of rice farmers to the questionnaire, estimate the time needed to complete the interview, validate the translation of key technical terms used, find out whether the respondents could understand the technical terms, and ascertain whether the sequence of the questions solicited the desired information. A survey enumerator was hired to assist in pretesting the instrument with 15 rice farmers.
The following observations and corresponding recommendations were made:
1) In this village, the first and second crops are both grown during the wet season, so the reference to the first one as the wet season crop and the second one as the dry season crop does not apply. Recommendation: Provide the necessary indicators for the interviewers to guide them if they are confronted with decision problems in this area.
2) The second cropping was not realized because of drought (question 4: Last year for the second cropping, what was the area on which you direct-seeded and/or transplanted? Some farmers did not plant rice and instead planted watermelon or tomatoes. Recommendation: Point this out to the interviewers during the orientation.
3) In question 5 (What cropping pattern did you practice?), the farmers found the term “cropping patterns” difficult to comprehend. The question cannot be properly answered without explaining the term or making it a leading question. Recommendation: Modify the question to instead ask what crop was grown in a particular season. The farmers also mentioned a third cropping season during which they plant crops” such as watermelon, tomatoes, and mungbean in lieu of rice.
4) Question 6 (Last cropping season, how much (in pesos) did you spend for rice seeds, insecticides, fungicides, herbicides, fertilizer, irrigation and labor for pesticide application?) entailed a lot of time to answer because the interviewer often had to add up the figures given by farmers. Recommendation: Remind the interviewers to bring calculators with them.