One common thread that runs through the media campaigns and entertainment-education projects that my partners and I have done is systematic monitoring and evaluation (M&E) of communication materials, process and effects. I decided to write about this because there seems to be an interest in this area. Just last week, a colleague was invited to give a talk on this subject at a planning workshop for a tree seedling systems project. I discussed with her our monitoring and evaluation framework which, I suggested, could be adapted for her talk.

So if you are starting a new development project which aims to achieve impact, it is assumed that communication will play a central role. Dr. Nora Quebral, the development communication guru, has emphasized this in the early 70s. For a development project to succeed, communication must be considered at the project’s planning stage and not just considered as an afterthought. To determine if communication can help bring about changes in knowledge, attitude and practices, then a monitoring and evaluation framework has to be in place.

What is monitoring & evaluation of IEC materials

  • It is a systematic and objective collection of information about activities, characteristics and outcomes of IEC materials — a media campaign, radio drama series, leaflets, booklets, posters, demonstrations, farmer experiments, etc.
  • It is conducted to make judgments about the performance, effects and impact of a campaign, drama series or a mix of communication materials.
  • It is done to inform decisions about future IEC projects.

Why systematic M&E is needed?

  • To develop program content and set targets
  • To track progress, troubleshoot problems, perform mid-course action
  • To determine if objectives were achieved
  • To quantify effects and impact
  • To understand cause and effect relationships
  • To help improve future communication strategies

- audience segmentation
- content design – framing, positioning
- branding – “Three Reductions, Three Gains”, “No Early Spray”
- media mix
- on-the-ground support

 

M&E events in the project cycle

In a typical project cycle, M&E events can take place at various stages, from planning stage (needs assessment) until project completion (review workshop). Below you will find various M&E activities occurring at different project milestones:

1. Needs assessment — focus group discussions, scoping studies, key informant interview
2.Project design and pre-implementation — stakeholders’ workshop, baseline survey, pretesting
3. Project implementation — site visits, key informant interviews
4. Monitoring of ongoing project — management monitoring survey, field visits
5. Evaluation of completed project — review workshop, field day

Communication research methods

In monitoring and evaluation of IEC materials, an array of communication research methods can be used. The matrix below presents various methods and the purpose of each.

In future posts, I will discuss each method in detail and upload sample instruments that we have used for audience analysis, baseline survey, management monitoring and post-tst survey. Content analysis and pretesting have been covered in earlier posts. But if you’re in a hurry and need some advice on specific communication research methods, please post a comment and I will help, pro bono always.

Method

Purpose

1. Audience analysis

To characterize audience (demographics, communication environment) to develop content of materials, set campaign targets

2. Baseline survey

To assess knowledge, beliefs and behavior – to document current scenario

3. Pretesting of prototype materials

To determine appeal, understandability of materials (radio drama, campaign materials)

4. Management monitoring survey

To track implementation plans and make adjustments as needed

5. Content analysis

To analyze the content of audience feedback

6. Post-test survey

To determine whether the project has achieved its objectives