Monitoring and Evaluation (M&E) has moved from being a peripheral administrative requirement to become an important strategic function that supports accountability and evidence-based decision-making. Doing so, however, requires the collection of high-quality, actionable data through surveys and sampling. Poorly designed surveys and flawed sampling methodologies do more than just produce inaccurate reports; they lead to wasted resources, missed opportunities for intervention and the potential for systemic policy failure.
To address these technical gaps, the Foundation for Professional Development (FPD) offers the Advanced Certificate in Monitoring and Evaluation. This programme provides practitioners with the technical framework required to optimise data collection and drive meaningful social impact.
Institutional Excellence: The FPD Advanced Certificate
Established in 1997, the Foundation for Professional Development is one of South Africa’s leading private higher education institutions. The Advanced Certificate in Monitoring and Evaluation (NQF Level 6) is designed for working professionals, delivered over 18 months through an asynchronous online platform. This allows students to integrate 12–15 hours of weekly study with their professional responsibilities.
The curriculum covers the entire statistical value chain, ensuring that M&E systems are managed as strategic assets rather than mere compliance exercises. Core modules include:
-
M&E Fundamentals & Theory of Change: Establishing causal pathways and results-based management.
-
Programme Monitoring Implementation: Designing data collection processes and allocating responsibilities.
-
Managing Data Quality: Identifying risks to data integrity and preparing for routine assessments.
-
Designing, Planning, and Managing Evaluations: Developing evaluation scope, questions and terms of reference.
-
Evaluation Results Utilisation: Reporting findings and applying recommendations in practical contexts.
M&E Survey Design: A Blueprint for Better Data
The primary keyword in modern data collection is purpose. In the absence of well-defined survey design, practitioners often succumb to fishing trips where vast amounts of data are collected without purpose in the hope that a meaningful pattern will emerge.
Defining Objectives and the Theory of Change
Effective M&E survey design begins long before a questionnaire is drafted. It starts with the development of a Theory of Change (ToC) and a logical framework (LogFrame). These tools, central to the FPD curriculum, allow practitioners to identify the specific indicators that must be tracked to measure success. A ToC provides a comprehensive description of how and why a desired change is expected to happen, ensuring that every survey question is directly linked to a critical outcome.
Designing Validated Instruments
A high-quality survey must be valid, reliable and culturally sensitive. FPD’s Programme Monitoring Implementation module equips students with the skills to select the right methodology for their specific intervention, whether qualitative, quantitative or a mixed-methods methodology.
For a multilingual country such as South Africa, survey design must also account for the linguistic diversity of the population. Fieldwork tips from across the region suggest that building trust and using local languages is essential for obtaining accurate and honest responses.
Sampling Methodologies: Precision with Efficiency
Sampling is the technical art of selecting a subset of a population that truly reflects the characteristics of the whole. Proper sampling is the primary mechanism for reducing costs while maintaining the integrity of findings.
Probability Sampling
Probability sampling ensures that every member of the target population has a known, non-zero chance of being selected, which is crucial for reducing selection bias.
-
Simple Random Sampling: Each unit has an equal probability of selection.
-
Stratified Random Sampling: The population is divided into subgroups (strata) based on characteristics like geographic ward or gender. This is particularly relevant in South Africa’s diverse socio-economic landscape to ensure marginalised voices are captured.
Determining the Right Sample Size
Choosing a sample size is a balance between being scientifically thorough and staying within a project's budget. Instead of surveying every single person in a community, we select a group that is just large enough to provide a reliable answer without wasting resources.
Three main factors generally influence this number:
-
The total population size: How many people are in the group you are studying?
-
The confidence level: How sure do you need to be that your results are accurate? (95% is the standard in the development sector).
-
The margin of error: How much "wiggle room" can you tolerate? A 5% margin is standard, meaning if your results show 60% of people benefitted, the true number is likely between 55% and 65%.
For most South African community projects, a sample of 200 to 400 people typically provides a robust enough snapshot for making sound programme decisions. For those interested in the maths and formulas behind these calculations, the website SurveyMonkey provides a detailed breakdown of the formulas and a practical tool for calculating sample size.
Managing Data Quality and Mitigating Bias
Data quality is defined by its fitness for use. The FPD module on Managing Data Quality teaches students to identify systemic errors that can skew results.
The Pervasiveness of Fieldwork Bias
In the African development sector, several specific biases are common:
-
Courtesy Bias: Respondents provide positive feedback because they perceive the evaluator to be associated with an NGO providing aid.
-
Diplomatic Bias: Evaluators avoid challenging inconsistencies to maintain relationships with local authorities.
-
Researcher Allegiance Bias: Evaluators loyal to a specific approach may inadvertently dismiss findings that contradict their beliefs.
To reduce bias, the FPD curriculum promotes triangulation, which is the cross-verification of findings using different sources, such as surveys, focus groups, and direct observation.
Digital Transformation: Reducing Rework
The shift from paper-based data collection to electronic data collection is the most significant advancement in reducing M&E waste. Digital tools like KoboToolbox and ODK offer real-time analytics and superior quality controls.
A validation study in South Africa found that an electronic questionnaire had a significantly lower error rate (0.17 errors per 100 questions) compared to paper methods (0.73 errors per 100 questions). EDC systems utilise "skip logic" and "range constraints" to prevent invalid data entry, such as recording a child's age as 150 years. This real-time validation eliminates the expensive rework associated with manual data cleaning.
Regulatory Compliance and Ethical Integrity
In South Africa, all data collection must comply with the Protection of Personal Information Act, ensuring confidentiality and informed consent. Furthermore, organisations seeking to have their statistics designated as "official" must adhere to the South African Statistical Quality Assessment Framework (SASQAF). SASQAF evaluates data against eight dimensions, including methodological soundness, accuracy and integrity. The FPD programme provides the foundational knowledge to navigate these frameworks, fostering a culture of high-integrity data management.
Conclusion
Mastering M&E survey design is not merely about learning to use a tool; it is about ensuring that every resource spent on data collection yields an equivalent value in insight. Through its Advanced Certificate in Monitoring and Evaluation, the Foundation for Professional Development empowers practitioners to lead with evidence, reduce waste and ultimately improve the lives of the communities they serve.
Frequently Asked Questions
How does a Theory of Change prevent "data waste" in surveys?
A Theory of Change maps the logical pathway from activities to long-term impact. By using a Theory of Change during survey design, practitioners can identify exactly which indicators are necessary to test their assumptions. This prevents the collection of irrelevant data that does not contribute to understanding the project’s success, thereby saving time and budget.
What is the advantage of stratified random sampling in South African projects?
South Africa has high levels of socio-economic diversity. Stratified random sampling involves dividing the population into subgroups (strata) based on characteristics like gender, age or location. This ensures that even small or marginalised groups are represented in the data, providing a more accurate and equitable picture of a programme's impact across different segments of society.
Why are electronic data collection (EDC) tools more cost-effective than paper?
While EDC requires an initial investment in hardware, it reduces long-term costs by eliminating the need for manual data entry and extensive data cleaning. Features like skip logic and real-time error checking ensure that the data collected in the field is high-quality and ready for analysis almost immediately.
What are the 8 dimensions of quality under the SASQAF framework?
The South African Statistical Quality Assessment Framework (SASQAF) defines quality through:
Adhering to these dimensions ensures that M&E data is scientific, transparent and capable of being certified as official statistics by the South African government.
How can I mitigate "Courtesy Bias" during fieldwork?
Courtesy bias occurs when respondents give positive answers to please the evaluator. To avoid this, practitioners should use independent enumerators who are not directly involved in service delivery. Additionally, ensuring respondent anonymity, building trust before data collection, and using triangulation (comparing survey results with direct observation) help verify the truthfulness of the data.
