This Campbell systematic review examines the effectiveness of farmer field schools in improving intermediate outcomes (such as knowledge and pesticide use) and final outcomes (such as agricultural yields, incomes and empowerment) in low- and middle-income countries, as well as implementation factors associated with programme success and failure. The review sythesises evidence from 92 impact evaluations, of which 15 were of sufficient quality for policy-oriented findings, and 20 qualitative studies.
Farmer field schools improve farmers’ knowledge and adoption of beneficial practices, and reduce overuse of pesticides. This leads to positive outcomes for farmers: on average, a 13% increase in agricultural yields and a 20% increase in income. Farmer field schools also reduce pesticide use and environmental degradation. However, the evidence for these outcomes comes from short-term evaluations of pilot programmes, and no studies with a low risk of bias are available.
In programmes that were delivered at a national scale, studies conducted more than two years after implementation did not show any positive outcomes from the programme. For large-scale programmes, recruiting and training appropriate facilitators was problematic.
After almost three decades of decline in public support, agriculture is now back on the development agenda. Since the late 1980s, support to agriculture has shifted from top-down approaches to those identifying technologies and methods of communicating technologies which are suitable to support farmers’ livelihoods in a sustainable manner, including participatory approaches based on the notion of creating spaces for farmer self-learning. One such approach is the farmer field school (FFS), an adult education intervention which uses intensive “discovery-based” learning methods with the objectives of providing skills in such areas as integrated pest management (IPM) and empowering farmers and communities. FFS have been implemented in 90 countries worldwide, reaching an estimated 10–15 million farmers. Farmer field schools may appear to be the latest tool, but what does the evidence say regarding their effectiveness?
This systematic review synthesises evidence on interventions identified as “farmer field schools” conducted in low- and middle-income countries. The review aims to provide answers to the following research questions:
Review question (1)
a) What are the effects of farmer field schools on final outcomes such as yields, net revenues and farmer empowerment?
b) What are the effects of farmer field schools on intermediate outcomes such as knowledge and adoption of improved practices (e.g. reduced use of pesticides)?
c) What are the effects on outcomes for non-participating neighbouring farmers living in the same communities as FFS farmers?
Review question (2)
What are the enablers of and barriers to FFS effectiveness, diffusion and sustainability?
Studies included in the review satisfied the following criteria.
Eligible participants included farmers growing arable crops, living in low- or middle-income countries at the time of the intervention. The review included those participating directly in the field school and also non-participant neighbour farmers who may benefit through spillover effects or more formal dissemination methods.
Eligible interventions were those identified as “farmer field schools,” regardless of the design or implementation, including FFS programmes providing training in IPM and other techniques. Studies combining FFS with other intervention components, such as input or marketing support, were also included.
Comparisons eligible for the effectiveness review were farmers who received no intervention, or access to agricultural extension services from another source, including IPM (or equivalent) training.
All outcomes reported were eligible for the review.primary Primary outcomes were agricultural outcomes , including yields and profits (net revenues). Secondary outcomes included other final outcomes such as environmental outcomes, health status and empowerment; and intermediate outcomes, including farmer knowledge and adoption of practices. Qualitative evidence on barriers to and enablers of effectiveness and sustainability were also included, including process and implementation information and measures of beneficiaries’ attitudes and experiences with FFS.
Eligible study designs for the effectiveness synthesis (review question 1) were measurable using counterfactual impact evaluations, including experimental or quasi-experimental study designs and methods of analysis. Studies eligible for the synthesis of barriers and enablers (review question 2) were based on primary data collected from FFS participants, extension agents or experts, analysed using qualitative methods or descriptive statistics. The qualitative studies needed to report at least some information on the research question, procedures for collecting data, sampling and recruitment, and at least two sample characteristics.
The search included electronic academic databases, internet search engines, websites and theses, as well as handsearches of key journals and literature snowballing. Searches included general social science sources as well as agriculture subject-specific sources of published and unpublished literature. All searches were updated in October 2012.
The farmer field schools evaluation community has generated a large number of evaluations. We screened the titles and abstracts of over 28,000 papers, the majority of which were irrelevant to the topic. Four-hundred-sixty (460) relevant papers on FFS were assessed for inclusion based on full text. After the final screen by two authors, 134 quasi-experimental studies comprising 92 distinct evaluations meeting the inclusion criteria were eligible for the review. The impact evaluations provide quantitative estimates of effects on outcomes for 71 FFS projects. However, only 15 of the impact evaluations meeting the inclusion criteria were judged to be of sufficient internal validity to make predictions for policy. The review also includes 20 qualitative evaluations meeting the inclusion criteria, which discuss the barriers to and enablers of change in 20 FFS projects. A portfolio review of 337 project documents was also conducted.
Two independent reviewers assessed the full text papers against the inclusion criteria; discrepancies were resolved by consensus or by a third author if needed. Two reviewers extracted data from included studies. Quantitative impact evaluation studies were critically appraised according to the likely risk of bias according to threats to internal validity (causal identification), external validity (generalisability) and file-drawer effects (publication bias). Qualitative evaluations were assessed according to adequacy of reporting, data collection, presentation, analysis and conclusions drawn.
We used a hypothesised programme theory of change (White, 2009) as the framework for integrating the evidence. We collected data on programme design, implementation, targeting and contextual factors, and linked individual studies by programme in order to assess whether heterogeneous programme effects were correlated with study design, implementation and context.
For the quantitative synthesis (review question 1), we extracted effect size estimates from included studies, calculating standard errors and 95 per cent confidence intervals using data provided in the studies, where possible. We used random effects meta-analysis, estimating average effects of farmer field schools on the different outcomes, and examining heterogeneity. The results of the publication bias analysis suggested under-reporting of small sample studies with negative or insignificant findings for studies reporting evidence on agricultural yields, which is evidence for possible publication bias.
For the synthesis of qualitative evidence (review question 2), we used a thematic approach (Thomas & Harden, 2008), combining predetermined themes based on the links and assumptions in the theory of change model, as well as any other themes emerging from the detailed coding of the included studies.
In the final stage of analysis, we used an iterated approach in which some effect moderators identified during the qualitative synthesis were tested in meta-analysis and meta-regression.
Review question (1)
No studies with a low risk of bias were identified for the review of effects and only 15 (out of 92) quasi-experimental studies were assessed as being of medium risk of bias and therefore policy-actionable. The results of these medium-risk-of-bias studies (reported in Summary of Findings Table 1) suggest farmer field schools impact positively on intermediate and final outcomes for participating farmers in the short to medium term.
Findings for intermediate outcomes were as follows:
However, there is no evidence of effects on outcomes over the longer term (follow-up surveys greater than two years after implementation) in programmes which have been scaled up nationwide.
For IPM farmer field schools, there is no evidence that diffusion from FFS participants to non-participating neighbour farmers usually happens:
Review question (2)
Qualitative evaluations (reported in Summary of Findings Table 2) in the review helped us to understand the different types of farmer field schools implemented around the world, the reasons for heterogeneous impacts among FFS participants, and the limited diffusion to non-participating neighbour farmers. FFS use discovery-based learning methods which differ from agricultural extension interventions that tend to focus on disseminating knowledge of more simple practices, for instance application of fertiliser and pesticides, or adoption of improved seeds. However, there are several barriers to spontaneous diffusion of knowledge and practices. The FFS curriculum is complex and the training should be experience-based, so that farmers are able to observe that FFS practices have a relative advantage over conventional farmer practices. Existing levels of social capital, the reach of social networks, and approaches to targeting FFS participants were found to be potentially important factors in influencing diffusion. More generally, the studies identify some of the more common problems in implementation, notably where a top-down “transfer of technology” approach has been implemented for an intervention which is intended to be based on a “bottom-up” participatory approach. All qualitative evaluations presented some evidence of use of triangulation to verify their findings, although most studies had weaknesses in reporting on sampling, analysis, and presentation of data, making quality appraisal of this evidence base challenging.
Farmer field schools can have beneficial effects for participating farmers, in pilot programmes in the short term. The impacts on agricultural outcomes may be of substantial importance to farmers, in the region of a 10 per cent increase in yields and 20 per cent increase in profits (net revenues). The effects are particularly large when FFS are implemented alongside complementary upstream or downstream interventions (access to seeds and other inputs, assistance in marketing produce) for cash crops. However, the few studies of scaled-up programmes measuring outcomes over the longer term (more than two years post-training) do not find any evidence of effects of FFS. Farmers may also feel more confident, but again very few studies have assessed empowerment outcomes rigorously.
There is little evidence of diffusion of improved practices or outcomes from FFS participants to non-participating neighbour farmers. Field schools targeting more educated farmers may be better able to diffuse simple practices, such as on reduced pesticide use, than field schools that target less educated farmers. However, there is no evidence that any diffusion of practices is sustained over time, nor any evidence for adoption of more complex IPM practices via diffusion.
As a method of rural adult education, FFS appear suited for gradual scale-up provided there is a clear focus on ensuring local institutionalisation (i.e. favouring intensiveness of coverage in each community over geographical breadth of coverage). On the other hand, FFS seem unsuited to solve the problems of large-scale extension. The approach may not be cost-effective compared with agricultural extension in many contexts, except where existing farming practices are particularly damaging, for example due to overuse of pesticides. This is because of the highly intensive (and therefore relatively costly) nature of the training programme, the relative successes in targeting more educated farmers as compared with disadvantaged groups, and failures in promoting diffusion of IPM practices.
Targeting FFS participants: Proponents of FFS have recommended targeting more highly educated farmers, those with greater land endowments, younger farmers and women, favouring those with relatively low opportunity costs of labour or farmers with relatively high pesticide costs. Problems were highlighted in targeting women who lived in household where they were not in a decision-making position, and youth who were unable to dedicate sufficient time to the FFS plot or their fields.
Where the aim is to include women and disadvantaged members of the community, implementers may need to tailor the intervention to enable their participation in the programme. The curriculum needs to be relevant and consistent with the needs and opportunities of women and the poor. Most obviously, in contexts where women are primarily responsible for growing subsistence crops, a curriculum that covers only commercial crops is unlikely to attract women participants. More generally, the curriculum and crops covered in FFS should also be adapted according to the local agricultural system and the needs of the farmers targeted by the programme. Curricula need to deal with the major challenges facing farmers. In most cases, these challenges will be multifaceted, highlighting the need to balance comprehensiveness with being able to cover all issues in sufficient depth to ensure appropriate learning. A cumulative approach over several seasons, including exchanges between field schools, may be preferable.
FFS facilitators: The evidence also suggests that appropriate targeting and training of FFS facilitators is important. The theory of change suggests FFS should be delivered according to a participatory and discovery-based approach to learning, including opportunities for farmers to experiment and observe new practices, particularly if farmers are to be empowered with lifelong skills capacity development. Attempts to target facilitators based on education or literacy levels may be less effective than targeting based on ability to communicate, and appropriate training which enables facilitators to use a bottom-up approach. This is most obviously a barrier in scaled-up programmes where FFS facilitators are recruited from extension staff who previously used more top-down agricultural extension methods. Recruitment of facilitators should take into account personal attitude, maturity, literacy, leadership skills, knowledge in local language and experience with farming. In many contexts the gender of the facilitator should be carefully considered. Facilitators should have access to ongoing support and backstopping from supervisors and technical experts connected to local research centres. Regular monitoring of facilitators may help to identify schools where additional support is required.
Complementary policies: Institutional actors involved in FFS should consider farmers’ needs and interests in the design and implementation of the FFS programme. In some contexts stronger policies and regulatory measures may be necessary to counteract the activities of the pesticide industry, including the promotion and sale of pesticides by extension workers who are promoting FFS. New policies facilitating participatory agricultural extension approaches, replacing earlier extension policies aimed at promoting off-the-shelf technologies and input packages, may also be necessary.
Local institutionalisation: Formal support and encouragement of FFS alumni, including technical assistance and backstopping, may be important for the sustainability of FFS practices and related activities. Given the skills-based nature of the practices promoted in FFS, formal community-building activities, support and successful attempts to institutionalise the approach, to encourage FFS graduates to train other farmers, are likely to be needed for any broader diffusion to non-participating neighbour farmers, although the evidence base does not indicate that such attempts have been successful in the past.
The majority of FFS impact evaluations (68 out of 92) use designs of questionable internal validity, and are therefore of limited value in determining whether farmer field schools have made a difference to outcomes. We were not able to locate any completed evaluations which used randomised assignment, an approach which is feasible for FFS. In three-quarters of evaluations, no serious attempts were made to control for confounding through statistical matching or other statistical analysis, and in one-third of cases statistical significance tests were not reported. The likely consequence, as indicated in the meta-analysis, was the systematic overestimation of effects for all outcomes. The extent of resources that has been devoted to farmer field schools evaluations might therefore be usefully re-allocated to conducting fewer but more rigorous impact evaluations, particularly those based on a solid counterfactual, with prospective cluster-level assignment (randomised or otherwise) to allow measurement of community-wide diffusion and to assess effects on agriculture and empowerment outcomes in the medium to longer term (three years or more).
Evaluations should report information on both intervention design and implementation processes so that it is possible to assess whether programme causal chains break down because the intervention design is simply not appropriate for the context or because of poor implementation.
Many qualitative evaluations need to report aspects of the research process in greater detail to allow users to assess their credibility and applicability. In particular, clear reporting on objectives, on methods of sampling, data collection and analysis should be provided. Greater use of structured abstracts will facilitate easier access to quantitative and especially qualitative research. Future studies should include data on views and experiences of FFS facilitators and agricultural extension workers.