Heterogeneous quality of agricultural commercial inputs and learning through experimentation

Lead Research Organisation: Paris School of Economics
Department Name: Research

Abstract

This proposal aims to answer three research questions regarding small holder technology adoption of great interest to our partners and to the policy makers aiming for higher agricultural productivity in Africa. First is the heterogeneous and hidden quality of inputs a barrier to technology adoption? If so, this would lead to the recommendation of interventions that guide the farmers toward the right inputs, through regulations or through the diffusion of information. Second, do estimates regarding returns to new technologies coming from agricultural research in on-farm trials provide biased estimates for the response to inputs in real life conditions? If so, what are the most important sources of such bias, and how can trials be designed to avoid them? The study hence aims to reinforce the bridge between agronomic and development economics work. Third, does learning-by-doing and learning-from-others regarding new agricultural technologies differ depending on soils, skills and gender? And does this heterogeneity provide useful lessons for the design of more inclusive extension models and on the role of own experimentation in such models?

To answer these questions we propose two Randomized Control Trials (RCT). The first "research trial RCT" will allow the farmers in the randomly selected villages to participate to the on-farm trials carried out by the agronomists of IITA in order to test inputs of different quality. Second, in the "technology dissemination RCT" village based advisors will be trained in randomly selected villages to promote the best inputs identified in the research trials. For each RCT we will collect several rounds of panel data to study the dynamic adoption processes. This will be complemented with information from soils sample and with detailed information on farmers' cognitive, non-cognitive and technical skills. The study has been jointly designed with IITA scientists during a series of meetings and field trips, drawing on the expertise of agronomists and economists. A specific objective of the collaboration between PSE and IITA is capacity building for IITA social scientists in rigorous impact evaluation methods, through hands-on collaboration in all the different steps of the research.

The research will provide guidelines on how to address the issue of heterogeneous input quality to our partners at IITA, and more broadly to policy makers and stakeholders interested agricultural productivity in Africa. It will also aim to provide guidelines for future research by scientists within and beyond the CGIAR, and provide key insights to economists and other social sciences studying the puzzle of low adoption in Sub-Saharan Africa. We will therefore actively engage with different stakeholders throughout the project, and aim to publish the results in high quality peer reviewed academic journals.

This proposal addresses a number of key issues highlighted under the call for proposals' Theme 1 "Agriculture and Growth". It is also of relevance to Theme 3 "Innovation and productivity growth in Low -Income Countries".

Planned Impact

This research project addresses various questions of key relevance to policy makers and practitioners focused on agricultural productivity and agricultural research in Sub Sahara Africa. To reach it's policy potential, an important part of the project will involve engagement with, feedback from, and result dissemination to various policy stakeholders. Policy impact is sought at different levels, from our partners at IITA and the specific context of the Compro II input regulation and dissemination practices and IITA's farmer managed trial practices; to other community-based extension and farmer managed trials in Kenya and elsewhere in Sub Sahara Africa; to the broader international debate on input quality, extension and the profitability of agronomical research.

This research will show whether heterogeneous input quality constitutes a major barrier to technology adoption, and doing so provide strong causal evidence regarding the potential impact of related policies. Policy makers can improve quality controls and regulatory mechanisms. They can also support technology dissemination activities, to provide information on the quality of the inputs to the farmers at the local level. Understanding the extent to which the presence of low quality inputs in the market potentially limits farmers experimentation and therefore might hamper adoption of high quality ones can provide a powerful additional rational for such policies. The research has the potential to directly influence regulation and information dissemination activities under IITA's Compro II project, which aims to reach 420,000 households in 6 participating countries with information about input quality.

The study will also shed light on farmers' learning from their own experimentation and from behavioral marketing. An excessive role of marketing over the farmer's learning would highlight the risk of pushing farmers toward technologies that are not profitable in their conditions. Understanding to what extent own experimentation can potentially overcome heterogeneity of skill levels and soil conditions and enable male and female farmers to learn about returns to technologies that are relevant for them, will have direct implications for the potential returns to extension approaches that rely on such experimentation. Given the popularity of such approaches, and the widely recognized need for rigorous causal evidence on cost-effective and inclusive extension models this will be directly relevant for the wide community of extension practitioners and policy makers in Africa.

Finally strengthening the bridge between research in agronomy and in development economics will tell us whether the data of on-farm trials can be used to estimate the returns of existing technologies when disseminated among farmers. It will lead to specific guidelines on how to adjust the designs of agronomical trials to limit some of the potential important biases, and provide strong causal evidence to support those ideas. This in turn has the potential to significantly increase the lessons that can be learned from future agronomical trials, and can contribute to the evaluation of the large-scale impact of CGIAR and other agronomic research. It will also contribute to a better decision making on which technologies to be promoted and how to allocate funds between agronomic research and dissemination efforts.

The answers to the 3 research questions are hence expected to lead to some very clear policy implications, relevant for a wide set of actors. We consider dissemination of these results a key part of this proposal and expect to do so both in a number of high quality peer reviewed publications and in a clear set of policy guidelines that will be widely disseminated through a variety of mechanisms, such as stakeholder meetings, policy conference and workshop presentations, websites, policy briefs and outreach publications (see pathways to impact appendix).
 
Description The research has led to important contributions in 3 different domains.

First, narrowing agronomic yield gaps is believed to be key for addressing global poverty and hunger, and evidence from agricultural trials suggests large yield gains can be obtained through agricultural intensification in developing countries. But the promise of new technologies does not always carry over from agronomical trials to real-life conditions, and the diffusion of many new technologies remains limited. This research draws on a set of research trials with large increments in potential yield for tested input packages for maize and soybean in Western Kenya. We exploit variation in the method to select farmers and detailed data collection to adjust for both selection and behavioral responses. We obtain estimates from the trials that are much closer to the yields farmers obtain on their own plots. Moreover, we show that estimates of yield increments of two different input packages are substantially affected by the calculation method, but not necessarily in easily predictable ways. We show how combining different calculations can widen lessons learned from trials, and offer practical recommendations on how to implement these different calculations to obtain relevant measurements.

Second, information barriers can be an important constraint preventing adoption of a profitable technology. Whether such information constraints exist and persist likely depends on farmers' ability to learn about the use of, and the returns to new technologies, through learning-by-doing or through learning-from-others. However farmers' experimentation and learning does not always happen, which can be puzzling given that in many cases the cost to experiment a technology may seem relatively small compared to the long-term benefits of technology adoption. This research explores a number of reasons that can explain this puzzle.
This research hence focuses on the dynamic processes underlying farmers' learning about heterogeneous returns to new inputs. As numerous interventions aim at increasing technology adoption through learning a better understanding of how these different factors affect farmers' learning arguably is key for effective policy design. We provide strong causal evidence on the impact of providing information on the returns to specific combination of inputs that rely on experimentation on the farmer's own land. We pay special attention to the heterogeneity in returns and learning due to local soil conditions and differences in skill levels of farmers. Beyond providing unique evidence on learning-by-doing regarding input quality and suitability, the research also contributes by analyzing learning-from-others. First of all, we analyze learning within the household, building on a rich baseline datasets with individual skill measures for the two main farmers in the household. Second, we analyze learning by other farmers in the village, and how differences between neighbors and participating farmers affect the learning process.
The findings show that experimentation on farmers' own plots results in clear learning gains. Farmers' learning is slow but it matches well the agronomic findings and after several seasons many identify which inputs worked best and increase the demand for those specific inputs. Community selected farmers learn faster and more, but differences with randomly farmers decrease over time. And learning is not limited to specific inputs, but farmers' also grasp wider lessons regarding optimal agronomical practices, and apply those on their own plots. Learning increased the willingness to purchase the inputs, but only partially translates into purchase, pointing to important remaining constraints, in particular on the supply side.
Learning-by-doing is to a certain extent accompanied by learning from others. Indeed learning is strong across treatments: farmers with maize trials learn about soya and vice versa, suggesting high communication among participating farmers in the village. Indeed, we find that participation in the trials increases the communication among the participating farmers, and this increases over time. Yet learning of neighboring farmers that themselves did not participate in any trials appears more limited. In contrast, we find significant learning spillovers within the participating households.

Third, and finally, the research also contributes to the measurement of cognitive, noncognitive and technical skills of farming households in developing countries. Measures of cognitive, noncognitive, and technical skills are increasingly used in development economics to analyze the determinants of skill formation, the role of skills in economic decisions, or simply because they are potential confounders. Yet in most cases, these measures have only been validated in high-income countries. The data generated as part of this research project was instrumental to tests the reliability and validity of some of the most commonly used skills measures in a rural developing context. Prior to the start of this project, a survey with a series of skills measurements was administered to more than 900 farmers in western Kenya, and the same questions were asked again after three weeks to test the reliability of the measures. To test predictive power, the research uses data collected as part of the ESRC-DFID project on agricultural practices and production during the four following seasons. The results show the cognitive skills measures are reliable and internally consistent, while technical skills are difficult to capture and very noisy. The evidence further suggests that measurement error in noncognitive skills is non-classical, as correlations between questions are driven in part by the answering patterns of the respondents and the phrasing of the questions. Addressing both random and systematic measurement error using common psychometric practices and repeated measures leads to improvements and clearer predictions, but does not address all concerns. The paper provides a cautionary tale for naïve interpretations of skill measures. It also points to the importance of addressing measurement challenges to establish the relationship of different skills with economic outcomes. Based on these findings, the paper derives guidelines for skill measurement and interpretation in similar contexts. Follow-up research to establish the external validity of these findings is ongoing.
Exploitation Route - The most direct policy impact will come through the adoption of the lessons learnt in IITA' Compro II's team (our partner in this project), and the wider international IITA and CGIAR research community, for whom the lessons related to learning from agronomical trials are very pertinent.
- More broadly, findings are useful for member of the agronomical research and extension community in Kenya and internationally, as they provide insight on the design of their interventions
- results also feed into the academic literature on extension, learning and skills - and can help shed light on open questions regarding the lack of adoption of profitable technologies
- finally, insights on skill measurement have important implications for data collection by research and practitioners in developing countries, and are already leading to changes in practices.
Sectors Agriculture, Food and Drink,Education

 
Description Lessons regarding learning from agricultural trials provide valuable lessons regarding targeting and return to agricultural research efforts. This approach is being recognized by SPIA, the Standing Panel for Impact Assessment of the CGIAR as a valuable example to follow for other technologies and centers, and considered for scale up and has been discussed in various fora during 2018. It has also received interests from agricultural research practitioners. This is evidenced, for instance, by references to the work under the grant in a perspectives article in Global Food Security, co-authored by many leading CGIAR researchers, indicating that insights developed under the work in this grant are leading to changes in practices on the ground.
First Year Of Impact 2016
Sector Agriculture, Food and Drink
Impact Types Policy & public services

 
Description Insights on skill measurement leading to adjustments in measurement of noncognitive skills in national representative large household surveys by the World Bank
Geographic Reach Multiple continents/international 
Policy Influence Type Influenced training of practitioners or researchers
 
Description Opinion piece drawing in part on project's research findings
Geographic Reach Multiple continents/international 
Policy Influence Type Citation in other policy documents
URL https://www.sciencedirect.com/science/article/pii/S2211912418301056
 
Description PI nominated as Chair of the Standing Panel on Impact Assessment of the CGIAR
Geographic Reach Multiple continents/international 
Policy Influence Type Participation in a guidance/advisory committee
URL https://ispc.cgiar.org/blog/announcing-new-chair-standing-panel-impact-assessment-spia
 
Description LSMS-ISA measurement research project
Amount $200,000 (USD)
Organisation World Bank Group 
Sector Public
Country United States
Start 10/2015 
End 10/2019
 
Description research funds eMBed
Amount $48,000 (USD)
Organisation World Bank Group 
Sector Public
Country United States
Start 07/2016 
End 06/2017
 
Title alternative calculation of returns to agronomical trials 
Description we develop a method to obtain measurement from agronomical trials that account for different selection and behavioral adjustements and allow to draw lessons regarding average returns for average farmers, to complement the usual agronomical calculations of yield potentials. 
Type Of Material Physiological assessment or outcome measure 
Year Produced 2018 
Provided To Others? No  
Impact The paper outlining the method was invited for resubmission at the Proceedings of the National Academy of Science. It was resubmitted mid February, and the article outlining the methods will be available for use by others as soon as it is accepted. 
 
Description skills measurement 
Organisation University of Texas
Department Department of Psychology
Country United States 
Sector Academic/University 
PI Contribution we are leading a new collaboration on the measurement of personality in household surveys in developing countries, involving economists from the World Bank and the Inter American Development Bank, as well as psychologists from the University of Texas
Collaborator Contribution Researchers from the World Bank and the Inter American Development Bank have facilitated access to data from a variety of settings and collaborated on conceptualization of the project and research ideas. Researchers from the University of Texas have contributed by providing access to a very large complementary dataset, and provided feedback to the analysis.
Impact a working paper is under preparation. this is indeed a multidiciplinary collaboration between economists and psychologists.
Start Year 2016
 
Description skills measurement 
Organisation World Bank Group
Country United States 
Sector Public 
PI Contribution we are leading a new collaboration on the measurement of personality in household surveys in developing countries, involving economists from the World Bank and the Inter American Development Bank, as well as psychologists from the University of Texas
Collaborator Contribution Researchers from the World Bank and the Inter American Development Bank have facilitated access to data from a variety of settings and collaborated on conceptualization of the project and research ideas. Researchers from the University of Texas have contributed by providing access to a very large complementary dataset, and provided feedback to the analysis.
Impact a working paper is under preparation. this is indeed a multidiciplinary collaboration between economists and psychologists.
Start Year 2016
 
Description A stakeholder workshop in Nairobi 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact A stakeholder workshop was organized in Nairobi, in collaboration with IITA, in March 2016 to disseminate the early results of the study. Participants included representatives of the Kenyan government, different international institutions, academic institutions in Kenya, the private sector (input suppliers). The workshop provided an opportunity to share and receive early feedback on the results.
Year(s) Of Engagement Activity 2016
 
Description BMFG lunch presentation 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach National
Primary Audience Supporters
Results and Impact key insights of the research were included in lunch presentation at the Bill and Melinda Gates Foundation, and sparked questions and discussions implications.
Year(s) Of Engagement Activity 2018
 
Description NRM workshop 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Insights of research discussed during a workshop on natural resource management research in the CGIAR. Implications for the design of agricultural trials discussed with practioners.
Year(s) Of Engagement Activity 2018
URL https://ispc.cgiar.org/meetings-and-events/assessing-impact-research-managing-natural-resources-sust...
 
Description Podcast: VOXDEV: Why promising trials often fail to result in agricultural yield gains: Evidence from Kenya 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Public/other audiences
Results and Impact Read "Reconciling yield gains in agronomic trials with returns under African smallholder conditions" by Rachid Laajaj, Karen Macours, Cargele Masso, Moses Thuita, and Bernard Vanlauwe here.

To address global poverty and hunger, increasing the adoption of improved agricultural technologies is essential. Yet, while experimental trials might lead to positive outcomes, technologies are not always adopted at scale.

In this VoxDevTalk, Rachid Laajaj discusses insights from his research with Karen Macours, Cargele Masso, Moses Thuita, and Bernard Vanlauwe on smallholder farmers in rural areas of Western Kenya. The research team assessed the puzzle of why farmers might choose not to adopt or might not see success from promising technologies, even with financial and informational support to do so.

Their research provides quantitative evidence on how failing to adapt trials to farmers' specific conditions has direct implications for the take-up of new technologies. It also highlights some of the problems with the ways in which research is conducted, where the presence of a researcher impacts farming behavior. Better accounting for and enabling of real-world conditions early on in the research process would ensure experimental results are more credible, with significant implications for improving agricultural outputs among some of the world's poorest people.
Year(s) Of Engagement Activity 2020
URL https://voxdev.org/topic/agriculture/why-promising-trials-often-fail-result-agricultural-yield-gains...
 
Description Presentation World Bank 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Research findings discussed at lunch seminar at the World Bank. Seminar was very widely attended, including participation on web scream, by more than 90 participants. Results on skill measurement leading to re-evaluation of using such measures in upcoming surveys.
Year(s) Of Engagement Activity 2017
 
Description SPIA-PIM conference Nairobi 
Form Of Engagement Activity A talk or presentation
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Presentation of research results at a major conference organized jointly by the Standing Panel on Impact Assesment (SPIA) and the Policy, Institutions and Markets common research program of the CGIAR. Nairobi, 6-8 July. Conference was attended by 150-200 biophysical and social science practitioners, researchers and science leaders from CGIAR centers across the world, policy makers, donors and academic researchers. The presentation triggered a lot of interest, and follow-up discussions with various attendees are ongoing on the implications of the research findings from agricultural research.
Year(s) Of Engagement Activity 2017
URL https://ispc.cgiar.org/meetings-and-events/conference-impacts-international-agricultural-research-ri...
 
Description blog "Is the Current Agriculture Technology Strategy Unsuitable for Getting Smallholders Out of Poverty?" 
Form Of Engagement Activity Engagement focused website, blog or social media channel
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Professional Practitioners
Results and Impact Blog post was written by ISPC (with assistance of the PI, Karen Macours) to disseminate key insights resulting from funded research.
Year(s) Of Engagement Activity 2018
URL https://ispc.cgiar.org/blog/current-agriculture-technology-strategy-unsuitable-getting-smallholders-...
 
Description expert consultation presentation IFAD-FAO-WB-ISPC 
Form Of Engagement Activity A formal working group, expert panel or dialogue
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Policymakers/politicians
Results and Impact Key insights resulting from the funded research were shared at the presentation at the second Expert Consultation on "Focusing Agricultural and Rural Development Research and Investment on Achieving SDGs 1 and 2", jointly organized by FAO, IFAD, ISPC/CGIAR, and the World Bank on January 10-12, 2018. The presentations sparked lot's of interest and questions, and a request for a follow-up blog post.
Year(s) Of Engagement Activity 2018
 
Description presentation at academic workshop on learning and agricultural extension 
Form Of Engagement Activity Participation in an activity, workshop or similar
Part Of Official Scheme? No
Geographic Reach International
Primary Audience Other audiences
Results and Impact Results were also presented at an academic workshop on learning and agricultural extension in Clermont-Ferrand (France), organized by FERDI and SPIA. The 2.5 day workshop included presentations of about 10 high-level academic papers, and participation of about 20 academics working on the topic. The research from Kenya contributed by highlighting the complexity of the learning process and the implications for agricultural extension efforts. The workshop resulted in a monograph of policy briefs of the different presentations, including a policy brief on the early Kenya results.
Year(s) Of Engagement Activity 2016