The Learning Crisis: Combining Incentives and Inputs to Raise Student Achievement

As school enrolment in low- and middle-income countries has increased substantially in the last couple of decades, attention has instead turned to the poor quality of education. This ”learning crisis” (UNESCO 2013) manifests itself in primary school students without basic skills in language and mathematics, and high school students being vastly outperformed by their peers in high-income countries (World Bank 2018). In this policy brief, I give a very brief background to the learning crisis and report on a research project we have implemented and evaluated in the Democratic Republic of Congo (DRC) with the aim of improving student learning in primary education. The intervention consisted of an incentivized program to stimulate more usage of existing textbooks for self-study, and the impact was evaluated through a randomized experiment (Falisse, Huysentruyt and Olofsgård 2019).

Education systems in many low- and middle-income countries fail to deliver actual learning at the level necessary for people and societies to thrive. According to leading international assessments of literacy and numeracy, the average student in low-income countries performs worse than 95 percent of the students in high-income countries. According to an assessment of second-grade students in India, more than 80 % could not read a single word from a short text or conduct two-digit subtraction. Students perform poorly also in some European middle-income countries; more than 75 % of students in Kosovo and the Republic of North Macedonia perform worse than the 25^th percentile in the average OECD country (World Bank 2018). The reasons behind the learning crisis are of course many, ranging from poorly trained and absent teachers, lack of financial resources for infrastructure and learning material, malnutrition and lacking early childhood development, and sometimes weak demand.

Textbooks for Self-Study in the DRC

The learning crisis is particularly evident in fragile, low income countries. This is also where the major challenge to achieve the 2030 Sustainable Development Goal 4 of quality education to all lies (World Bank, 2018). Yet, very few interventions targeting student achievement have been evaluated in the most fragile countries of the world (Glewwe and Muralidharan 2016). This is a concern, since interventions that work in poor but stable environments may not be feasible or effective in even more resource constrained and violent environments (Burde and Linden 2013). In particular, there is an extra value in identifying interventions that are not only cost efficient, but also low cost in absolute terms and simple and transparent.

Projects focusing on school inputs have often yielded surprisingly disappointing results (Glewwe and Muralidharan 2016). One example is interventions focusing on textbook distribution despite belief in their effectiveness and investments from donors and governments (Glewwe, Kremer and Moulin 2009; Sabarwal et al. 2014). One major challenge with textbooks is that they for different reasons are often not used by teachers or pupils, and certainly not to their potential (e.g. Sabarwal et al. 2014). This raises the question of whether the potential of textbooks can be leveraged through incentives on their usage. A couple of recent papers have found that it is indeed the combination of inputs (including textbooks) and incentives that is critical to yield a significant impact on student test scores (Mbiti et al. 2019; Gilligan et al. 2018).

Following up on this idea we collaborated with the Dutch NGO Cordaid that is running a program in primary education in South Kivu, in eastern DRC, in 90 schools. We designed an intervention that encouraged 5th and 6th grade students from 45 randomly selected schools to regularly take home textbooks and use them for self-study. We used a mix of financial and non-financial incentives focused on the students, such as a public display of stars assigned to each student that brought math and French textbooks home and back in good condition, and an in-kind gift of pens and pencils for all students in classes regularly participating in the routine. We also offered participating schools a small flat compensation to compensate for lost and damaged books. The main goals of the intervention were to increase student achievement and to affect their aspirations for further study and more qualified careers.

To measure student achievement, we rely on self-conducted tests in the French language and math, but also high stakes national exam scores that determine eligibility to secondary education. Following the literature, we analyze test results using a model that assumes that baseline test scores capture student learning up to that point, so once this is controlled for end line results capture cleanly the added value of the intervention introduced. We also carefully address potential statistical problems due to slight unbalance between treatment and control groups, students from baseline not present at end line and poor compliance with the intervention in a small set of schools. The results are generally robust across different specifications of the details of the model.

We emphasize three main sets of results. First, we find that the students in the treatment schools (those selected to receive the books) scored significantly better than those in control schools on the French language tests. The estimated improvement was 1/3 of a standard deviation, which compares favourably with other interventions in developing countries targeting student test scores (Kremer et al. 2013). On the other hand, we found no significant impact on math scores. We cannot tell for sure why we observe this difference between French and math, but it should be noted that both textbooks were in French, suggesting that language could be learned from both books. It has also been suggested that math requires more supervision than language and that math is more ”vertical” in terms of skills progression while language is more ”horizontal”. That is, if students are far behind the curriculum in the textbook, they don’t have the necessary basic building blocks to understand the math problems. But for language, this matters less, as progress can be made in different areas more independently.

Secondly, students in treatment schools were more likely to sit and pass the national exam. This is important as this is a requirement for the continuation of schooling at a higher level. More qualified jobs, and jobs that require more French language skills, typically require at least secondary schooling. This is also consistent with the finding that students exposed to the intervention were more likely to aspire to non-manual jobs. Finally, the intervention was low cost and cost-efficient. In particular in fragile environments with very limited resources, this is essential. The intervention is also easy to implement and transparent and does not give raise to incentives to cheat as has been the case in some interventions linking incentives directly to student test performance.

Conclusions

The current key challenge in education policy in low- and middle-income countries is to improve student achievement while continuing the successful increase in enrolment despite often serious constraints in complementary inputs in the education production function. Financial resources for school infrastructure and material are limited, competent and motivated teachers are in short supply, and weak parental support and little early childhood development leaves children unprepared for sometimes too ambitious curricula. In such circumstances simple and low-cost interventions that make better use of existing resources are particularly valuable. In this project we designed and evaluated such an intervention, using incentives to stimulate more usage of existing textbooks, in a particularly challenging environment, Eastern DRC. We find a positive impact on French language skills and higher student aspirations as shown through greater participation in national exams required for continued education. On the other hand, we find no impact on math test scores. Serious sustainable improvement in student learning in a country like the DRC requires wholesale reforms to the education sector and substantially increased financial resources. Realistically, this is a long-run ambition. In the meanwhile, small low-cost interventions that match incentives with existing resources can significantly increase student achievement also in the short run.

References

• Burde, Dana and Leigh L. Linden, 2013. “Bringing Education to Afghan Girls: A Randomized Controlled Trial of Village-Based Schools.” American Economic Journal: Applied Economics, 5(3), 27-40.
• Falisse, Jean-Benoit, Marieke Huysentruyt and Anders Olofsgård, 2019. “Incentivizing Textbooks for Self-Study: Experimental Evidence on Student Learning from the Democratic Republic of Congo”, Working Paper.
• Gilligan, Daniel O., Naureen Karachiwalla, Ibrahim Kasirye, Adrienne M. Lucas, Derek Neal, 2018. “Educator Incentives and Educational Triage in Rural Primary Schools.” NBER WP 24911.
• Glewwe, Paul, Michael Kremer, and Sylvie Moulin, 2009. “Many Children Left Behind? Textbooks and Test Scores in Kenya.” American Economic Journal: Applied Economics, 1(1): 112-35.
• Glewwe, Paul and Karthik Muralidharan, 2016. “Improving Education Outcomes in Developing Countries: Evidence, Knowledge Gaps, and Policy Implications”, in Handbook of the Economics of Education, pp. 653-743. Elsevier.
• Kremer, Michael, Conner Brannen, and Rachel Glennerster, 2013. “The Challenge of Education and
• Learning in the Developing World.” Science 340, 297-300.
• Mbiti, Isaac, Karthik Muralidharan, Mauricio Romero, Youdi Schipper, Constantine Manda, Rakesh Rajani, 2019. “Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania.” NBER WP 24876.
• Sabarwal, Shwetlena, David K. Evans, and Anastasia Marshak, 2014. “The permanent input hypothesis: the case of textbooks and (no) student learning in Sierra Leone”, Policy Research working paper, no. WPS 7021. Washington, DC: World Bank Group.
• UNESCO, 2013. “The Global Learning Crisis: Why every child deserves a quality education”, UNESCO, Paris.
• World Bank, 2018. “World Development Report 2018: Learning to Realize Education’s Promise”, Washington DC: World Bank.

Disclaimer: Opinions expressed in policy briefs and other publications are those of the authors; they do not necessarily reflect those of the FREE Network and its research institutes.