Improving foundational learning in Tanzania through teacher rewards

Background

Six out of ten children and youth worldwide – and 88% in Africa – do not reach minimum proficiency levels in reading and numeracy (UNESCO 2017). Most of these children complete primary school but do not learn foundational skills. These basic skill deficiencies carry over to skills that require children to process and act on information (Gust et al. 2022). 

There is a growing consensus that “business as usual” policies, including traditional education investments in school inputs, are not going to improve learning levels in low-income settings (Glewwe and Muralidharan 2016, World Bank WDR 2018). Teacher management is of particular concern: teachers play a central role in the education production function (Hanushek and Rivkin 2012, Chetty et al. 2014b,a); a large share of the typical education budgets is devoted to their compensation. Further, the significant levels of classroom absence among teachers has important fiscal implications (Bold et al. 2017, Muralidharan et al. 2017).   

Given these challenges, teacher performance pay could be an important policy lever to catalyse the productivity of the education sector in developing countries. Teacher performance pay (teacher incentives) offers a financial reward for teacher performance, typically based on independent assessments of students’ learning. Teacher performance pay systems have been shown to improve student learning, particularly in low-income settings (Bruns and Luque 2014), but questions about how to best design them remain. For example, it is unclear how incentives should be structured and communicated to maximise impact. 

In our recent research (Mbiti et al. 2023), we use an experiment to compare a performance pay scheme that rewards teachers based on the skill level their students attain versus designs that use pay for percentile incentive designs (a type of rank-order tournament where rewards are based on learning growth within a series of rank order contests). This question is relevant for researchers, practitioners, and policy makers interested in the feasibility of scaling performance incentives.

Cash-incentives for foundational learning

We investigate the incentive design question in a teacher performance pay programme named KiuFunza in Tanzania, implemented by Twaweza East-Africa, a civil-society organisation. Our study examines the effectiveness of two incentive designs: a Pay for Percentile system (a rank-order tournament) and a “Levels” system that features rewards for multiple skill thresholds. The multiple-threshold Levels is an innovation, based on an earlier experiment with a single threshold incentive design (Mbiti et al. 2019). The two incentive programmes were implemented in 2015-16, as part of a nationally representative field experiment in 180 public primary schools (60 schools P4Percentile, 60 schools Levels, 60 control schools).

The main difference between the two designs that we evaluated is the mapping between test results and teacher earnings:

• Percentile Pay is a tournament system, in which students are placed in starting ability groups across all participating schools. Teachers’ earnings increase with the ranking of ‘their students’ within these groups (see Barlevy and Neal, 2012). 
• Levels Pay: a novel proficiency design, where a teacher earns a bonus for each curriculum skill test (for example, reading words and sentences, and doing addition) that a student passes. Harder skills (with lower passing rates) have higher payouts.

Both designs targeted grade I-III teachers in public primary schools and had equal per-student bonus budgets, with a per teacher cash reward of about 3.5% of the average annual teacher salary in 2016.

Our study offers four sets of insights

First, both programmes improved learning by the equivalent of about one third of a year of schooling. Compared to the effect sizes found in other interventions, the effects in these experiments range from middling (0.10 SD or the median effect size in the distribution presented by Evans and Yuan 2022) to substantial (0.22 SD, close to the 75th percentile in the effect distribution). We use student assessment data from both incentivised tests – that are used to calculate the bonus – and non-incentivised research tests: the learning effects are consistent across both types of data. We also find that both programmes increased the number of students sitting the exam – which we interpret as a sign that students have become more valuable in the eyes of the teachers. 

Second, Levels had a slightly larger learning impact than Percentile Pay, particularly for Kiswahili reading. Levels reduced the share of students repeating a grade by 24% (a decision by the school committee that is unrelated to bonus pay), but Percentile Pay did not. 

Third, we reject the hypothesis that the Levels programme leads to “threshold effects”, with teachers focusing effort only on students close to the passing thresholds. In the second year, we cannot reject the hypothesis that the treatment effects for each baseline score quintile are equal. 

Fourth, we find no effects, positive or negative, on learning in grades or subjects that are not given incentives – so nothing to suggest that teacher and/or head teachers cut back on non-incentivised subjects, or shifted resources to incentivised grades. 

Finally, in this paper we find suggestive evidence of changes to teacher behaviour including, increased teacher attendance and more frequent use of positive teaching practices (e.g. calling students by name). 

Key takeaways

Overall, our findings show that offering teachers a simple and modestly sized performance reward linked to learning can improve their performance at current levels of professional development. An important practical finding is that the Levels system is easier to implement, manage and communicate. Moreover, the Levels system has higher cost-effectiveness than Pay for Percentile and is relatively cost-effective in comparison to other interventions that aim to improve learning. 

Based on the 2015-16 impact results, the Tanzanian Ministry for Local Government asked Twaweza to test a performance pay programme that can work at larger scale. This follow-up program (KiuFunza 3) used the Levels incentive design evaluated in our paper and focused on practical innovations to reduce unit costs while maintaining quality and trust. KiuFunza 3 was implemented in the 2019, 2020 and 2021 school years, and evaluated in a new sample of 100 treatment and 100 controls schools. We find that the learning impact in this implementation is higher and, with lower implementation costs, the cost-effectiveness of the incentive program has increased substantially. We describe the programme and preliminary results in a briefing (Schipper et al. 2022). Based on the KiuFunza 3 results, Twaweza is currently implementing a limited scale-up of KiuFunza in 265 in the worst performing schools in Tanzania, with support from the Hempel Foundation. 

References

Barlevy, G and D Neal (2012), “Pay for Percentile.” American Economic Review 102: 1805–31. https://doi.org/10.1257/aer.102.5.1805

Bold, T, D Filmer, G Martin, E Molina, B Stacy, C Rockmore, J Svensson, and W Wane (2017), “Enrollment without Learning: Teacher Effort, Knowledge, and Skill in Primary Schools in Africa.” Journal of Economic Perspectives 31: 185–204. https://doi.org/10.1257/jep.31.4.185

Bruns, B and J Luque (2015), “Great teachers: How to raise student learning in Latin America and the Caribbean.” World Bank Publications.

Chetty, R, J N Friedman, and J E Rockoff (2014a), “Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates.” American Economic Review 104: 2593–2632. https://doi.org/10.1257/aer.104.9.2593

Chetty, R, J N Friedman, and J E Rockoff (2014b), “Measuring the impacts of teachers II: Teacher value-added and student outcomes in adulthood.” American Economic Review 104: 2633–2679. https://doi.org/10.3386/w19424

Evans, D K, and F Yuan (2022), “How Big Are Effect Sizes in International Education Studies?” Educational Evaluation and Policy Analysis 44: 532–540. https://doi.org/10.3102/01623737221079646

Glewwe, P, and K Muralidharan (2016), Chapter 10 - Improving Education Outcomes in Developing Countries: Evidence, Knowledge Gaps, and Policy Implications, in: Hanushek, E.A., Machin, S., Woessmann, L. (Eds.), Handbook of the Economics of Education. Elsevier, pp. 653–743. https://doi.org/10.1016/B978-0-444-63459-7.00010-5

Gust, S, E A Hanushek, and L Woessmann (2022), “Global Universal Basic Skills: Current Deficits and Implications for World Development”, RISE Working Paper No. 22/114.

Hanushek, E A, S G Rivkin (2012), “The Distribution of Teacher Quality and Implications for Policy.” Annual Review of Economics 4: 131–157. https://doi.org/10.1146/annurev-economics-080511-111001

Mbiti, I, K Muralidharan, M Romero, Y Schipper, C Manda, and R Rajani (2019), “Inputs, Incentives, and Complementarities in Education: Experimental Evidence from Tanzania.” Quarterly Journal of Economics 134: 1627–1673. https://doi.org/10.1093/qje/qjz010

Mbiti, I, M Romero, and Y Schipper (2023), “Designing Effective Teacher Performance Pay Programs: Experimental Evidence from Tanzania.” The Economic Journal. https://doi.org/10.1093/ej/uead010

Muralidharan, K, J Das, A Holla, and A Mohpal (2017), “The fiscal cost of weak governance: Evidence from teacher absence in India.” Journal of Public Economics 145: 116–135. https://doi.org/10.1016/j.jpubeco.2016.11.005

Schipper, Y, I Mbiti, and M Romero (2022), “Designing and Testing a Scalable Teacher Incentive Programme in Tanzania.” Research on Improving Systems of Education (RISE). https://doi.org/10.35489/BSG-RISE-RI_2022/044

UNESCO (2017), More Than One-Half of Children and Adolescents Are Not Learning Worldwide. Fact Sheet No. 46, September 2017 UIS/FS/2017/ED/46. 

World Bank (2018), World Development Report 2018 —LEARNING to Realize Education’s Promise. World Bank, Washington D.C.

World Bank (2020), COST-EFFECTIVE APPROACHES TO IMPROVE GLOBAL LEARNING: What does recent evidence tell us are “Smart Buys” for improving learning in low- and middle-income countries? Recommendations of the Global Education Evidence Advisory Panel. World Bank, Washington D.C.