A key dilemma in Indian education is that while children are enrolled in school, they are not actually learning. Based on an experiment in Odisha, this article explores possible solutions to the learning deficit in secondary schooling. It finds that tailored remedial programmes improve learning and correct teachers’ beliefs regarding learning levels. However, gains from increasing teacher autonomy in lesson plans are not significant.
Editor's note: This article originally appeared on Ideas for India.
Raising student learning is an international priority – United Nations Sustainable Development Goal (SDG) 4 strives to attain quality education and learning for all. Amidst a global educational crisis, India stands at a crossroads. School enrolment rates in the country have increased at both the primary and secondary levels, but schools are not delivering quality education. Although students are in school, they are not actually learning, leaving many students multiple learning levels behind what should correspond to the grade they are in. Not only are learning levels low, but these also vary widely across students. Secondary school students have substantially different abilities even within the same classroom. Thus, there is an increased emphasis worldwide on reducing this within-classroom variation in learning levels and improving foundational skills (World Bank, 2018).
How can India ensure equitable and quality education for all? In recent research (Beg et al. 2023), we seek to address this question by exploring innovative solutions to bridge learning gaps in second school. We focus on two pivotal elements: remedial education programmes tailored to student levels, and the empowerment of teachers through increased autonomy.
Possible innovative interventions in secondary school learning
Existing evaluations from primary schools show that teaching children at their learning level instead of grade level can significantly improve overall educational achievement (Banerjee et al. 2017, Banerjee et al. 2010, Duflo et al. 2020, Duflo et al. 2011, Lakshminarayana et al. 2013). However, whether the same approach works in secondary schools is an open question for two reasons. First, it is unclear how wide the gaps are. On the one hand, to enter secondary school, students typically must pass high-stakes standardised tests. In principle these tests should screen out the weakest students, but our study finds that they are not effective at doing so. Moreover, secondary school students are older than primary school students, and thus gaps between the strongest and weakest students have had more time to widen. Second, even if large gaps do exist, remedial education interventions are largely untested at the secondary level. Is secondary school too late?
Moreover, even if a remedial learning approach is successful at the secondary level, how to scale it effectively is unclear. How should the approach be designed: should teachers have more flexibility and autonomy in content delivery, or should lessons and content be more standardised?
Giving teachers autonomy can matter for their performance, and thus for eventual learning outcomes. Indian government schools are a classic example of a highly regimented public sector service delivery with poor outcomes. Teachers (who can be considered front-line civil servants) are directed to follow and complete the grade-level curriculum on a strict timetable, even though the curriculum is at least one grade level beyond the learning level of about half the students. Such standardised teaching approaches and expectations for uniform implementation might be practical in a system with limited opportunities for mentoring or constraints in terms of teacher background and training. The alternative is to let teachers decide what and how to teach, increasing the potential for creativity and intrinsic motivation (Pearson and Moomaw 2005, Skaalvik and Skaalvik 2014) but also making it more possible for teachers to shirk by not carrying out the intervention. Is regimentation bad, taking discretion away from local decision-makers, or does it help ensure that programmes are delivered properly?
Experiment in Odisha
We study these two issues in the context of ninth grade students in Odisha. Odisha is one of India’s poorest states, with some of the lowest literacy rates in the country. It exemplifies the challenges of heterogeneous learning progress: our data shows that 50% of secondary school students fail to meet a basic international benchmark of mathematical knowledge; in contrast, the top 5% of students excel, outperforming top students from other low-income countries.
Our study had two broad goals. First, we evaluate an innovative programme called Utkarsh (‘flourishing’ in Hindi), which strives to improve learning at the secondary level within the constraints of the Indian public education system. Second, we test whether teacher flexibility affects the delivery of the programme and its benefits to students. We partnered with Transform Schools, People for Action (PFA), and the Odisha Department of School and Mass Education to conduct a randomised evaluation. We randomly assigned 300 government secondary schools in the Jaipur and Dhenkanal districts to one of three groups:
- Standard Utkarsh group: these schools received the original version of Utkarsh where teachers implemented the programme’s topics and lesson plans according to a pre-specified schedule.
- Flexible Utkarsh group: these schools received a version of Utkarsh where teachers were explicitly given more autonomy and partial flexibility to adapt topics and/or timelines according to the needs of their students.
- Comparison group: these schools did not receive the Utkarsh.
We collected extensive primary data for this study through baseline and endline surveys, and monitoring visits during the school year. During the baseline, students were administered a short survey and assessed on their Odia (the language of Odisha), Math, and English skills. Midline/monitoring visits were conducted to see whether the study students were present and if the programme was being implemented as intended. We then assessed the students again after the intervention. We also interviewed subject teachers and the headmaster from each school at baseline and endline.
Main findings
We report four main findings based on one school year of implementation for Class 9 students. First, students were substantially behind grade level – an average Class 9 student was 4 to 5 grades behind in English, Math, and Odia with substantial within school heterogeneity. In contrast to work by Djaker et al. (2022) in India and Bangladesh, teachers were largely aware that students were significantly behind grade level, although there was substantial uncertainty in their estimates. Second, both interventions (standard and flexible Utkarsh groups) improved student learning – by about 0.1 standard deviations (SDs)1, equivalent to a 58% increase in learning compared to the comparison group. Both versions of the intervention increased the likelihood of achieving Class 3 and Class 5 competency, but did not bring students up to Class 9 level mastery in the one-year period of the experiment. The interventions neither ‘crowded-out’ grade-level knowledge nor harmed students that were already at Class 9 competency, a key concern of introducing remedial lessons. Indeed, we find gains throughout the distribution of endline test scores and of baseline competency levels.
Third, implementation quality was high in both intervention arms despite no additional incentives for teachers to teach under the programme. Flexibility did not induce shirking. Teachers usually taught Utkarsh during the designated times but did not always teach the exact lesson prescribed for that day. Teachers in both standard Utkarsh and flexible Utkarsh groups modified the programme, at times choosing different topics and other times teaching non-Utkarsh or curriculum-level topics.
Math teachers in the flexible Utkarsh group were more likely to take-up the flexibility to teach more-advanced topics. This flexibility paid dividends in terms of improving PISA scores2 in Math and did not appear to come at the expense of performance on other remedial subjects or for weaker students. Finally, teachers formed more accurate beliefs about their students’ learning levels. Therefore, even though they believed that they and their students benefited from the programme, they assessed their students as being weaker compared with teachers in the comparison group.
Lessons and policy implications
Limited evidence exists for the effectiveness of remedial instruction at the secondary school level; existing evidence either uses interventions outside of the school day with tutors from outside the school system (Muralidharan et al. 2019) or focuses on grade-level material (Beg et al. 2022). However, we show that remedial instruction provided by existing teachers at the secondary level can increase student learning. This study is highly policy relevant, as this programme has been scaled within Odisha (with a population of 40 million) and the southern state of Karnataka (with a population of 60 million).
In our context, teachers were teaching grade-level content even when their students were well behind grade level for potentially three reasons: (i) they felt that they must complete the curriculum or perhaps they enjoyed teaching at that level, (ii) they were not aware of their students’ learning levels, or (iii) they found engaging with remedial teaching to be costly in terms of time and effort. This programme helped alleviate these concerns by explicitly telling teachers that setting aside time for this alternative, remedial curriculum was important, giving them the freedom to deviate from the grade-level curriculum. Teachers were aware that their students were below grade level but underestimated the extent of the deficit – the Utkarsh programme gave teachers more correct beliefs about the learning levels of their students. Finally, the programme provided materials and guides that made it simple to engage with remedial teaching. Teachers had been focused on completing the curriculum partly because that was the expectation, and partly because they did not have the tools to engage in remedial teaching. Once given tools and training that reduced the costs of remedial teaching, and permission to focus on it, teachers implemented the programme with a high degree of fidelity, increasing student learning without hurting grade-level competencies.
Our findings also provide new evidence on whether allowing point-of-service modifications improve the functioning in highly rigid bureaucracies. Much of the research on improving public-sector service delivery focuses on providing incentives to service providers or empowering community members to register (Bjorkman and Svensson 2009, Duflo et al. 2012). A growing literature emphasises that highly rigid bureaucracies may improve service delivery by increasing the autonomy among front-line civil servants (Bandiera et al. 2020, Bloom et al. 2015), though it is difficult to find optimal practices in a hierarchical bureaucracy (Banerjee et al. 2021). Whether these reforms would improve educational quality, however, is unclear: Piper et al. (2018) argue that scripted lessons improve student learning, but that some amount of flexibility is optimal. In our study flexibility is beneficial at the margin. Teachers use the opportunity to teach harder material when appropriate, improving students’ performance on the more difficult questions without decreasing performance on foundational knowledge. We also find evidence that flexibility might be beneficial for teachers’ morale and outlook of their jobs. However, the gains from providing additional flexibility are not large in magnitude, likely because teachers already exerted the freedom to deviate from the prescribed lesson plans.
Notes:
- Standard deviation is a measure used to quantify the amount of variation or dispersion of a set of values from the mean (average) value of that set.
- PISA or the Programme for International Student Assessment is a worldwide study by the Organisation for Economic Co-operation and Development (OECD), intended to evaluate educational systems by measuring 15 year-old students’ performance in mathematics, science, and reading.
References
Bandiera, O, MC Best, AQ Khan and A Prat (2020), ‘The Allocation of Authority in Organizations: A Field Experiment with Bureaucrats’, NBER Working Paper 26733.
Banerjee, Abhijit, et al. (2017), “From Proof of Concept to Scalable Policies: Challenges and Solutions, with an Application”, Journal of Economic Perspectives, 31(4): 73-102.
Banerjee, Abhijit, Raghabendra Chattopadhyay, Esther Duflo, Daniel Keniston and Nina Singh (2021), “Improving Police Performance in Rajasthan, India: Experimental Evidence on Incentives, Managerial Autonomy, and Training”, American Economic Journal: Economic Policy, 13(1): 31-66.
Banerjee, Abhijit, Rukmini Banerji, Esther Duflo, Rachel Glennerster and Stuti Khemani (2010), “Pitfalls of Participatory Programs: Evidence from a Randomized Evaluation in Education in India”, American Economic Journal: Economic Policy, 2(1): 1-30.
Beg, Sabrin, Waqas Halim, Adrienne M Lucas and Umar Saif (2022), “Engaging Teachers with Technology Increased Achievement, Bypassing Teachers Did Not”, American Economic Journal: Economic Policy, 14(2): 61-90.
Beg, S, A Fitzpatrick, J Kerwin, A Lucas and KW Rahman (2023), ‘Teacher Flexibility and School Productivity: Remedial Secondary Education in India’, Working Paper.
Björkman, Martina and Jakob Svensson (2009), “Power to the People: Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda”, The Quarterly Journal of Economics, 124(2): 735-769.
Bloom, Nicholas, Renata Lemos, Rafaella Sadun and John van Reenen (2015), “Does Management Matter in Schools?”, The Economic Journal, 125(584): 647-674.
Djaker, S, AJ Ganimian and S Sabarwal (2023), ‘Out of Sight, Out of Mind? The Gap between Students’ Test Performance and Teachers’ Estimations in India and Bangladesh’, EdWorkingPaper No. 23-750, Annenberg Institute at Brown University.
Duflo, A, J Kiessel and A Lucas (2020), ‘Experimental Evidence on Alternative Policies to Increase Learning at Scale’, NBER Working Paper 27298.
Duflo, Esther, Pascaline Dupas and Michael Kremer (2011), “Peer Effects, Teacher Incentives, and the Impact of Tracking: Evidence from a Randomized Evaluation in Kenya”, American Economic Review, 101(5): 1739-1774.
Duflo, Esther, Rema Hanna and Stephen P Ryan (2012) “Incentives Work: Getting Teachers to Come to School”, The American Economic Review, 102(4): 1241-1278. Available here.
Lakshminarayana, Rashmi, et al. (2013), “The Support to Rural India’s Public Education System (STRIPES) Trial: A Cluster Randomised Controlled Trial of Supplementary Teaching, Learning Material and Material Support”, PLoS ONE, 8(7): e65775.
Pearson, L Carolyn and William Moomaw (2005), “The Relationship Between Teacher Autonomy and Stress, Work Satisfaction, Empowerment, and Professionalism”, Educational Research Quarterly, 29(1): 38-54. Available here.
Piper, B, Y Sitabkhan, J Mejia and K Betts (2018), ‘Effectiveness of Teachers’ Guides in the Global South: Scripting, Learning Outcomes, and Classroom Utilization’, Occasional Paper, RTI Press.
Skaalvik, Einar M and Sidsel Skaalvik (2014), “Teacher Self-Efficacy and Perceived Autonomy: Relations with Teacher Engagement, Job Satisfaction, and Emotional Exhaustion”, Psychological Reports, 114(1): 68-77.
World Bank (2018), ‘World Development Report 2018: Learning to Realize Education’s Promise’.