There are large gaps between policy and practice in education. New systematic analysis shines a light on this 'policy-practice' gap in 50 countries during COVID-19 and motivates increased focus on policy implementation.
Read 'Mind the gap between education policy and practice,' & 'Understanding gaps between policy and practice,' by Noam Angrist and Stefan Dercon.
Education was hit hard by COVID-19 with over a billion children out of school at the height of the pandemic. In this episode of VoxDevTalks, Noam Angrist speaks to Tim Phillips about how the pandemic provided an opportunity to examine policy implementation and effectiveness.
Measuring education policies that were introduced during the pandemic
Governments responded rapidly by closing schools and attempting to introduce a range of new educational policies such as internet-based learning, educational radio show rollouts, encouraging teachers to call-in, and sending paper packets home. The World Bank, UNICEF and UNESCO teamed up to survey government officials in around 100 countries to see which policies were actually being introduced by governments. Although not perfect, the data provided a systematic review of policies that were introduced during different stages of the pandemic.
Measuring educational policy effectiveness
Capturing policy effectiveness requires measures of both the policies introduced and the actual policies that households received. To measure this effectiveness, Noam Angrist and Stefan Dercon link data on policies that were introduced with household level survey data that captured what households said they were actually receiving.
They design a new measure of ‘policy practice’ that quantifies the gap between policy and practice for around 30 countries. They then consider how much it systematically varies between countries and regions and what components may be explaining the gap.
‘We need to focus both on good policy, but crucially on policy implementation, that we have historically understudied.’
Although they do not measure educational outcomes, understanding what policies actually reached households is critical; households receiving educational policies is a necessary condition for learning. In the past there was significant focus on getting kids into schools (i.e. the Millennium Goals were focused on this rather than learning outcomes) whereas more recently we have seen a shift towards improving learning outcomes (e.g. the Sustainable Development Goals or the World Bank Human Capital project).
How does the policy-practice gap vary between countries and regions?
There is large variation in the policy-practice gap between regions. Latin America stands out as a bright spot with a gap of only around 5% compared to sub-Saharan Africa, where they observe that over 70% of households were not receiving a policy that the government said it was rolling out.
Interestingly, there does not seem to be a strong positive correlation between the gap and GDP per capita. Noam Angrist points to the Middle East which has high GDP per capita but a big policy practice gap (mirrored in learning outcomes). Having abundant resources does not automatically lead to effective service delivery.
‘This is about service delivery. This is not a problem that you can only throw money at. You have to have really high-quality implementation systems to ensure that policy practice gaps are closed and ensure learning.’
Were government educational policies realistic?
Part of these policy-practice gaps may be explained by policies being ‘naïve’, i.e. a government that introduces a national radio educational programme when the majority of households do not have radios. In their research, they find that on average most policies were realistic and that the majority of the gap observed can be explained by ineffective service delivery.
Since poor service delivery may explain a significant portion of the policy-practice gap, this research should motivate a more in-depth research agenda, with greater emphasis on policy implementation rather than just policy formulation.
The large policy-practice gap in sub-Saharan Africa
Although the COVID-19 pandemic brought rapid policy change and adaptation, service delivery remained poor. To bridge the policy-practice gap, greater emphasis must be placed on effective policy implementation. Pre-COVID data on early childhood development paints a similar story of weak implementation and regional variation with Latin America standing out.
Future work on policy implementation in education
Noam Angrist points to other potential research areas including measuring civil service effectiveness and capturing human capital in the civil service, that might unpack why Latin America, for example, had a small policy-practice gap.
The establishment of the What Works Hub for Global Education in Oxford, funded by FCDO and the Gates Foundation, will help spearhead this research, generating and supporting implementation science in education.
References
Angrist, N, and S Dercon (2024), “Understanding gaps between policy and practice,” What Works Hub for Global Education, Working Paper.
Angrist, N, and S Dercon (2024), “Mind the gap between education policy and practice,” Nature Human Behaviour. https://doi.org/10.1038/s41562-024-02013-4.