The multidimensionality of school choice: Evidence from Trinidad and Tobago

Student performance on standardised tests is typically the main measure of school quality. Important decisions (such as school closures, the design of teacher performance pay schemes, and school accountability procedures) often rely on test score metrics. These metrics may be simple test score averages, or more sophisticated measures of school effectiveness at improving test scores. The more sophisticated measures approximate schools’ causal impacts and are often called value-added. Schools do affect student test scores, but they also influence important longer-run outcomes such as crime, risky behaviours, college-going, and earnings (Hanushek 1971, Heckman, Stixrud and Urzua 2006, Kautz et al. 2017, Jackson 2018, Jackson et al. 2020). This leads one to wonder whether schools that improve longer-run outcomes are the same schools that causally improve academic test scores. If not, basing policy decisions on test-score impacts (which is common) may not improve the broader adult outcomes that policymakers and parents may value. 

Analysing the causal impacts of attending public secondary schools

In a recent paper (Beuermann, Jackson, Navarro-Sola and Pardo 2022), we use administrative data covering the full population of Trinidad and Tobago to identify the causal impacts of attending 132 individual public secondary schools (98.5% of all schools in the country) on multiple dimensions. To remove the influence of selection and uncover causal effects, we exploit the fact that most students (who we can identify in our data) are assigned to secondary schools for reasons beyond their (or parents' and principals’) control. 

We show that schools have meaningful causal impacts on many academic and non-academic outcomes, such as test scores in high-stakes and low-stakes exams, school dropout, youth crime, teen motherhood, and adult formal employment. However, the relationship between school impacts on high-stakes tests and on other outcomes is surprisingly low (Figure 1). For example, the correlations between value-added on high-stakes exams and non-dropout or being formally employed are 0.12 and  0.15, respectively. These results imply that school output is multidimensional, such that schools that improve high-stakes test scores are often not those that improve outcomes related to broader adult wellbeing (which parents may value). Accordingly, policymakers should be cautious (and thoughtful) regarding using test-score impacts as the sole measure in accountability systems and incentive pay schemes, and should instead adopt a more holistic view of school quality. 

Figure 1 Relationships between school-level value-added across different outcomes

Note: The x-axis plots the high-stakes value-added of each school. The y-axis represents the value-added of each school on the other measured outcomes. Tendency lines and the correlation coefficients (net of estimation errors) are added to each panel.

Parental preferences for schools based on test-score outcomes 

In the second part of our work, we seek to better understand parental preferences for schools. If parents can discern schools that improve outcomes (i.e. that have high value-added), school choice policies can increase efficiency in the production of education by aligning schools’ incentives with parents’ preferences (Friedman 1955, Chubb and Moe 1990). However, previous evidence shows that parental preferences for schools are not systemically related to school value-added on test scores (MacLeod and Urquiola 2019, Beuermann and Jackson 2022). The few studies that directly examine preferences for school academic value-added conclude that parents may not discern value-added per se (Rothstein 2006, Abdulkadiroglu et al. 2020), casting doubt on the likely efficacy of school choice. However, we point out that this evidence may not imply that school choice is not beneficial. That is, if parents value schools that improve non-academic outcomes, which are weakly related to test score impacts, school choice may actually improve outcomes valued by parents but are simply not captured by standardised tests. If school impacts on these other dimensions are not jointly analysed with the test score value-added, it may lead to wrong inferences about parental preferences and the benefits of school choice.

The influence of different types of outcomes on school choice

To assess this possibility, we link parents’ ranked lists of schools to the estimated causal impacts on the different academic and non-academic outcomes, and employ discrete-choice models to infer preferences for schools. Our results show that parents assign higher rankings to schools that are closer by, higher-performing schools, and those with higher-achieving peers, consistent with existing studies (Burgess et al. 2015, Abdulkadiroglu et al. 2020). We also present several novel results. First, parents of higher-achieving children choose schools with larger positive causal impacts on high-stakes exams test scores. Second, looking at non-academic outcomes, parents prefer schools that reduce youth arrests, reduce teen births, and increase formal labour market participation. These patterns persist even conditional on the average of the school outcomes, the distance to school, and the academic quality of the peers. 

These results suggest that parents may use reasonable measures of school quality when making investment decisions for their children – a requirement for the potential benefits of school choice (Friedman 1955). The fact that parents do not only choose schools that improve academics but also those that improve non-academic and longer-run outcomes suggests that the benefits to school choice may extend to a wide range of outcomes above and beyond test scores. This result also provides a plausible explanation for the weak relationship between parental preferences for schools and schools’ test score impacts (MacLeod and Urquiola 2019), and it suggests that policy evaluations based solely on test scores may be misleading about the welfare effects of school choice.

Differential school choice patterns by incoming academic achievement

We also document key differences in the choices between parents of high-achieving students and low-achieving students: high-achieving students’ choices are relatively more strongly related to schools’ impacts on high-stakes exams than impacts on non-academic outcomes, whereas the opposite is true for the choices of low-achieving students (Figure 2). 

Figure 2 Relative comparisons of choice model’s estimated coefficients

Note: The connected lines represent the difference between the choice model estimates on the school impacts of three non-academic outcomes and the choice model’s estimate on the school impacts of the high-stakes exam measure, scaled by the log distance estimate. This difference is computed separately for each (ventile of baseline academic achievement)×(gender) cell. The x-axis represents the individual baseline academic achievement level ventile (measured before secondary school enrolment). The estimates result from the a choice model that includes schools’ causal impact estimates for all outcomes, the school-level averages for all outcomes, peer quality and log distance, school-level control variables, a measure of the estimated likelihoods of school rejection, and interactions of these likelihoods with all school attributes.

This suggests that market forces may drive competition more strongly to raise test scores among schools serving high-achieving populations and to raise non-academic outcomes among schools serving low-achieving populations. If these differences reflect parents’ true preferences, this may be efficient. However, if they reflect differences in information, there may be value to providing information to parents regarding the causal impacts of schools (as opposed to school averages) on a wide array of both academic and non-academic outcomes.¹ This may improve the decisions of all parents and could increase the potential allocative efficiencies and competitive benefits of school choice.

References

Abdulkadiroglu, A, P Pathak, J Schellenberg and C Walters (2020), “Do Parents Value School Effectiveness?”, American Economic Review 110: 1502–1539.

Ainsworth, R, R Dehejia, C Pop-Eleches and M Urquiola (2020), “Information, Preferences, and Household Demand for School Value Added”, NBER Working Paper 28267.

Beuermann, D W and C K Jackson (2022), “The Short and Long-Run Effects of Attending The Schools that Parents Prefer”, Journal of Human Resources 57: 725–746.

Beuermann, D W, C K  Jackson, L Navarro-Sola and F Pardo (2022), “What is a Good School, and Can Parents Tell? Evidence on the Multidimensionality of School Output”, Review of Economic Studies, forthcoming.

Burgess, S, E Greaves, A Vignoles and D Wilson (2015), “What Parents Want: School Preferences and School Choice”, The Economic Journal 125: 1262–1289.

Chubb, J E and T M Moe (1990), Politics, Markets, and America’s Schools, The Brookings Institution.

Friedman, M (1955), The Role of Government in Education, University of Chicago Press. 

Hanushek, E A (1971), “Teacher Characteristics and Gains in Student Achievement: Estimation using Micro Data”, American Economic Review 61: 280–288.

Heckman, J J, J Stixrud and S Urzua (2006), “The Effects of Cognitive and Noncognitive Abilities on Labor Market Outcomes and Social Behavior”, Journal of Labor Economics 24: 411–482.

Jackson, C K (2018), “What Do Test Scores Miss? The Importance of Teacher Effects on Non-Test Score Outcomes”, Journal of Political Economy 126: 2072–2107.

Jackson, C K, S Porter, J Easton, A Blanchard and S Kiguel (2020), “School Effects on Socioemotional Development, School-Based Arrests, and Educational Attainment”, American Economic Review: Insights 2: 491–508.

Kautz, T, Heckman, J J, R Diris, B Ter Weel and L Borghans (2017), “Fostering and Measuring Skills: Improving Cognitive and Non-cognitive Skills to Promote Lifetime Success”, NBER Working Paper 20749.

Macleod, W B and M Urquiola (2019), “Is Education Consumption or Investment? Implications for the Effect of School Competition”, Annual Review of Economics 11: 563–589.

Rothstein, J M (2006), “Good Principals or Good Peers? Parental Valuation of School Characteristics, Tiebout Equilibrium, and the Incentive Effects of Competition among Jurisdictions”, American Economic Review 96: 1333–1350.

Endnotes

1 In an experimental study in Romania, Ainsworth et al. (2020) find that distributing information on school academic value-added led households (particularly those with low-achieving students) to attend more effective schools.