Can we use experiments to understand institutions?

Read “Experiments about Institutions” by Michael Callen, Jonathan L. Weigel and Noam Yuchtman.

Last week Daron Acemoglu, Simon Johnson and James A. Robinson won the Nobel prize in Economic Sciences “for studies of how institutions are formed and affect prosperity.” In this week's episode of VoxDevTalks, Michael Callen and Jonathan Weigel explore the importance of institutions in economic research, the rise of experimental tools to measure institutional change and the concept of critical junctures.  

What are institutions?

The study of institutions has emerged as a result of social scientists’ aim to understand why some countries are rich and some are poor. Initially, this question was approached by examining proximate causes, such as variations in capital inputs or differences in human capital. However, this approach held no information on why some countries had greater capital stocks or more educated populations. This prompted economists to explore the underlying, fundamental drivers of growth such as culture and institutions. 

Douglass North famously defined institutions as the rules of the game (1990). These rules shape the incentives that govern our behaviour. Such ‘rules’ range from whether the rule of law prevails to the specific market structure of the labour market. Similarly, political structures shape incentives for both politicians and citizens. The motivations of an officeholder in a democracy, who must appeal to the average voter to secure re-election, differ greatly from those in an autocracy.

‘Which rules of the games exist in a given country crucially shape the behaviour of individuals and their incentives. That’s why we think they’re a first order determinant of all outcomes that economists care about.’ 

Since institutions are a fundamental cause of growth, they are often deep-rooted and hard to change. As a result, challenging the status quo and seeking power through alternative means is difficult. 

The evolution of studying institutions in economics

The study of institutions can be traced back to early philosophical theory that considered how humans interact. Economists first engaged with institutions, by using countries as the unit of analysis, with their own production function that combines labour, capital and a residual, that includes institutions. They then brought data to these models to examine how much of these differences in growth can be explained by the residual. 

These methods suggested that institutions did matter but the question remained on whether a causal link could be drawn between institutions and growth. The seminal work of Acemoglu, Johnson and Robinson, using settler mortality as an instrument for institutional quality, was a first econometric attempt at making a causal link between the two and hinted at the mechanisms that might be at work. Simultaneously, from the early 2000s, RCTs became increasingly popular due to an increased emphasis on casual identification. Consequently, it was logical to apply RCT tools to address key questions related to institutions and governance.

Can experiments be used to measure institutions?

Generating random variations in institutional quality is more challenging than the typical RCT. However, there are a growing number of experimental studies on institutional quality, often requiring close collaboration with local stakeholders. 

Callen, Weigel and Yuchtman (2024) measure the growing prevalence of this research (i.e. field experiments that study the fundamental ‘rules of the game’) by looking at research in the top 13 economic and political science journals. They find a rapidly growing group of papers at the intersection of these methods.

What are critical junctures? Why do they matter?

Callen and Weigel emphasise the importance of defining critical junctures as moments of deep uncertainty about future institutions. Both have lived and worked in countries that have gone through critical junctures and seen the importance of these moments in analysing institutional change. To study and analyse this concept they focus on beliefs i.e. what are individuals thinking institutions will look like over a given time horizon. 

Weigel emphasises the importance of also studying critical junctures where we do not observe institutional change. He argues we must not ignore these moments and the importance, therefore, of studying events in real time. By surveying elites and citizens, it is possible to identify disagreement or uncertainty, which can be indicative of a critical juncture. Some recent examples of critical junctures that did not lead to institutional change include: 

• Large anti-Netanyahu protests in Israel in 2023
• The Prigozhin march on Moscow

‘These are all moments in which there really was a time of fundamental uncertainty about which way it would go, it could of almost been like a coin flip. Which way it did land, that coin, shaped crucially the institutional pact and the outcomes that ensued.’

Experiments and their advantages in measuring institutional change

Despite the criticisms that are levied at RCTs regarding their external validity, they can offer the ability to draw causal linkages between institutions and outcomes. For example, simply comparing countries with higher tax collection and measuring civic engagement makes it difficult to establish clear causal relationships, whereas conducting an RCT, as done by Weigel in the Democratic Republic of Congo (DRC), that varied property tax rates to measure civic engagement can capture a causal link. 

There are clear benefits of such experiments, including the ability to perform primary data collection in real time. Often the measures required to understand institutional questions are not available in archival data, so RCTs offer unique opportunity to measure exactly the variable need.

Examples of experiments that have captured institutional change

There is increasing evidence that it is possible to embed RCTs in moments of big institutional change. Some highlighted examples include:  

• Olken (2010) – direct versus representative democracy in project selection in Indonesia. It is not possible to randomise a country into different electoral structures whereas varying these at a more granular level is possible. 
• Berman et al. (2019) - reducing fraud in elections increases citizens willingness to engage with the state, for example by paying taxes.  

The policy relevance of research on institutions

This type of research is particularly interesting as it involves working closely with partner organisations and being embedded in a country’s government or civil society. Weigel draws on his own experience working closely with the DRC Tax Authorities, and points to the work of Casey and others in Sierra Leone, building trust and common ground with local stakeholders.

There are now lots of examples of how these experiments can have significant policy relevance. For example, Callen reflects on how his own work in Afghanistan on a new monitoring technology that has now been scaled up elsewhere in Kenya and South Africa (Callen and Long 2015).  

References

Berman, E, M Callen, C Gibson, and J Long (2019), “Elections and government legitimacy in Afghanistan,” Journal of Economic Behavior & Organization, 168: 292–317.

Callen, M, and J D Long (2015), “Institutional corruption and election fraud: Evidence from a field experiment in Afghanistan,” American Economic Review, 105(1): 354–381.

Callen, M, J L Weigel, and N Yuchtman (2024), “Experiments about institutions,” Annual Review of Economics, 16: 105-131. Working paper available at: https://www.nber.org/papers/w31964.

North, D C (1990), Institutions, Institutional Change and Economic Performance, Cambridge University Press.

Olken, B A (2010), “Direct democracy and local public goods: Evidence from a field experiment in Indonesia,” American Political Science Review, 104(2): 243–267.

Weigel, J L (2020), “The participation dividend of taxation: How citizens in Congo engage more with the state when it tries to tax them,” The Quarterly Journal of Economics, 135(4): 1849–1903.