Information is power: Evidence from Mexico City’s Labour Court

Recent research has shown that incremental institutional reforms can improve the delivery of public services in lower-income countries (e.g. Khan et al. 2016, Duflo et al. 2013). While there have been many attempts to improve the functioning of courts, progress is hampered by a lack of detailed knowledge about the sources of inefficiency in legal institutions. 

Courts play a fundamental role in underpinning markets and constraining private power in developed markets, but they function poorly in most developing countries. Outcomes are unpredictable, parties are misinformed, and inefficient processes lead to slow decisions and large case backlogs. Although the ineffectiveness of courts in developing countries is widely acknowledged, formulating effective reforms leading to better-functioning institutions is impractical without a more detailed understanding of why the institutions fail.

The study: Case outcomes and Mexico City Labour Court’s (in)efficiency

We carry out a randomised field experiment with the Mexico City Labour Court (MCLC), providing information on predicted case outcomes to parties in ongoing cases (Sadka et al. 2018). The predicted case outcomes are developed using machine learning techniques on 5,000 historical cases. 

The MCLC receives more than 30,000 filings per year from workers who claim to have been involuntarily separated from their jobs and seek severance pay due to them according to Mexican labour law. The experiment and data from the historical case filings provide insight into the underlying causes of inefficiency in the court.

The legal context: Inefficient courts and uniformed, and over-confident plaintiffs

We first use the historical data and surveys of parties to cases to document some stylised facts about court processes and outcomes. We find that case durations are very long, with 30% of cases filed in 2011 still ongoing four years later. Settlement rates are around one-third lower than in labour cases filed in other OECD countries, and plaintiffs recover only 20% of the amount they claim. Recovery rates are particularly low when the cases proceed to judgment, at least in part because workers are unable to collect in around half of the cases where the judge rules in their favour. 

Finally, plaintiffs are both uninformed about the law and the contents of their case and are over-optimistic regarding case outcomes. Indeed, when we ask both the plaintiff and defendant the probability they will prevail in the case, they sum up to almost 150%, indicating excessive optimism more generally. Plaintiffs with private lawyers are particularly misinformed and recover a smaller share of their initial claim. These stylised facts show an environment in which workers are uninformed and parties to the case are overconfident on average. 

The experiment: Personalised statistics and settlement rates

Our experiment is designed to address a fundamental question: Does the provision of personalised statistical predictions increase settlement rates? Theory has shown that asymmetric information between two bargaining parties can generate rational delay as a screening or signalling device, and that overconfident expectations can lead to delay even without asymmetric information. Our interventions aim to increase information and reduce overconfidence, so that we can observe the resulting effect on settlement.

1. Calculator treatment: Our main treatment consisted of providing statistical information customised to the individual case and based on outcomes in the 5,000 historical case filings. Subjects in this arm were presented with a single sheet of paper that was given to all of the parties present at the hearing. Trained enumerators then spent about five minutes explaining to the parties the meaning of the numbers. 
2. Conciliator treatment: In the initial phase of the experiment, the design also included a second arm. The court provided free conciliators, who are court employees, with the objective of facilitating settlement between the parties. Parties in the conciliator arm were asked to sit with the conciliator before that day’s hearing began. 

The experiment was implemented in two phases: 

1. The first intervened in ongoing cases assigned to sub-court 7, one of the 20 sub-courts at the MCLC, that had a hearing between 2 March 2016 and 27 May 2016. 
2. The second phase scaled up the experiment in four additional sub-courts, involving a sample of the first hearing, for cases that were held between October 2016 and March 2017.

The results: Higher settlement rates when the plaintiff is present

Both experimental arms produced significantly higher settlement rates. However, the increase in settlement rates occurred only when the plaintiff was present at the hearing. We use administrative records of the court to show that both of these patterns persisted for 24 to 30 months after the treatment. That is, the treatment was effective at increasing settlement rates, even over the longer term, but only when the plaintiff was present to receive the treatment directly. There is no evidence that the lawyer later shared the information on predicted outcomes with the plaintiff.  

The treatment effects were large: the calculator and conciliator treatments both increased same-day settlement by five percentage points, to 11% in phase 1, nearly doubling the settlement rate as compared to the control group which received no treatment. However, we find the calculator treatment had a fairly precisely estimated zero effect for cases where only the lawyer representing the employee attended the hearing, while settlement rates were 16 percentage points higher when the employee was present. 

Although the presence of the employee is likely endogenous to the prospects for settlement, this result survives when we instrument employee presence using the randomly allocated hearing time. The effect of the treatment when the employee is present increases settlement rates by enough to significantly close the gap with those of developed countries referenced above.

Implications: Legal representation, motivations, and settlement value

Collectively, these results suggest that the lawyers do not share the calculator information with clients, and hence, that agency issues may be important in this context. We provide one further piece of evidence on this by separating plaintiffs according to whether they are represented by public or private lawyers. We would expect agency to be particularly important where plaintiffs use private lawyers. 

Supporting this assumption, we document that employees using private lawyers are significantly less informed about the contents of their case than are plaintiffs using public lawyers, and that the calculator-employee present interaction is driven entirely by the sample of cases with private lawyers.

If private lawyers receive a share of the award, why does the agency issue remain? Using a simple theoretical structure, we show that differences in attitudes toward either risk or discount rates drives a wedge between the preferences of lawyers and their plaintiffs. Because lawyers have many simultaneous cases, their risk is diversified, while their client’s risk is not. Lawyers may be more willing to go to trial in search of a big win at some future time than the clients they represent, providing motivation for the agency concerns. 

The model suggests that the increase in settlement rates could make plaintiffs better off, but is this so? We match historical cases that ended in a judges’ decision in a trial, with cases that ended in a settlement in our experiment. Under an assumption of selection on observables, we find that on average employees receive higher present-value amounts in settlement than they would have recovered in trial. 

Conclusion: Information and efficiency 

The increased settlement rate generated by both treatments helps the court meet its goals of reducing the case backlog.  But the treatments also appear to leave plaintiffs better off. Moreover, the experiment shows the importance of considering agency issues between lawyers and plaintiffs. Almost all of the plaintiffs are first-time users of the courts, and they often find lawyers in a haphazard manner. 

Our results also point to opportunities for future research. In particular, the results suggest the need to understand more about how plaintiffs choose their lawyer, and to explore ways to create and disseminate information on the quality of lawyers, allowing higher-quality lawyers to develop a reputation.

Policy recommendations

Both the analysis of historical cases and the experiments have policy implications, and both were instrumental in discussions that produced a broad labour reform passed in Mexico in May 2019. Evidence of the importance of an inability to collect helped support a strengthening of the law to allow use of information from the National Banking Commission and the Social Security Administration. Supported by our findings, the law also requires that parties must meet with a conciliator prior to filing a case. 

Our evidence also suggests that providing case-specific information to parties is beneficial, at a minimum by making the conciliation process more efficient. The fact that cases in most of Mexico’s labour courts are not currently digitised makes this difficult to implement more broadly. However, as cases become fully digitised, provision of information to all cases will become more feasible. 

References

Duflo, E, M Greenstone, R Pande and N Ryan (2013), “Truth-Telling by Third-Party Auditors: Evidence from a Randomized Field Experiment in India,” Quarterly Journal of Economics 128 (4): 1499-1545. 

Khan, A, A Khwaja, and B Olken (2016), “Tax Farming Redux:
Experimental Evidence on Performance Pay for Tax Collectors,” The Quarterly Journal of Economics 131(4): 219–271.

Sadka, J, E Seira and C Woodruff (2018), “Information and Bargaining through Agents: Experimental Evidence from Mexico's Labor Courts”, NBER Working Paper No. 25137.