The importance of trust in health systems during crises: Evidence from Sierra Leone

Across many low- and middle-income countries, millions die from treatable conditions. To a large extent, this is driven by the low quality of health systems (Das et al. 2016).  Faced with low quality of care,  patients lack confidence in health providers and tend to under-utilise health facilities (Kurk et al. 2018, Dupas 2011). This can have severe consequences in the context of health crises such as Ebola or COVID-19, when complying with response efforts through early testing and treatment helps contain these infectious diseases. 

Improving the performance of health workers and building trust in health facilities is therefore a critical policy priority. One promising approach allows communities to hold service providers accountable through public discussions of their performance. Recent programs have focused on community monitoring, where health providers and community members are brought together to jointly address obstacles to adequate healthcare provision. Early work in Uganda shows significant and substantial improvements in utilisation and health outcomes (Bjorkman and Svensson 2009, see also Raffler et al. 2022). Other approaches leverage competition between service providers (Besley and Ghatak 2005), by giving status awards to high performers as a form of motivation. Related work by Ashraf and colleagues (2014) shows that social incentives (such as awards) can boost worker performance.

In our study, we evaluate how these types of accountability interventions affect health care utilisation and health outcomes in Sierra Leone – first, during “normal” conditions; and second, during an acute health crisis: the 2014–2015 Ebola outbreak (Christensen et al. 2021).

The approach: Accountability interventions in Sierra Leone

Two years prior to Sierra Leone’s Ebola outbreak, we teamed up with the Government of Sierra Leone, three international NGOs, and the World Bank to design and implement two accountability interventions – community monitoring and non-financial awards – that aimed to increase health care utilisation and health outcomes in 254 primary health care clinics across the country.

We randomly assigned the clinics to one of these two interventions or a control group in equal proportions. The interventions included the following components:

• Community monitoring: facilitators introduced health scorecards with information regarding the state of health care in each clinic, and organised three sets of meetings over nine months with community members and health facility staff. At these meetings, community members voiced concerns about health outcomes or services and clinic staff made commitments to improve services. For example, staff made commitments to reduce absenteeism but also pressed women in the community to utilise clinics more to give birth and seek pre- and post-natal care.
• Non-financial awards: we facilitated competition among clinics, and gave (non-financial) status awards to the highest performing and most improved clinics in each district. Staff at winning clinics received letters of commendation from politicians and an award at a public ceremony.

We conducted a follow-up survey to measure medium-term impacts approximately a year after these interventions were implemented. We surveyed each clinic and households in two communities nearby, collecting data from over 5,000 households.

A year after our surveys, the Ebola epidemic struck Sierra Leone.  We utilise data from the epidemic period to examine impacts on Ebola-related outcomes. While unplanned, this represented a rare opportunity to examine outcomes in the context of a major health crisis. We spatially merged information on the location of health clinics in our study with Ebola case records from the Viral Hemorrhagic Fever (VHF) database, which was maintained by the Government of Sierra Leone with support from the US Centers for Disease Control and Prevention. We used this data to analyse Ebola records: we examined counts of both confirmed Ebola cases and patients who test negative for the virus, alongside deaths among these patients.

It is important to note that our measure of Ebola cases are based on records of patients, and this data has limitations since some infections were never reported. However, no data exist that provide a measure of overall incidence independent of reporting and the VHF is the most comprehensive source tracking data from the crisis. Moreover, reported Ebola cases are a highly informative metric, as they reflect the extent to which individuals are willing to come forward and be tested and treated, which requires trust and confidence in health facilities and workers. If the accountability interventions increased utilisation and trust in clinics, this would imply that potential Ebola patients were more willing to come to clinics during the crisis. Since this is the precise margin our interventions were designed to influence, examining Ebola cases gives us a unique means of examining how accountability interventions can influence trust in health systems.

The findings: Improvements in health outcomes and Ebola reporting

Medium run effects under normal conditions: Improvements in clinic utilisation, health and satisfaction.

In the medium run, both interventions directly affected a number of health-related outcomes. Figure 1 summarises the impacts on one key metric: clinic utilisation. (Outcomes here are expressed in Standard Deviation (SD) changes relative to the control group, which we centred at mean zero and SD one.)

We see a sizable increase in general utilisation of about 0.1 SDs.  In terms of actual visits, this effect implies that respondents in treatment areas were about 5 percent more likely to have sought care in a government-run clinic.

The Community Monitoring intervention also increased maternal utilisation (by 0.13 SDs, driven by more deliveries at Western-style clinics) and improved child health outcomes (by 0.17 SDs). Notably, this intervention reduced the likelihood of a household experiencing the death of a child under age five by 38 percent. These substantial effects are qualitatively similar in size to the findings of Björkman and Svensson (2009).

In addition, we find that patient satisfaction with clinic staff increased (by about 0.1 SDs), as did trust in Western medicine relative to traditional and spiritual healers, suggesting that the interventions improved how patients perceived the quality of health care provided in treated areas. 

Figure 1: General utilisation by treatment

Notes: Figure 1 plots the treatment effects of Community Monitoring and Non-Financial Awards on an index of general utilisation in standard deviation units. We also report outcomes on 9 other pre-specified outcomes, see Christensen et al (2021).

Long run effects during crisis: Increases in Ebola reporting

When we analyse the longer-run outcomes related to the Ebola epidemic, we find that both interventions increased reporting of Ebola cases. In control areas (administrative units called Sections) there were typically 0.28 cases are reported per week; in Community Monitoring and Non-Financial Awards areas, this increases to 0.49 and 0.43 respectively (see Figure 2). Pooling the treatments, average reported cases were approximately 60 percent higher compared to control sites.

Figure 2: Total reported Ebola cases by treatment

Notes: Figure 2 plots the total cases reported per section per week. The dot represents the raw count, and the error bars represent 95% confidence intervals. Treatment effects are 0.173 for Community Monitoring and 0.204 for Non-Financial Awards. A section is a small administrative unit that is typically about 40 square kilometres in size with under 2,500 residents. This analysis was restricted to the 160 sections with a single health clinic.

These results indicate that individuals were more likely to come forward and report to clinics if they suspected they had Ebola symptoms in locations where the accountability interventions had increased confidence and trust in health workers and facilities. What is more, effects of these magnitudes could have meaningfully limited the spread of Ebola: a simple back of the envelope calculation based on the epidemiology literature (Pronyk et al. 2016) suggests that the increase in reporting and testing in treatment areas helped slow the reproduction rate of the virus by about 19 percent. 

We also find that reported deaths among patients declined, particularly in areas where Community Monitoring had been implemented. In treatment areas there is about one death per seven cases, compared to one in four in control. Two caveats are warranted: first, we do not observe overall deaths (as this data was not available), but instead examine deaths among reported cases; and second, this analysis makes stronger assumptions, as it uses Ebola cases (a post-treatment outcome) as a moderator, imposing a specific functional form assumption by linearly interacting Ebola cases with the treatment.  

Overall, these findings suggest that getting the community directly involved, particularly through the Community Monitoring intervention, may not only improve reporting, but also improve patient outcomes amid disease outbreaks.

Policy implications

We find that accountability interventions can improve perceptions of health facilities and workers and health outcomes. Under normal conditions, these interventions engender trust and lead to greater use of clinics, ultimately reducing mortality rates of children under five in clinics with community monitoring. During health crises, such as the Ebola outbreak, these improvements in trust encourage individuals to come forward and seek medical care, which could help to contain the epidemic. 

Thus, not only can accountability programs produce gains over the medium-run under “normal” conditions, they may also produce substantial gains during crisis conditions, when trust in the health system becomes critical, as individuals face decisions about whether to cooperate with response efforts, such as complying with quarantines or seeking medical testing. These results are directly relevant for areas with frequent outbreaks of Ebola across many parts of Africa, including the DRC, and most recently, Uganda. They also naturally extend to other efforts to contain other pandemics and epidemics, including COVID-19.

References

Ashraf, N, O Bandiera, and B K Jack (2014), “No margin, no mission? A field experiment on incentives for public service delivery”, Journal of Public Economics 120: 1–17.

Björkman, M and J Svensson (2009), “Power to the People: Evidence from a Randomized Field Experiment on Community-Based Monitoring in Uganda”, Quarterly Journal of Economics 124(2): 735–769.

Christensen, D, O Dube, J Haushofer, B Siddiqi and M Voors (2021), "Building resilient health systems: Experimental evidence from Sierra Leone and the 2014 Ebola outbreak.", Quarterly Journal of Economics 136(2): 1145-1198.

Das, J, A Holla, A Mohpal and K Muralidharan (2016), “Quality and accountability in health care delivery: Audit-study evidence from primary care in India”, American Economic Review 106(12): 3765–99.

Dupas, P (2011), “Health behavior in developing countries”, Annual Review of Economics 3(1): 425–449.

Kruk, M E, A D Gage, C Arsenault, K Jordan, H H Leslie, S Roder-DeWan, O Adeyi, P Barker, B Daelmans, S V Doubova and M English (2018), “High-quality health systems in the sustainable development goals era: Time for a revolution”, The Lancet Global Health 6(11): e1196–e1252.

Pronyk, P, B Rogers, S Lee, A Bhatnagar, Y Wolman, R Monasch, D Hipgrave, P Salama, A Kucharski, M Chopra, and UNICEF Sierra Leone Ebola Response Team (2016), “The Effect of Community-Based Prevention and Care on Ebola Transmission in Sierra Leone”, American Journal of Public Health 106(4): 727–732.

Raffler, P, D N Posner and D Parkerson (2022), “Can Citizen Pressure be Induced to Improve Public Service Provision?”, Working Paper.