Learning management through matching: Experimental evidence from Ethiopia

To become a successful manager or entrepreneur, individuals must gain experience in many practical tasks. For instance, they must learn how to monitor performance against targets; how to organise relationships with clients, suppliers, and colleagues; and how to deal with regulators and navigate the business environment. These skills are typically learned while working in close proximity with more senior managers. As a result, many high-productivity enterprises are founded by individuals who have worked as an employee in another business.

Management experience comes in many different varieties

The way of learning management by experience is scarcely uniform, not least because management practices of enterprises are highly heterogeneous (Bloom et al. 2014). For example, individuals placed in firms with more structured management practices (i.e. those that emphasise systematic processes, targeting, and monitoring) may learn to place a higher weight on such practices than intuition or informal interpersonal relationships. 

Moreover, the effect of learning might further depend on the relation between firm and participant. For instance, individuals with more ‘business-like’ educational and professional backgrounds might learn more from firms with structured management practices. In turn, these differences in learning from the host firm might translate into differences in career outcomes and earnings of aspiring managers.

A new policy intervention: Management experience placements

In developing countries, the kind of large and efficient organisations that can incubate managerial skills are often few and far between. In a recent study (Abebe et al. 2019), we implement and evaluate an intervention to address the lack of role models that aspiring managers and entrepreneurs can learn from . Our programme places highly educated professionals inside established medium and large firms in Ethiopia, a fast-growing and transforming economy with few established private sector firms and limited placement and matching services. Individuals who participate in the programme spend one month at the firm, shadowing a middle manager in their day-to-day activities. 

Participation in the programme is randomly assigned to young professionals, as well as to firms, which allows us to estimate the average causal effect of being invited to a management experience placement. We also estimate treatment effects by variety of the treatment – in our application, the quality of management practices in the host firm – for different participants. A new method is developed to estimate these treatment effects, hinging on three features of our experimental design: 

1. Participants and host firms were purposefully matched using an established and replicable procedure – in our application, a deferred-acceptance matching algorithm (Gale and Shapley 1962). 
2. For logistical reasons, matching and placements were undertaken in small batches to which participants were invited in random order.
3. Participants and firms were asked to rank each other without prior meeting and we controlled the information that was available to them. 

Together, these features allow the simulation of counterfactual matches i.e. the matches that would have occurred if any given individual had been matched with a different mix of participants. Counterfactual matches are useful for two reasons. First, they allow the estimation of the causal impact of one type of match – or treatment variety – over another. Second, they allow us to calculate the effect of counterfactual policies that provide the same treatment under different assignment mechanisms. This margin of policy design has so far attracted little attention, even though it matters profoundly, as our results show. 

Management experience helps in finding a permanent job

On average, we find that the programme increased participants’ success in finding a permanent, professional wage job. It also increased their work hours and wage earnings, by about 10% relative to the control group. For the average participant, we find no effect on self-employment participation, hours, or profits. 

The evidence further suggests that at least part of the employment effects is driven by participants having learnt something about management, rather than being the result of improved signalling, business networks, or being hired by their host firm. Participants report increased confidence along the dimensions of management that they were mostly assigned to work on in the firm, such as planning and monitoring tasks and low-stakes relationship management. Further, we find suggestive evidence that participants who were self-employed before entering the programme significantly improved the management of their own microenterprise.

Figure 1 Average effects of the programme on employment

Note: This figure shows intent-to-treat estimates of the management experience placement on primary employment outcomes, based on an ANCOVA regression which controls for the outcome at baseline and for randomisation strata. For graphing, coefficient estimates were transformed into effect sizes relative to the control group mean at follow-up. Black whiskers represent 95% confidence intervals, based on standard errors clustered at the individual level.

Differential impacts by host firm management practices

We then apply our method to test the causal impact of being matched to a host firm with good management practices. We find that the type of host firm matters for career outcomes in self-employment. Placement in firms with structured management increases self-employment relative to other firms. There is no such difference for wage employment: any kind of host firm increases later success in wage employment. Moreover, we find evidence of a match-specific effect: individuals with a high likelihood of being matched to a firm with high structured management are particularly likely to start a highly profitable business if matched to such a host (see Figure 2).

Figure 2 A strong effect on business profits for high-propensity individuals assigned to high-management hosts

Note: This figure graphs business profits as functions of the propensity score under the implemented assignment mechanism, by type of host firm. In red are outcomes for participants matched with firms with management practice scores above the median, in blue for those below the median. Shaded areas are 90% confidence intervals. The distribution in grey at the bottom of the graph shows that the distribution of propensity scores is uniform across participants.

Counterfactual policy design

Was the average impact of the programme due to the specific way we assigned participants to firms? Would an alternative assignment mechanism have generated even larger effects? The answer to both questions is yes. In Figure 3, we compare the average increase in total income achieved by the assignment mechanism we actually used – dubbed ‘firm-proposing deferred acceptance’ – to some commonly used mechanisms. We see that the programme would have achieved a much smaller impact with random matching, a natural alternative. But we also see that a random serial dictatorship, a mechanism in which participants take turns in picking their preferred firm, would have resulted in even higher income gains. 

Figure 3 Counterfactual assignment mechanisms differ substantially in the income effects they generate

Note: This figure shows the counterfactual total monthly labour income under alternative assignment mechanisms, minus the control group mean. The actually implemented mechanism was firm-proposing deferred acceptance (DA).  The black whiskers show 90% confidence intervals of the difference in income effects between the mechanism in question and firm-proposing DA.

Policy implications

An immediate policy implication is that high-growth entrepreneurship can be fostered by encouraging business start-ups by individuals with management experience in established firms. This could potentially be achieved by introducing a programme similar to ours, or perhaps by incentivising start-up experimentation by managers employed in existing firms. As our results demonstrate, the details of how matches are assigned matters. Our counterfactual simulations suggest that assignment mechanisms that place a large weight on the preferences of participants can lead to larger welfare gains (Narita 2019). More generally, attention should be paid to how treatment varieties are assigned to participants. The method we developed can be used to study the heterogeneous effects of interventions in education, training, consulting, peer support, and many other applications.

Editors' note: This article is based on this PEDL project.

References

Abebe, G, M Fafchamps, M Koelle and S Quinn (2019), “Learning management through matching: A field experiment using mechanism design”, NBER Working Paper, No. 26035.

Bloom, N, R Lemos, R Sadun, D Scur and J van Reenen (2014), “The new empirical economics of management”, Journal of the European Economic Association 12(4).

Gale, D and L Shapley (1962), “College admissions and the stability of marriage”, The American Mathematical Monthly 69(1): 9-15.

Narita, Y (2018), “Towards an ethical experiment”, Cowles Foundation Discussion Paper No. 2127.