Finance: Research, Policy and Anecdotes

The World Bank’s decision to permanently suspend the Doing Business project is the sad end to an initially very good idea! My latest Vox column discusses how we got from a research-based data collection effort as an important impetus into policy debate to a ranking exercise that undermined the usefulness of these data. A short summary:

20 years ago, the Doing Business project filled an important gap by providing detailed indices on specific institutions (e.g. credit registries) or specific business cases (enforcement of bounced check, registering a property of specific value). Over time, Doing Business has become by far the most successful data collection exercise and report of the World Bank, but also, as former World Bank chief economist Kaushik Basu (2018) stated, the most contentious publication. Controversies over Doing Business resulted in the resignation of Kaushik’s successor Paul Romer and have resulted in numerous inquiries and reviews, ultimately resulting in the pausing of the report in August 2020 and its permanent suspension in September 2021.

In order to maximise the impact of the data collection, the Doing Business report included rankings of countries based on Doing Business data. The report and ranking became part of a broader effort of the World Bank and other donors to assess countries’ investment climate and foster private sector development and thus growth in developing countries.  Countries that moved up the most in the ranking were crowned reformer of the year, which in turn has been used by governments to attract foreign direct investment. However, governments that care about these rankings, might start gaming the system – rewriting laws with an eye on improvements in the Doing Business ranking rather than focusing on the most important constraints for private sector development.

Credit rating agencies face a conflict of interest if they rate securities while providing advisory services to the issuing companies. Similarly, the Doing Business team faces a conflict of interest if it collects data used for country ranking while at the same time offering advisory services to governments on how to improve their business environment and thus their Doing Business Ranking.  This conflict of interest is not theoretical, but has been clearly shown in the Wilmer Hale (2021) report .

However, the conflict of interest is even deeper and cannot necessarily be remedied by introducing a firewall between data collection efforts and advisory services. Ultimately, the World Bank is owned by its member countries who are represented on the Executive Board.  The (ultimately successful) attempt by certain governments to take influence on the ranking (and thus the underlying data) clearly speaks to this conflict of interest.

Beyond these conflicts of interest, there is a broader concern of whether the political targeting of the Doing Business ultimately undermines the usefulness of the index.  Charles Goodhart stated in 1975 that “any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.”  Originally, this regularity (known nowadays as Goodhart’s Law) was used in monetary policy, but it can be applied in any economic policy area. Marilyn Strathern (1997) summaries Goodhart’s Law in a discussion on the UK evaluation framework for universities:  “When a measure becomes a target, it ceases to be a good measure”.

The events described in the World Bank report read like Goodhart’s Law in action.  As rankings were published on an annual basis and as improvements in the ranking became political targets, so did attempts at influencing ranking and thus data. The conflict of interests mentioned above facilitated such influence-taking; all that was needed were persons in place that would be open to such influences.  Beyond these conflicts of interests, however, it is hard to see how one can get around this more fundamental problem of declaring an index as policy target, without this index subsequently losing its usefulness. 

Where do we go from here?  The original idea of collecting data on the business environment in which companies across different countries (or even across different regions within countries) operate continues to be good and important – again for research purposes and to inform the policy dialogue (though not to pre-empt policy outcomes). In a previous VoxEU column in 2013, I called for moving away from rankings and to focus exclusively on data. Researchers do not need rankings, we need data.  Policy makers need data and comparisons, while questionable rankings do not only not help a constructive reform process, but narrows it down to what are not necessarily the best reforms. Finally, Doing Business data should be regarded as one of several gauges of the business environment firms face, forming part of a more broadly-based approach that draws on multiple data sources and analyses. Such an approach might not hit the headlines as often, but might be more effective in driving policy reforms.

Overview