Finance: Research, Policy and Anecdotes

When a data collection team is in the news, it is typically not good news and that is certainly the case of the Doing Business team at the World Bank. The Doing Business report has been in unfavourable news coverage quite a lot in the past years and I have written extensively about it (here and here). Two years ago , Paul Romer had to resign as Chief Economist as he (wrongly) accused the team of changing Chile’s numbers for political reasons; ultimately, it turned out to be methodological changes unrelated to political changes in Chile.  The latest alleged infringements, however, are much more serious – data were “adjusted” in return for paid advisory work in the respective country.  If true, this is corruption.  The publication of the latest report has been delayed while an investigation is taking place. Beyond this specific incident, however, there are broader governance and policy challenges.

First, the governance challenge: If one looks at the shake-up in the auditing profession a few years ago, one can see clear parallels – auditing a firm’s financial statements while at the same time providing consultancy services to this firm results in a conflict of interest. However, it took a scandal for reforms to be undertaken.  Similarly, if the same institution ranks countries and provides paid advisory services (that can help improve this ranking), this clearly creates a conflict of interest. One way out of this is to create Chinese walls within the institution between different groups; I am not privy to the institutional details of the Doing Business unit in the World Bank, but given that with few exceptions (such as the Independent Evaluation Group) all units are part of the same hierarchical structure (and under the same Executive Board where shareholders, i.e., countries, have representatives) it seems unlikely that such a structure would really work. So, unless data collection and ranking are either strictly separated with Chinese walls from the rest of the Bank or are outsourced, this governance problem is hard to solve.

Second, the policy challenge: This latest scandal brings to light broader concerns on the Doing Business database. One is the philosophical tendency towards ‘less regulation is always better” (which the Doing Business founder Simeon Djankov defended – loosely speaking – by classifying critics as Marxists). Two is the fact that rankings are taken at face value, although each country’s ranking depends on reforms (or reversal thereof) in other countries.  Three, there is the Goodhart critique: when a variable becomes a policy target, it is no longer a good gauge of what it is trying to measure. When reforms are undertaken just to improve rankings (as shown in this interesting piece on India, but certainly the case in other countries), they might not be the most important ones.  I have therefore argued previously, that the ranking should be taken out and the focus should be on the underlying data. However, even now the first graph one sees when checking on a specific country on the doingbusiness.org website are rankings. One step further than Goodhart’s law is Campbell’s law (per Justin Sandefur): “The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor."

Beyond Doing Business, this controversy has implications for other databases. In the mid- to late 2000s, when I was part of a group of World Bank researchers starting data collection on financial inclusion, we discussed whether we should aim for a country ranking. Back then, I argued strongly against it, for several reasons (some of the following is discussed in more detail in this paper). One, policy makers should care about efficient and sustainable financial inclusion, not just a number. Two, the Goodhart critique also applies to the Global Findex headline indicator of the share of adults with a financial account. It is relatively “easy” to open an account for everyone in an economy through a state-owned bank; this does not imply that these are accounts are being used and that account holders are thus financially included in a meaningful way.  While the Global Findex has wisely decided not to rank countries, its headline indicator is often seen as critical policy target in many developing countries.  The underlying report, however, provides a balanced discussion, drawing on many different indicators collected. Further, the Global Findex has developed over its three rounds, with the focus expanding from bank accounts to mobile money accounts and from account ownership to use.

In summary, data are critically important – if you want to reform something, you have to measure it first. It is also clear that data collection does not and cannot take place in a completely agnostic and a-priori neutral way. We collect data on the business environment because we think it is important; we collect data on financial inclusion because we think it is important.  The World Bank is in a unique position to host such data collection efforts and I would argue that one of its main contribution to development (as part of its broader development research effort) has been exactly this. However, the World Bank has also an important responsibility in the use of these data; the use by others of World Bank data is obviously outside its control, but the way data are being presented and being used in World Bank reports sends an important signal to policy makers across the globe and other stakeholders.

Overview