ODI Cape Conference 2013

Budgeting in the Real World: from positive deviants to models for success

Stephanie Allan

The national budget lies at the intersection of all important decisions to be made in the public sector. By allocating resources as a means to realise policy goals, the budget is intrinsic to the policy-making process itself. Getting it right, then, deserves a concentrated, multi-shareholder effort, and in this vein the 2013 ODI-CAPE conference brought together over 100 individuals, including Ministry of Finance officials, PFM practitioners, development partners and leading researchers, to grapple with the grandiose theme of “budgeting in the real world”.

With such a wealth of expertise and experience converging in one room in South East London, you might expect that there would emerge a rich account of the successes of PFM reform and a fine-tuning of methodologies. You’d be sadly disappointed. To the credit of CAPE and the PFM sector as a whole, the sobering lack of evidence to assuredly link PFM reform with improvements in development outcomes is openly acknowledged and well documented. But in an age of value-for-money and against a backdrop of rapid growth in aid to institutional reform over the last decade1 this should give us pause for thought; as a non-PFM specialist present at the conference noted with a degree of concern: “if you guys can’t sell PFM to each other, how do you expect to convince anyone else?”

The problem was unpacked over the course of two days, through rich presentations and vibrant discussion. Firstly, PFM reform has tended to focus on the wrong kind of reforms, ones which we know don’t really work, so why then should we be surprised when positive outcomes and impacts are not registered? Secondly, the relationship between PFM reform and development outcomes is an inherently difficult thing to measure, so even when we get it right, we’re not systematically demonstrating it.

The resounding consensus from the venerable line-up of panelists was that the report card for institutional reform doesn’t read very well. The spate of programmes across the developing world has, at best, a mixed record of success, with an intervention failure rate of between 40% and 60%2. It appears that the problem lies in the tendency for external actors to impose off-the-shelf blue-print reform methodologies – deemed “best practice” internationally – with little concern for the political context, Government ownership or national capacities.

The collective experience of participants certainly confirmed this. The Collaborative Africa Budget Reform Initiative (CABRI) shared the findings of its recent study of 27 African countries with Medium-Term Expenditure Frameworks (MTEFs, an advanced tool which promises to deliver a comprehensive Government-wide spending plan that links policy priorities to expenditure allocations within a fiscal framework with forecasts over a three-year time horizon). The study found that the introduction of MTEFs had little functional impact on the operations of finance ministries; only three countries actually used their MTEFs for any practical purpose in the budget process, everywhere else they served presentational purposes only.

MTEFs, Performance-Based Budgeting and the slew of “best practice” reforms aren’t delivering as promised, in part because PFM is approached as a technocratic concern, somehow separated from the messy world of politics. However, where reforms come into conflict with the incentives of the political elite, failure is almost guaranteed. In her keynote address, Antoinette Sayeh, Director of the Africa Department of the IMF, lamented the “implementation gap” in PFM reform, which is most clearly demonstrated in substantial deviations that often occur between planned budgets and those which are actually executed. She argued that this was evidence of potential for the political environment to derail ill-considered reform, and that formal successes may mask functional failures (after all, preparing a reasonable budget could be considered less of a threat to political leadership where execution offers ample loopholes). An understanding of political economy must therefore be integrated into PFM reform, which requires sustained presence on the ground, and we should be realistic about the pace of reform by not expecting or searching out signs of success too early.

Consensus formed around a second major failing in PFM, which is that it tends to focus on systems with little regard to the people that are meant to run them. The CABRI study found that the culture of the civil service was a very important determinant in the success of PFM reform, with reforms more likely to become embedded in civil services where decisions are made by consensus rather than directive, where effort is recognized and rewarded, and experimentation is encouraged. Similarly, capabilities of a finance ministry in negotiation and coordination are very important in budget processes, yet there is a systematic under-investment in these “soft skills” from the donor community. The former Prime Minister and Minister of Finance of Mozambique, Luisa Diogo, recognised this, and spoke passionately of her emphasis on soft skills and mentoring talent as Finance Minister through a rising stars programme, whereby senior officials were formally required to have shadowing arrangements with talented young employees.

The indices we have to measure PFM performance mirror and reinforce this bias towards blue-print approaches. Many of these generic reform elements – including MTEFs – have been formalized into indices such as the Public Expenditure and Financial Accountability (PEFA) Performance Measurement Framework. Developed by a multi-donor group, the PEFA framework standardizes an approach to thinking about PFM systems that is presented as a tool that countries can use to benchmark their system against “existing good international practices” (PEFA, 2006; Andrews, 2009). These reforms also have the benefit of being easy to assess from an external viewpoint (which is always useful for an index), unlike some measures of the more nuanced, behavioural changes to the ways that Ministries of Finance actually function. Efforts to improve against these indices therefore encourage the continued use of these ill-fitting approaches, a bias which is often reinforced by partner countries themselves when chasing flexible donor support (such as budget support), which is often contingent on an improved PEFA score. Addressing this will require a consolidated effort on the part of the donor community to improve the way indices are used and what they measure. We have a long way to go – the World Bank’s admission that development policy operation indices are 9% behavioural and 91% non-behavioural is testament to this – but better indices are within our reach. We should be systematically measuring a Government’s functional PFM performance, including the deviation between planned and executed budgets, timeliness of public sector salary payments, and lags in issuing (legitimate) payments to contractors.

If there was a danger of general disillusionment amongst the conference’s participants, CAPE mitigated this by providing ample opportunity to hear from the “positive deviants”, that is, the countries, TA providers and academics involved in the PFM reforms which have bucked the trend in registering successes. ODI’s Budget Strengthening Initiative served to highlight the importance of approaching PFM reform as a non-linear process, where addressing one problem will likely uncover others. In South Sudan, the search to find a solution to donors’ across-the-board refusal to channel any aid through Government systems led to a reform path that included country-wide overhaul of sub-national PFM and HR processes, as well as a restructuring of the Government’s own inter-governmental fiscal transfers system. This message was echoed by Harvard Associate Professor Matt Andrews, who shared the preliminary findings of his ongoing study looking at a sample of 30 case studies of successful PFM reform. It indicates that good reform tends to be sparked by a locally-defined “crisis”, that is, a fairly narrow problem (e.g. salaries are not paid, or text books not delivered), which in turn delivers a broad change that goes beyond the immediate problem to address a systemic issue. The solution is often a hybrid blend of internal and external ideas which is both politically acceptable and practically possible (Andrews, 2013). This reform approach (which he coined as “Problem-Driven Iterative Adaptation”, or PDIA) is adopted during early stages of successful institutional reform. However, as the reform is rolled out, it may be locked in through more of a blue-print approach, with linear management techniques.

The remaining question, therefore, is how do we turn the positive deviants into models for success? Is there anything which can be standardised into a common approach, whilst being wary of the danger of replacing one orthodoxy with another? The World Bank candidly noted internal incentives as an obstacle to PDIA-type flexibility. In an organisation where there is a strong emphasis on getting board approval for a project, managers may be hesitant to change the project direction if that is likely to require resubmission to the board. An alternative would be to empower the Government to be able to select the best fit PFM reform. This would assume a menu of options and competition between providers, with detached funding arrangements, perhaps through a multi-donor trust fund or budget support.

There was not sufficient time for detailed discussion on what the architectural arrangements for PFM support should look like and how we can build these lessons more formally into PFM programmes, which was unfortunate as this was where it began to feel as if we were entering more uncharted territory. However, as a means of peer-learning across the whole spectrum of finance officials to practitioners to academics, it certainly succeeded in clarifying where we’ve got it wrong with PFM, and why. Positive models may emerge more clearly in time, as we build up more extensive case studies of successes, and perhaps ODI will consider this as a follow-up theme for the 2014 CAPE conference.

Download this article

You must be logged in to post a comment.