Stephen Turner in Tanzania

Stephen Turner is a Mokoro Principal Consultant. This year he worked as co-team leader (with a Senior Evaluation Adviser in the United Nations World Food Programme’s Office of Evaluation) of a strategic evaluation of WFP’s pilot Country Strategic Plans. CSPs are part of a major transformation of the way WFP plans and structures its services.

In the course of this year, my colleagues and I often paused to ask ourselves what ‘strategic’ really means, and what a ‘strategic evaluation’ is. At one level the answers seem simple. At another, they have sometimes evaded us.

Quite some years ago now, I indulged in a good-natured rant to a fellow, highly competent evaluator whose prose, despite this competence, had begun to include the word ‘strategic’ in almost every sentence. “What is ‘strategic’ supposed to mean?”, went the rant. “You seem to be using it as a general mark of approbation: if it’s right, good, helping towards the desired results, it’s strategic?” In other words, the term was losing most of whatever flavour it was supposed to have.

If you look them up, you’ll find that ‘strategy’ and ‘strategic’ originally had military meaning, referring to “the art of planning and directing overall military operations and movements in a war or battle”. That’s how one dictionary puts it, although it also refers to the newer definition of ‘strategy’ as “a plan of action designed to achieve a long-term or overall aim”. Probably, gentle reader, you are not directly concerned with military planning. Nor am I. That leaves us to focus on the woolly notion of ‘strategy’ and ‘strategic’ as concerned with the higher-level, longer-term way in which things are planned, structured and organised. More specific initiatives and actions like programmes and projects – themselves often the subject of evaluations – are subordinate to, presumably guided by, strategy. But what about policy? Does policy drive strategy, or is it the other way round? In practice either may happen, and it is questionable how clearly some organisations and analysts differentiate the two concepts.

WFP does have a separate category of strategic evaluation. According to Office of Evaluation (OEV) guidance, Strategic Evaluations were introduced in 2008/9 to focus on strategic and systemic issues of broad corporate relevance. ‘Strategic Evaluation’ is simply a label given to a type of evaluation. Policy evaluations focus on a single policy; country portfolio evaluations also do what the name suggests, as do operations evaluations and corporate emergency evaluations. Each of these has a specific scope set out in the WFP evaluation policy. Strategic evaluations are woollier, as the scope can cover a wide range of issues: it is a catch-all category. To calm ranting consultants, WFP could develop clearer sub-categories or tighten the definition, but this would reduce flexibility and there would always be subjects that didn’t fit into the narrower categories. Again, it’s just a label. From an evaluation perspective, the evaluation team needs to do what is asked in the terms of reference and according to the guidance, regardless of possible lack of clarity about what the category means.

There are, arguably, ways to tighten up the adjectival use of ‘strategic’. Reflecting its military roots, perhaps, the word is generally understood to refer to the optimal deployment and disposition of one’s resources for a clearly defined purpose, taking careful account of the circumstances – in particular, the opportunities and risks with which one must engage in working towards the objective. In other words, smart planning and action, making the best of the situation. But all that implies detailed, thorough analysis and planning, does it not? – while ‘strategy’ is mostly seen as big-picture thinking. We are often reminded of the distinction between strategy and tactics.

Confused? I have been, sometimes. But not that often, because mostly we have been engrossed in this global, comprehensive evaluation of how WFP is reorganising itself around multi-annual Country Strategic Plans – a significant development from the mostly short-term funding and planning that have dominated its work to date. We have also been getting used to the idea of having two team leaders for an evaluation – something new for WFP’s Office of Evaluation, at least in recent years. For some international evaluation agencies, it is normal practice for a staff member of the organisation to serve as lead evaluator, assisted by consultants. WFP has usually appointed an OEV staff member as evaluation manager, and hired a team of consultants – one of whom serves as team leader.

My co-team leader was also the evaluation manager, who in the normal model would undertake quality assurance of the evaluation team’s outputs. But how would this work when he would be assuring the quality of his own joint efforts? This was one of a number of practical challenges that OEV and/or the evaluation team had to overcome.

Initial concerns were also raised over the independence of the exercise, but the evaluation team was reminded that the independence comes from the status of the OEV, not just the independence of the evaluation team. The WFP evaluation policy states that the WFP Director of Evaluation heads an independent evaluation function within the WFP Secretariat and has full discretion over evaluation selection, approval and issuance of evaluation reports to the Executive Board. The evaluation budget is approved by the Board in the context of WFP’s Management Plan, and the Director of Evaluation has full discretion and control over the allocation of resources. The WFP co-team leader and evaluation manager was backed up by oversight from colleagues in OEV.

Assured of our independence and mandate, we co-leaders and our colleagues on the Mokoro team were able to work smoothly through this complex, wide-ranging, high-level task, whose findings and recommendations were probably of more than usual interest to the WFP Executive Board and management. Fortunately we two leaders of this strategic evaluation were able to get on well together and suffer the inevitable stresses and strains of such a major exercise with reasonable good humour. So I can confidently conclude that we were double strategic. I thank my co-TL and all the team for a strong and successful experience together.

Download this article.