Evaluating Terrorism Prevention (and Public Diplomacy) ProgrammesFebruary 13, 2013
The Center on Global Counterterrorism Cooperation has just put out a paper recording the discussions at a seminar they ran on Evaluating Terrorism Prevention Programs. It’s worth a look in part because of the discussion of some of the practical problems of doing evaluations that apply to public diplomacy as well as their area of interest.
It strikes me that the issue that is only partially addressed is that many programmes (I’m British so I need two mm’s) aren’t actually designed with evaluation in mind. When faced with a challenge an organization needs to do something and reaches into its repertoire and puts some ‘programming’ together. Alternatively, if it’s not faced with a new challenge organizations just carry on doing what they always done. The result is that the programme is evaluated it’s against its stated objectives (which may or may not have been feasible in the first place) or a broader set of policy objectives (look at the Dutch example in the paper ) not on whether its actually producing the intended real world results.
At least one of the presentations points to the significance of ‘theories of change’ in programme design. I’ve blogged about this before. This is the movement, particularly in the development community, to get away from formalistic planning models and to engage with what’s happening in the ‘real world’. Your theory of change explains how your intervention is going to produce the desired effect. If you can do this you are in a better position to work in appropriate evaluation metrics. You may also do some useful learning in that you discover that your intervention had the desired result for the reason you’d expected, that your causal mechanism operated but didn’t produce the desired results or that you got your results but not through the change mechanism that you’d expected.
The question of theories of change is where academics can potentially make a bigger contribution to Public Diplomacy practice by helping to get a better model of how public diplomacy can and cannot produce the desired effects.