The State of Evaluation

Given the amount of time that people in the PD community spend worrying about evaluation you might be interested in a recent report from the UK National Audit Office on Evaluation in Government…put it this way given the size of PD budgets there are a lot of people with much bigger problems:

The main  findings

1.  Despite polices that require evaluation of the impact of interventions British government actually evaluates in a pretty random way, departments don’t have a clear view of what they evaluate or why they do it.  A graphic casually points to £51 Billion  of defence expenditure that isn’t being evaluated at all (ie roughly 25 times the entire FCO Budget)
2. Most evaluation fails basic standards of methodological adequacy
3. Departments don’t use evaluation evidence in developing policy.
4. Only a small fraction of requests for funding from The Treasury are supported by evidence from evaluations.
5. Evaluation reports that are weaker in supporting the causal impact of interventions  make bigger claims for policy effectiveness.

So the next time someone asks you to justify the impact of public diplomacy expenditure you will be perfectly at liberty to ask them about the evidence that any other government activity actually does anything.  The point is not that government activities don’t do anything (even though this might be the case) but that government isn’t very good at producing good evidence that they do.