Month: March 2011

Measurement and Evaluation in Public Diplomacy

    At the ISA Public Diplomacy Working Group meeting in Montreal one of the topics of discussion was the state of evaluation and measurement of PD efforts.  Neither the academic community nor PD organizations are very happy with the state of evaluation.  This is obviously an important issue  because the ability to demonstrate impact is important for justifying PD budgets.

    The discussion took in evaluation at programme level (for instance surveys of exchange participants ) and national level measures such as Simon Anholt’s Nation Brand Index.  The point was made that commercial indexes such as the Nation Brands index are widely followed in Foreign Ministries – for instance look at the London Olympics Public Diplomacy Plan that makes explicit mention of the index.  At the programme level MFAs are increasing the provision in budgets for PD activities for evaluation of the impact of the activity.

    In listening to the discussion I got the feeling that there is a quite a big gap between these two levels of evaluation – a gap that reflects problems with the way that we understand the impact of public diplomacy activities.  At the macro level changes in national reputation may be unconnected with PD activities – if your country attracts attention because of a disaster, a sporting event, or a policy change.  At the micro level it is relatively easy to survey or interview participants in a programme.  But it is much harder to find the connection between the programme activities and the overall national perception.  Part of the difficulty is that the impact of a PD programme may actually be much more complex and reach far beyond participants who can be easily identified and monitored..  For instance to take the example of the US assistance to Brazilian public education that I blogged about a few weeks ago there are at least four sets of impacts.

  1. Effects on people who directly participate in the programme
  2. The impact of programme participants on their contacts
  3. Impact from media coverage of the programme
  4. Impact on Brazilian governmental actors of the willingness to undertake the programme.

    5. My thought is that part of the solution  to questions of evaluation is actually conceptual;  that is the need to think through what PD programmes are trying to do rather than rely on standardized measurement techniques or as problem of research method.

    Before the PD community rushes off to invent new measurement strategies it’s worth looking at how other people handle these challenges.  I’ve recently become aware of the volume of work done in the International Development area  on these topics.  A big part of current development work is influencing policy and faces similar problems to PD activity.  Because so much of development work is done on a project basis with other people’s money development organizations are under pressure to demonstrate that they are getting good value.  Also there is a lot more money in development work than PD so they’ve had the resources to think quite extensively about these issues (not to say that they’ve actually solved them).

    The basic approach is to design evaluation activities alongside the development of the programme. A programme should lay out a ‘theory of change’ that is the assumptions about how the intervention will have its effect.   This theory is often expressed in the form of a ‘logic model’  a flow diagram that lays out inputs>activities>outcomes>impacts .  Explicitly laying out the theory of change does two things.  Firstly, it allows the planners to make assumptions explicit and to evaluate their plausibility.  Secondly, this process allows the identification of possible measurement and evaluation points – particularly at interim stages of the intervention.  Laying out the theory is important because it reduces the temptation to use measures that might be easy but not directly helpful in measuring progress towards the objective

    Over the next few weeks I’m going to spend some time looking at some of this literature because I think that it casts light both on the question of measurement and evaluation and of the range of influence mechanisms that can be deployed.

    Here’s a very useful introduction to work in the area that includes a list of useful resources.

    Jones, H. (2011) A Guide to Monitoring and Evaluating Policy Influence. London: Overseas Development Institute. Available here

Bruce Willis and the Libyan Revolution

In the cablegate files there’s a report on a visit to Derna in Eastern Libya (its further to the east than Benghazi)  in June 2008.  The Libyan government monitoring seems to have got screwed up and the American diplomat was able to interact with local people relatively freely.  Derna had a reputation as a source of foreign fighters in Iraq something which is supported by the diplomat’s report.

Depictions on al-Jazeera of events in Iraq and Palestine fueled the widely-held view in Derna that resistance to coalition forces was “correct and necessary”. Referring to actor Bruce Willis’ character in the action picture “Die Hard”, who stubbornly refused to die quietly, he said many young men in Derna viewed resistance against Qadhafi’s  regime and against coalition forces in Iraq as an important last act of defiance. 12. (C) Claiming “most Libyans” shared that sentiment, xxxxxxxxxxxx proudly said the difference was that Derna’s sons actually acted on their beliefs. Derna had historically resisted “occupations of all kinds – Ottoman, Italian, American (a reference to the 1805 attack on Derna led by William Eaton), and Qadhafi’s.” Derna’s role in opposing the Italian occupation in the early20th century helped foster the near-deification of Libyan resistance leader Omar al-Mukhtar, who hailed from eastern Libya. A visit to the al-Sahab mosque near the town’s center was telling. Large murals on the mosque’s exterior (inaccurately) depicted Islamic warriors besting what appeared to be Roman soldiers. The mosque’s imam showed P/E Chief a series of small shrines to medieval holy men and a small cemetery filled with graves of “martyrs” who had resisted Ottoman and Italian occupation. Many of the markers were garlanded with flowers; xxxxxxxxxxxx said families  often come to picnic in the mosque’s garden on holidays and pay their respects at the cemetery.

The report notes that local people see the US as propping up Gadaffi’s regime: their view shaped by the mosque and satellite TV.

Also see my earlier post about the US offer of ‘soft power programming’ in Eastern Libya to the government.

Theoretical Implications of Strategic Communication Part 3: Cutting SC Down to Size

Just to finish off (for now anyway) my thoughts about the rise of the concept of strategic communication part 1 here and part 2 here

  1. The problem of multiple objectives can’t be resolved by better coordination (although better coordination is desirable) it comes from the fact that countries have multiple  objectives.  It then follows that states have to make choices and tradeoffs between objectives.  It is important not to allow the military aspects of strategic communication to limit the recognition of the set of relevant objectives at the national level;  for instance economic objectives (exports, investment, tourism)  will feed into the trade offs that need to be made.  For example giving prominence to communications and actions focused on combating violent extremism  might undermine the attractiveness of the US as a place to invest, work or study in.  Deciding how to make these tradeoffs is the role of politics.
  2. Hence rather than seeing strategic communication as an overarching national level concept it is better seen as something that should be conceptualized  at theatre, operational or issue levels.  The consequence is that the employment of these means is subordinate to national level objectives.   I get the impression that strategic communication has grown from something that operates at a theatre level to something that exists at a national level.  Of course this reflects the nature of the modern communication environment but also reflects the the absence of a national level strategic communication concept that has created space for an operational level concept to expand to fill the gap.
  3. What would a national level concept look like?  One idea would be for a national branding strategy that provide a context for (and limit on) subordinate strategic communication strategy.  The branding strategy would have the responsibility for balancing different priorities (and keeping the in mind that there is more to the US than combating violent extremism). At present there seems to be a tendency to see the US public diplomacy problem through the lens of CVE and neglect the strength of other aspects of the US position eg here and here.

The French Institute

A few months ago I posted some material about the plan to reorganize French Cultural Diplomacy.  The  new Institut Francaise launched in early January.

The basic thrust of the reforms seems to be get better coordination between the cultural work of the foreign ministry and of the French cultural institutes that exist in many cities around the world.  Up until now these institutes have worked independently of each other and the intent of the new organization is to create something rather more centralized like the British Council or the Goethe Institute – organizations that are explicitly mentioned in much of the discussion.  However, it looks like most of the existing cultural institutes will continue to operate indepently for the moment.  There is a plan for cultural activities in ten countries to come under the direct control of the new Institute for an experimental period – the success of this will have implications for the rest of the network.  The countries involved in the experiment are Cambodge, Chili, Danemark, Emirats arabes unis, Géorgie, Ghana, Grande-Bretagne, Inde, Koweït, Sénégal, Serbie, Singapour, Syrie.

Although the new Institute is supposed to be a joint enterprise of the foreign ministry and the ministry of culture it’s not clear from the documentation that I’ve seen how this will work in practice.

One other point is the priority that France continues to give to Europe in its cultural work, the British Council has been scaling back its work there in order to give more attention to the middle east and the rising industrial powers.

The web site of the Institut Francais is here. Here’s a press release in English on the launch of the organization and a fuller description in French.  There’s a collection of articles from the French press here.

As a bonus here’s an English Org Chart for the Directorate-General of Global Affairs, Development and Partnerships the part of the Foreign Ministry that deals with (among other things) cultural matters

Five Things I Learnt at the ISA

Darryl Copeland has already posted about PD at ISA in Montreal last week but here’s five things that I learnt

1. A lot of the PD panels at ISA were sponsored by the Diplomatic Studies Section rather than by International Communication and I think that this points to the growing impact of the Diplomacy/Public Diplomacy fusion on both practice and academic perspectives. Jan Melissen made the comment (based on his work in diplomatic training) that the younger generation of diplomats ‘get it’ – you no longer have to explain what PD is and why it matters.

2. There were several interesting papers on the domestic aspects of PD – for instance on UK counter radicalization efforts that link the UK and Pakistan, or the domestic elements of campaigns for NATO membership.

3. There was an interesting panel on PD and the EU which pointed to the fact that PD is not just something that the EU tries to do externally but is a tool used by members inside the EU.

4. The American colleagues are increasingly sceptical about US international broadcasting.  The view was expressed on several occasions that the money given to the BBG might be better spent on other activities.

5. A fair amount of head scratching over the question of evaluation – but I’ll come back to this in a separate post.

 

BBC asks for US funding – not quite

There’s a certain amount of excitement about the news that the BBC has applied for funding from the State Department:  how can state be giving money to foreigners when funding for US broadcasting is under pressure.

It’s not actually the BBC it’s the BBC World Service Trust which is a separate charity.

The BBC World Service Trust is the BBC’s international charity. We use media and communications to reduce poverty and promote human rights, thereby enabling people to build better lives.

Independently funded

We are funded by external grants and voluntary contributions, mainly from the UK’s Department for International Development (DFID), the European Union, UN agencies and charitable foundations. We receive a small amount of core support from the BBC (both in kind and cash).

Vision

We believe that independent and vibrant media are critical to the development of free and just societies.

We share the BBC’s ambition to provide accurate, impartial and reliable information to enable people to make informed decisions.

We aspire to a world where individuals and civil society use media and communications to become effective participants in their own political, economic, social and cultural development.

More about the World Service Trust and what it does here

Public Diplomacy/Soft Power Papers from ISA 2011

I’m flying back from Montreal today.  As well as pre-conference working group on public diplomacy there is a growing interest in PD.  Over the next few days I will post some comments and thoughts on the state of the field.  In the mean time I’m posting links to a selection of papers on PD and related topics that have been posted to the online conference archive.  This list isn’t complete and I will add to it in due course.  There are a number of interesting papers that haven’t been posted yet and some of the most interesting discussions occurred in roundtables that don’t require written papers.

Note – I’ve put these up quickly and I haven’t had a chance to check all the links yet – for links that take you to a page where you can click to view online you can go to the link to Allacademic at the bottom of the page and get a pdf.

 

Public Diplomacy and Prosumption: Limitations and Contradictions of U.S. Web 2.0 Engagement Activities

Edward A. Comor: University of Western Ontario

Diplomacy Action in the Age of Networks: the Agency Debate

John Robert Kelley: American University

Religious Transnational Actors and Soft Power

Jeffrey P. Haynes: London Metropolitan University

EU Public Diplomacy: A Coherent Message?

Mai’a Keapuolani Davis Cross: University of Southern California

City Diplomacy in the EU Context

Teresa La Porte: University of Navarra

Within the European Union: New Members’ Public Diplomacy

Beata Ociepka: University of Wroclaw

The Implicit Becomes Explicit? The EAAS and Public Diplomacy

Simon W Duke: EIPA

Exploring Diaspora Diplomacy through the Case of US Public Diplomacy toward Lebanon

Deborah L. Trent: George Washington University

Domestic Face of Public Diplomacy

Jana Peterkova: University of Economics, Prague

Introducing Corporate Political Activity as a Part of Business Diplomacy

Jennifer Kesteleyn: Ghent University

International Persuasion, Teleological Identities, and Pragmatics

Sanjoy Banerjee: San Francisco State University

Reporting the Nation: Understanding the Role of Journalism in 21st Century Public Diplomacy

Cristina Archetti: University of Salford

Hearts and Minds’ vs. Heads on the Block: Winning Domestic Support for King Henry VIII’s Divorce Diplomacy with Pope Clement VII

Geoffrey Allen Pigman: Bennington College

Practices of Public Diplomacy in Communicating NATO and EU Values with the Domestic Public in Croatia

Suzana Simichen Sopta: Ministry of Foreign Affairs and European Integration, Iva A. Tarle: Ministry of Foreign Affairs, Croatia, Mladen Andrlic: Ministry of Foreign Affairs and European Integration, Croatia

Securing the State through Dialogue at Home: The Bringing Foreign Policy Home Initiative: Pakistan Outreach

Caroline Clennell‐Jaine: South Asia Group, Steven Curtis: London Metropolitan University

Comparing the Soft Power and Public Diplomacy Capacity of China and Taiwan

Gary Rawnsley : University of Leeds

Public Diplomacy as Social Networking

Robin Brown: University of Leeds

I’ve seen this somewhere….

In 1937 Kingsley Martin visited the Paris Exhibition and described the British Pavilion

When you went in, the first thing you saw was a cardboard [Neville] Chamberlain fishing in rubber waders and, beyond, an elegant pattern of golf balls, a frieze of tennis rackets, polo sets, riding equipment, natty dinner jackets and, by a pleasant transition, agreeable pottery and textiles, books finely printed and photographs of the English countryside.

I read this and instantly had a mental picture of a Ralph Lauren store….

Quote from Philip M. Taylor, The Projection of Britain: British Overseas Publicity and Propaganda 1919-1939 (Cambridge: Cambridge University Press, 1981), p. 122