Archive for the ‘Planning, Evaluation and Measurement’ Category

h1

British Council Corporate Plan 2011-15

November 29, 2011

A quick look at the British Council’s corporate plan for 2011-15 in comparison with the 2008-11 version.

Priorities: In 2008-11 the plan was organized around three themes; intercultural dialogue; creative industries; climate change.  For the new period the priorities are arts, English and education.

Funding: The Council is looking at a 26% cut in direct government funding across the period at the same time it is looking to increase its income by 8% per annum with the result that by the end of the period the government grant will form only 16% of income.   This means that average annual growth rates for other income sources have to grow at high annual rates so we get projected growth of  teaching 13%, exams 9%, partnership 15%, contracts 13%.

Priority countries: Priority countries have been identified on the basis of 1) strategic importance defined as ‘relevance to government and stakeholder priorities and British Council objectives’ 2) potential for impact at scale and 3) business feasibility.  Within each country activities are shaped by a) level of economic development and b)’ openness to people, knowledge and ideas from other cultures’

So  priority countries are

Americas:  Brazil, Mexico and USA

East Asia: China, Indonesia, Japan, Thailand, Vietnam

Europe: France and Germany

Middle East and North Africa: Egypt, Iraq, Libya, Palestinian Territories, Saudi Arabia, UAE

South Asia: Afghanistan, Bangladesh, India, Iran, Pakistan

Sub-Saharan Africa: Nigeria, South Africa

UK

Wider Europe: Israel, Russia, Turkey

It looks like the reduction in the FCO grant is accentuating the need to follow the money with the Council focusing on activities where it can generate income.  Having said this the new corporate plan does not provide much detail on how it will achieve the growth in income required by the projections.   The reorientation of the Council’s work is being accompanied by job cuts and an effort to reduce the costs of real estate by using partner organizations to host their activities.

I’m certainly getting the impression that the past decade’s effort to build a comprehensive public diplomacy activity for the UK has run out of steam but I’ll pick this up in a later post.

h1

Premises for Propaganda

November 14, 2011

Continuing my campaign of digging into some of the older Public Diplomacy literature, this  morning’s offering is Leo Bogart, Premises for Propaganda: The United States Information Agency’s Operating Assumptions in the Cold War.  Published in 1976 this was an abridged version of a study that was carried out in 1953-54.  The intention was to investigate the working assumptions of the USIA in the belief that this exercise would reveal areas where practice diverged from best practice. According to Bogart when the report was submitted it was classified and not acted on and was not published until freedom of information legislation was passed in the early ’70s.

Bogart’s project interviewed 142 members of the USIA across different levels and sections of the organization.  The sample is light on field workers and overrepresents on the management levels.  The interviews took around two hours and covered a comprehensive range of topics.  The book explores views on the objectives of the Agency, how it makes policy, who its targets should be, the extent to which it should focus on projecting America vs attacking communism, images of the audience, views on truth and credibility, the importance of different media, the quality of personnel and evaluation.

If the expectation was that the study would reveal limited areas of disagreement it spectacularly failed : in every area there are substantial disagreements over what the Agency should be doing and how to do them.  This comprehensive lack of consensus may explain the reluctance to publish the report.  The report concludes with a list of 113 questions for further research and 23 questions that need to be resolved at a policy level.

The major weakness of the report is the lack of any systematic effort to explain the lack of agreement.  There are plenty of possible explanations; differences between field and headquarters, generational differences, differences between personnel with media and government backgrounds but there is no effort to identify clusters of beliefs that would allow a deeper understanding of what is happening.

At the same time anyone who has followed the debates over US public diplomacy over the past decade will recognize that almost all of the arguments that Bogart reports have recurred.  Two explanations come to mind.  Firstly, it’s simply a matter of a failure to conduct (and remember) enough research to resolve these issues. Secondly, there’s actually a deeper source in the way that public diplomacy stands between communications and politics which generates a familiar pattern of disagreement over what PD can do.

Bogart, L. (1976) Premises for Propaganda: The United States Information Agency’s Operating Assumptions in the Cold War. New York: Free Press.

h1

Evaluating Commonwealth Scholarships

November 7, 2011

The Commowealth Scholarships Commission in the UK has recently put out two reports evaluating the impact of its work in the Asia-Pacific region and a broader report on the impact of Commonwealth Scholarships in four priority policy areas: Governance, International Relations, Social Inequalities and Human Rights, and Conflict Resolution/Humanitarian Assistance.  The reports conclude that the scholarships are making a positive contribution to their objectives in particular the second report argues that scholarships in all areas a making a contribution to good governance.

Two comments. These reports were released to coincide with the  The Commonwealth Heads of Government Meeting in Perth, Australia at the end of October- so emphasizing the positive impacts of Commonwealth activities is not surprising.

The methodology of the reports is a survey of programme alumni. In looking at scholarship programmes this is typical.  The problem is that is that it is almost guaranteed to produce positive evaluations.  To really assess impact you would need to compare with a control group of non-scholarship recipients and find a way to evaluate ‘impact’ that doesn’t rely on self reporting – both of these steps are possible but complex and expensive and it’s understandable that scholarship organizations would rather spend their money on extra students.  From the point of view of assessing the public diplomacy impact of study in the UK self-reporting might actually be a perfectly good way of evaluating impact.

 

h1

British Council Corporate Plan 2011-15

October 26, 2011

Talking about the British Council as I was earlier I thought that I would check whether their new corporate plan had appeared- and here it is on the website.  I can’t tell when it appeared and there doesn’t seem to have been any media coverage.  As with a lot of these documents it probably needs to be read in conjunction with the annual reports and the previous plan.

h1

Management and the British Council

October 26, 2011

Public diplomacy sits between communications, politics and organization.  I think that the organizational dimension is essential in explaining why PD activities turn out the way that they do rather than following the dictates of strategy or the prescriptions of effective  communication.

With this in mind this morning’s offering is a pointer towards a couple of papers on the management and organization of the British Council.  I’ve written before about the emphasis on plans, management and strategies in British Government.  In a 1995 paper J.M. Lee discussed the impact of ‘the new public management’ on the BC. Lee’s point is that the Council’s move towards a more  ‘strategic’ language about it’s activities was driven not by an analysis of the changing international environment but by the changing culture of British government.  The result of this was a concern with value for money, efficiency and explicit management strategies.  The BC had to be able to present itself in a way that gave it credibility with its funders.  Although there a language of strategy it is organizational strategy not public diplomacy strategy.  The new managerial model  creates an incentive to ‘follow the money’ even if this pulls in a different direction to an externally oriented PD strategy.  I think that this tension between strategy as a tool of organizational management and as a way of influencing the external world is still not fully recognized in UK government.

The second paper by Venters and Wood looks at the BC’s efforts to deploy information technology to build a networked organization in which ‘communities of practice’ shared good practice across the globe.  To put it more simply how could it get it’s country directors to talk to each other? The basic answer is that they couldn’t.  In the pre internet era the country director had a link back to the central British Council but didn’t talk to other countries.  The new more businesslike  BC sought to shrink the centre of the organization, empower the country offices and encourage horizontal linkages.  Venter and Wood find that as centre shrank it became less useful to the Country Directors who used their improved access to IT to a)google for the information that they needed instead of asking the centre and b) develop  ‘communities of expertise’ with people outside the organization.

In thinking about effective PD one issue to keep in mind are the incentives that actually operate for organizations and the people within them.

Lee, J.M. (1995) ‘The Reorganization of the British Council: Management Improvisation and Policy Uncertainty’, Public Administration, 73: 339-55.

Venters, W., and B. Wood (2007) ‘Degenerative Structures that Inhibit the Emergence of Communities of Practive: A Case Study of Knowledge Management in the British Council’, Information Systems Journal, 17: 349-68.

h1

The Heritage Foundation on the Objectives for US Strategic Communication

September 5, 2011

Last week Helle Dale at the Heritage Foundation posted a piece about the progress of the US Center for Strategic Counterterrorism Communication within the State Department.

Within the post Dale offers a list of six priorities that should be used by Congress to measure the Administration’s progress on strategic communication and public diplomacy.   In looking at this list I was struck by the way that it mixed up different types of objectives.  In talking about PD a greater degree of precision would be helpful.

Here’s the list followed by my comments in italics.

  • Responding rapidly to misinformation.  The Digital Outreach Team needs to counter anti-American conspiracy theories in Pakistan and elsewhere.  The U.S. government badly needs the rapid response capability to counter enemy propaganda and other misinformation.  The cumbersome clearance process within the State Department is antithetical to the concept of rapid response, which means that this capability may better be housed in another agency—NCTC, for instance.
  • Previous research has pointed to the length of time that the DOT can take to respond to posting so streamlining the process seems like a good idea.  Also speed of response provides an objective metric.  But how important is the DOT in  countering misinformation in Pakistan given relatively low internet penetration?
  • Combating radical Islamism.  The U.S. needs to craft an official anti-Islamist narrative that can help discredit the ideology of jihad against non-Muslims.  In order to do this, analysts need a better understanding of the narratives that motivate young Muslims to become radicalized.  Both the Pentagon’s Office of Information Support (formerly psychological operations) and the CSCC are doing work in this area.
  • This doesn’t have a specific objective attached.  This is an area where great importance is attached to the idea of the narrative.  I’m not convinced that there is a single narrative and that this is worthy of being made a key objective? 
  • Aiding Iran’s Green Movement.  U.S. agencies should spotlight Tehran’s past assassination campaigns against opposition leaders in exile, as well as its continued financial support of terrorist organizations like Hamas and Hezbollah while the Iranian people are increasingly impoverished. These are classic public diplomacy targets.
  • This is turning a specific message into an objective – it seems a bit strange.   Who are the targets for this message?  Is this a specific counterterrorism message rather than a general PD message.
  • Formulating a multi-tiered Internet freedom strategy. The U.S. should go beyond funding for Internet circumvention technology and should mount a strong push for international cooperation through a coalition of nations willing to stand up for freedom of expression
  • Clearly this isn’t a job for the CSCC rather it’s an overall foreign policy objective.
  • Securing broadcast cooperation. The U.S. should work with the Broadcasting Board of Governors to make international broadcasting part of an integrated government-wide U.S. counterterrorism communications strategy. The firewall established by the U.S. International Broadcasting Act of 1994 between State and BBG to ensure editorial independence for the broadcasters has turned into a detriment in terms of resource allocation and lack of congressional oversight.
  •  The argument over US international broadcasting and its link to PD is  a very old one.  Dale is taking one side and Kim Andrew Elliott is taking the other side that international broadcasting is an autonomous function.  My comment is that if you were going to come down on the ‘broadcasting is PD’ side of the argument it would make more sense to subordinate it to an overall vision of PD not to a single element, counterterrorism, of the overall programme. 

It makes sense to set clear objectives for PD work but that objective setting process has to take into account where responsibility for different activities sit.  Part of doing this is being clear about the place of PD within foreign policy and the place of counterterrorism within foreign policy and public diplomacy.  There’s more to PD than counterterrorism and forgetting that is not helpful to America’s reputation.

 

 

h1

Quote of the Day

August 26, 2011

I’m reading Leo Bogart’s, Premises for Propaganda: The United States Information Agency’s Operating Assumptions in the Cold War (New York: Free Press, 1976), this is based on a report that he compiled for the agency in 1953-54. From page 55 an anonymous USIA official commenting on the targets for public diplomacy:

‘Any given area, country, or city can be divided into 5 percent operators, 10 percent stooges,  and  85% slobs’

 

 

h1

Evidence on Perceptions and Foreign Investment

June 20, 2011

Does nation branding have economic significance? Via Nation Branding we learn of a German study that claims that a one point rise in your score on the Anholt Nation Brands index leads to a 27% increase in foreign direct investment. This sounded a bit too good to be true so I’ve tracked down the published version of the research.

Kalamova and Konrad are interested in the impact of investor perceptions on FDI decisions relative to economic fundamentals. They look at FDI flows and statistically test the extent to which these patterns can be explained by standard economic models and by the impact of perceptions. They find that a one point rise in your NBI score is associated with a 27% greater FDI flow above what would be predicted by the standard economic model. I think that this is an important qualification – the article is not saying that a 1 point increase in perceptions will make your flow 27% regardless of everything else. This relationship is statistically significant at the 1% level which implies a very strong association.

I’m convinced that perceptions do matter but I’m a bit sceptical about the size of the effect. FDI flows tend to be quite volatile and for smaller countries single investment decisions can have a big impact on the overall figures. For this reason it would be really helpful to have the study extended to cover more than two years. It’s also worth emphasizing Anholt’s point that the NBI is measuring a whole set of factors including things like governance and economic conditions so an improvement in NBI is assumed to be based on more than nation-branding as advertising.

Kalamova, M.M., and K.A. Konrad (2010) ‘Nation Brands and Foreign Direct Investment’, Kyklos, 63: 400-31.

h1

Planning, Evaluation and Public Diplomacy Part 2: MFAs, Militaries and Aid Agencies

April 11, 2011

Looking at the material on planning for part 1 of this post raised a broader issue about planning and diplomacy.   From the point of view of scholars of International Relations or Communications planning seems like a tedious topic but actually says quite a lot about the nature of foreign ministries and their relations with the rest of government – particularly with Ministries of Defence and Aid Agencies organizations where planning has been taken much more seriously.  Planning is where organizational culture and an evolving external environment intersect with each other.

Military organizations are inveterate planners. This is not just to say that they draw up contingency plans but that planning is how they deal with the world and define who they are.  Military staff training is about how to plan;  if something happens the response is to start planning.   What the military means by planning is how do we organize things so that we can achieve a defined end state.  Planning is the means by which the various elements of the military organization are brought together.

I’ve also seen it said (I think in relation to Bosnia) that the reason that the US Army finds it hard to work with the State Department is because State doesn’t plan (my italics) , also that the problem with other armies is that they don’t plan (how much you plan is relative the British military plans much more than the FCO but not as much as the US military) . What this says to me is that the process of planning is something close to the heart of the military identity.

Given the range of moving parts in a military operation is obvious that plans are necessary  but planning also has other consequences.   In looking at the development guides in the first part of this post  I was reminded of Eisenhower’s maxim -’ plans are worthless but planning is essential’; the development community sees planning as way to build relationships.  Planning is a way to  learn about potential problems and opportunities, you learn about the people that you are working with and you develop a common understanding of what the problem is and what you are going to do about it that is relevant even if the specific plan is never used or has to evolve in unexpected ways.  This is an important lesson for relational (or collaborative) public diplomacy; that the process of planning is important part of the whole excercise.

In contrast with armies and aid agencies  MFAs have generally taken the line that the international environment is so complex and unpredictable that strategic planning is a waste of time.  Historically MFAs have done some  ‘policy planning’;   that is trying to imagine what the world will be like or how some issue will develop and  preparing policy options for what should be done – but  policy planning does not set objectives or allocate resources (Hocking 1999, Rubin 1987, Lan 2007).  Over the past couple of decades  this position has come under pressure from changing models of government organization and the changing nature of diplomacy itself.  As the scope of diplomatic action has expanded MFAs have been expected to become more strategic but there are big differences in how far these developments have gone. The FCO has moved towards a thoroughgoing strategic management approach while there are suspicions that despite innovations in planning the State Department doesn’t quite take this seriously.     Lane (2007) makes the point that the adoption of a much more strategic approach by the FCO over the past 10 years has changed the notion of planning from the old policy planning model to something that is much more strategic. As I’ve noted before the FCO seems to be much successful in meeting objectives that involve its own processes (eg consular support)  than the ones that depend on foreigners which perhaps suggests that there is something to the traditional suspicion of planning.   Lane also warns that operational planning can  lead to organizations focusing on the immediate objective  and losing sight of how the changing environment may require a change of policy.

What is interesting though is a certain convergence between the planners and the diplomats.  There is a rather obscure  debate going on in the US Army over the topic of ‘design’ .  The starting point is the danger that the current way of thinking about plans leads to rigidity and an inability to adapt to rapidly changing environments.  The new edition of field manual  FM5-0 The Operations Process draws on concepts of complex systems to advocate a mode of thinking that encourages adaptation and learning in response to a complex and interdependent world non linear transformations are the norm. Unsurprisingly some of the reaction to this has been frustration;  how are these abstract exhortations to be translated into real plans? I think that this also parallels the way that aid organizations are trying to break out of the rigidities that can emerge with formal planning.  To some extent it suggests a movement towards the traditional diplomatic position that improvisation is the only really feasible approach to international relations.

I don’t think that the divide between the military and aid planners and the improvising diplomats is going to disappear anytime soon but it does appear that both sides of the divide are moving towards each other.

Hocking, B., ed. (1999) Foreign ministries : change and adaption. Basingstoke: Macmillan.
Lane, A. (2007) ‘Modernising the management of British diplomacy: towards a Foreign Office policy on policy-making?’, Cambridge Review of International Affairs, 20: 179-193.
Rubin, B. (1987) Secrets of state : the State Department and the struggle over U.S. foreign policy. New York: Oxford University Press.
h1

Planning, Evaluation and Public Diplomacy Part 1: Logframes and Beyond

April 10, 2011

A few days ago I suggested that in thinking about the questions of measurement and evaluation in public diplomacy there might be scope for looking at the way the development community deal with these issues.  In a comment Debbie Trent pointed towards the literature on ‘logframes’ or logic frame analysis (LFA).  I’ve been looking at some of the literature and it in turn has triggered some broader reflections on the role of planning and what it means in international affairs.

Firstly, a bit more on logframes.  LFA is a planning tool that has been around since the 1960s and is very widely used across the development community – it exists in different versions in different countries.

The signature of LFA is the frame or matrix itself which looks something like this (based on Gasper 2000)

Hierarchy of objectives Performance indicators Data sources Assumptions and risks
Goal – longer term impact      
Purpose – short term impact      
Outputs – deliverables      
Activities – work package for outputs      

The vertical dimension sets out a means-end chain showing how the elements of a project relate to each other.  The purpose is usually the direct objective of the project  while the outputs are the components of the project.  These diagrams can have more rows.  The horizontal dimension sets out how we can tell if the project if working, where the data to tell this will come from and the assumptions that we have to be true for the objectives to be met.

The matrix is intended to be the end point of a planning process that  involves  a systematic analysis of the existing situation and conversations with collaborators to ensure a common understanding of the problem.  The difficulty is that the relationship between the logframe and the planning process is sometimes rather looser that it should be  Gasper identifies four problems

  1. Logic-less frame -an  existing project is presented in matrix without any real analysis
  2. Jamming – an oversimplified version of the project is used so it will fit the matrix.
  3. Lack-frame – too much left out
  4. Lock-frame – failure to update the frame as the project develops and new issues become apparent.

This frustration has led to a search for alternatives for instance Davies (2008) advocacy of a ‘social frame’ that focuses on the changes in relationships that must occur for a project to be successful.  In the UK development community the emphasis on ‘theories of change’ seems to be aimed precisely at getting back to the process of research, communication and learning, that is supposed to underpin the matrix.

The basic thrust of thinking in the development community is that monitoring and evaluation always takes place in relation to what you are trying to do. This places a heavier burden on developing feasible and systematically worked through plans.  Of course this doesn’t always happen in the development community any more than it does in public diplomacy but I think that making the effort to lay out what the theory of influence that underpins any activity is a necessity for any PD activity.  This raises the question to what extent to PD organizations actually make use of systematic planning tools in developing their activities.

In the second part of this series I’m going to make some comments about the difficult history of planning in diplomacy.

Notes on reading

The article by Gasper and the responses to it (h/t do Debbie Trent for the references)  give some insight into the origins of the logframe approach

Gasper, Des. “Evaluating the ‘Logical Framework Approach’ Towards Learning-Oriented Development Evaluation.” Public Administration and Development, Vol. 20, No. 1 (2000): 17-28;

Bell, Simon. “Logical Frameworks, Aristotle and Soft Systems: A Note on the Origins, Values and Uses of Logical Frameworks, in Reply to Gasper.” Public Administration and Development, Vol. 20, No. 1 (2000): 29-31.

Smith, Peter. “A Comment on the Limitations of the Logical Framework Method, in reply to Gasper, and to Bell.” Public Administration and Development, Vol. 20, No. 3 (2000): 439-441.

This is a comprehensive discussion of how to develop a logic frame including interaction with stakeholders and how decide which are the important assumptions.  This is part of a set of guides on project planning.

AusAid (2005), The Logical Framework Approach, AusGuideline 3.3 (Canberra: AusAid)

Davies, Rick (2008), ‘The Social Framework as an Alternative to the Logical Framework’ available here argues for rethinking the logic framework in social terms.

Keystone (2009), Developing a Theory of Change (London: Keystone Accountability).  This is part of a series of guides on Impact Planning, Assessment and Learning

Follow

Get every new post delivered to your Inbox.