Perverse Payment by Results: frogs in a pot and straitjackets for obstacle courses

03/09/2014

Robert ChambersRobert_Chambers200

Perverse  adj. deliberately deviating from what is regarded as normal, good, or proper (Collins English Dictionary)

The Department for International Development (DFID) claims to be a world leader in developing results-based aid, and now Payment by Results (PbR).  PbR means that recipients will have to show results before they are paid. It will help to share risk, we are told, and radically re-balance accountability.  It makes sense for DFID ‘to take a tougher, more business-like approach by requiring results up front before payment is made.  Better sharing of risk in this way will drive value for money as partners become more incentivised to deliver’.

The drift of the past two decades away from participation and towards top-down controls and upwards accountability has been continuous and gradual, a heating of the water in the pot. The logframe, results-based management, upwards accountability, delivering value for money, business cases… these are motherhood and apple pie words with their mantras and procedures.  They have to be good.  More of them has to be better.  But what has been happening to the frogs in the pot as these procedures intensify and heat it up?

Inside the DFID pot, am I right? More managers have been recruited who have no experience of development. Total staff numbers have been reduced while budgets have increased! (Before he became Secretary of State Andrew Mitchell said in public on two separate occasions at the University of Sussex that to do this would be ‘ridiculous’). But it is worse than this.  The staff to funds ratio has declined while in the name of accountability and value for money internal procedures have become more and more laborious and staff intensive.  Not least of these is making the business case for a project, which is much the same time-consuming effort for £500,000 as for £50 million, 100 times as much.  So staff are more tied than ever to their headquarters and offices, have less and less time for ‘the field’ and actually grounding and learning the truth for themselves of what is going on, at a time when aid is going out in bigger dollops.  Value for money? The DFID frogs in the heating pot have forgotten (or the recent tadpoles never knew) what it used to be like. The frogs already there socialise them into acceptance. And how can they jump out?  They need the water – work, careers, and (unspoken) the power that comes with increasingly stringent upwards accountability.

Realities beyond the pot

And outside the DFID pot, the recipients or would-be recipients are facing obstacle courses of complexity, emergence and unpredictability.  This applies markedly with NGOs and with research – NGOs work in highly risk-prone environments, research would not be needed if the results were known – but  DFID funding has increasingly become a dysfunctional demoralising nightmare with escalating risks and transaction costs. Goals and targets have become more rigid, measurement has taken over from judgement, linear Newtonian thinking and action denies and obliterates non-linear realities, motivation has to be ‘incentivised’ by carrots and now with PbR the sticks have no carrots at all, relationships have become distant, trust, discretion and flexibility have gone out of the window and reporting becomes more of a misleading nightmare.  And as aid monies go in larger lumps to consortia so transaction costs are transferred from inside to outside DFID, and unsuccessful bidders suffer disappointment, distress and costs…  Am I wrong?  I invite rebuttal.  Let’s be evidence-based.

Take rural sanitation.  We have learnt a lot in the last decade:  the top-down building of toilets for people doesn’t work as CLTS (Community-Led Total Sanitation) has shown – people have to want to end open defecation, have to want toilets, and have to use them, keep them clean, and maintain and improve them; the astonishing spread of CLTS in African countries and elsewhere and its adoption by over 20 governments is evidence enough – despite problems common to all programmes that go to scale it is dramatically more cost-effective than earlier approaches, and this is well known in DFID.  But we all know that with good and enduring participation, you cannot predict speed.  Perversely there are evidently now targets, paid by results, for toilets.  Forced to achieve targets for payment after results, organisations may have to choose: abandon participation and do what they know won’t work, or go bankrupt, or lie (well, massage facts, gloss realities, mislead, report on what was planned not learn from what happened…).  I know a participatory organisation with an excellent track record which is being bankrupted through one of these contracts: the senior management have had no remuneration for six months.  If it goes under, this will be a result of PbR, evaluators please note, and please ensure, DFID, that negative externalities like this will be identified, costed and reported in all evaluations of PbR.

The rise of PbR – to what end and at whose cost?

Standing back, let’s ask:  what is going on?  How far will this go?  At what gratuitous cost to people living in poverty, and diminished cost-effectiveness of our tax payers’ money?  Is this what we had in mind when we campaigned for 0.7 per cent?   For the obstacle course of development, are straitjackets now to be de rigueur?  Brave new world.   And who is accountable to whom?  There isn’t the faintest whiff of accountability to poor people.  Has that been utterly forgotten?  And how many aid managers have development field experience (and by field I don’t mean in other capital cities)? Do they confuse accountability – the words are similar – with accountancy?  DFID is proud of meeting its spending targets!  Who in DFID recognises that not spending budgets can be an indicator of participation and empowerment?  That it is good to save money? Who rewards those who save money or spend it slowly because they are empowering people through participation?

The perversities of PbR and related approaches are set to diminish value for money:

  • Misfit with complexity. PbR and fixed goals, targets, milestones misfit complexity, unpredictability, flexibility, adaptability. Strait jackets for obstacle courses? Less that is good will be achieved. The Secretary of State has claimed that ‘DFID is … becoming a world leader in pioneering innovative Payments by Results programmes for tackling complex development problems’ (my italics) It is precisely with complex development problems where PbR’s misfit is most stark and tragic, and worst value for taxpayer’s money.
  • Misleading reporting.  Incentives can be perverse incentives to report results. Shoddy work: do what the numbers demand, never mind sustainability, ownership, empowerment – you can’t measure those so easily.  Bias to the measurable. DFID  ‘ …being paid by results… (drives up) performance standards, management and measurement {sic]’.  Yet Andrew Naitsios, former head of  USAID said ‘ …those development programs that are most precisely and easily measured are the least transformational, and those programs that are most transformational are the least measurable’ (No surprise that infrastructure is so much the rage – it can be measured)
  • View of human nature.  ‘Incentives’, ‘incentivise’ – in PbR there is an unspoken implication that people working in ‘aid’ are doing it for the money  (an insulting view of what drives people in development NGOs and research).  That is not what motivates the vast majority of the people I know.  They work in development because they actually care and want to make a difference
  • Disbursement targets.  It is a matter of pride that DFID is ‘meeting its spending targets’. Spend the budget by the end of the financial year.   Get the money out of the door in time.  Tick.  But too much aid too fast is a widespread complaint from recipients.

I could go on. In the world of NGOs and research at least, and probably much more widely, the effects of PbR seem set to be cruelly perverse.

In any case, where is the evidence that PbR is better than alternatives? Until PbR is soundly evidence-based, while DFID may  be a world leader in payment by results, for aid to be cost-effective, and for the sake of people living in poverty, let there be no followers.  If DFID want to be a world leader, perhaps this could be in honest, insightful evaluation of PbR which goes deep into its externalities and  the realities of all who are affected.

Alternative paths to the PbR highway

And let us note that outside the DFID pot, there is another, thrilling, world evolving.  Last month I was at the biennial Development Cooperation Forum in New York, convened by the Economic and Social Council of the UN.  There was talk there of ‘trust-based inclusive partnerships’ ‘horizontal relationships’, ‘South-South collaboration’ and the sense of a new dawn, a re-recognition of the vital significance of relationships which are not distorted by power.

So in PbR, where is the understanding of people, of communities, of participation, of complexity, where is the listening, where are the relationships in this world sanitised of humanity, and of people, of the personal?  Read the wonderful book Time to Listen: hearing people on the receiving end of international aid (Anderson et al 2012) whose researchers listened to almost 6,000 recipients of aid.  And what did they say? They said they appreciated aid but would prefer not to need it.  They said it comes too much and too fast.  They said they want and need relationships with donors, face-to-face, but contact is rare, if ever, and then rushed. They said they had to divert time and energy from action to demoralising and misleading reporting and upward accountability.  Again and again, they point to hidden transaction costs….  And the Time to Listen researchers found that wherever there was a good initiative or project there was always a person.

To conclude:  To give better value for money, please DFID

  • radically simplify and streamline your internal procedures and free up staff time,
  • recruit many more staff with grassroots field experience and who understand development realities, and free them to be in touch with the rapidly changing realities outside their pot, and
  • put PbR on hold until a few examples can be evaluated in depth over time.

If you did these three things, you would indeed, truly, be leading the world.  And there is one more thing.  To deliver astronomical value for money (and without a business case) the Secretary of State could authorise the purchase of 2000 copies of Time to Listen and send them to all DFID staff.  As a starter, I am mailing her a copy.

Robert Chambers is a Research Associate in the Participation, Power and Social Change research team at IDS.

Read other blog posts from Robert Chambers:

 

Advertisements

The power of results and evidence artefacts

13/12/2012

Rosalind Eyben photo miniRosalind Eyben

The Big Push Forward is inviting case studies as evidence about the politics of evidence for sharing at next April’s conference. Please send us your stories. If you prefer your name not to be published, we will help you edit the story to protect your anonymity.

Development projects and programmes are increasingly being planned, appraised, implemented and evaluated in terms of ‘results’ and ‘evidence’. ‘Evidence’ is about what works to solve a problem which leads to action – intervention or treatment of the problem – that delivers ‘results’ which are reported upon and possibly also evaluated. The two discourses share in common a particular understanding of causality, efficiency and accountability that originated in and remains more prevalent in countries with an anglo-saxon empiricist tradition. However, through the dominance of English-language based global institutions such as the World Bank, they are spreading widely within the international development sector and into developing countries.

Results and evidence discourses shape our working practices through artefacts, such as logical framework analyses. These gain their power through incentives (carrots) and mandatory requirements (sticks). Different results and evidence artefacts are used in different processes at various stages in the development sector’s funding cycles. Results artefacts are used for planning, implementation, monitoring and reporting purposes e.g.

  • Payment by results
  • Results reports
  • Performance measurement indicators
  • Logical framework analysis
  • Theory of Change
  • Base-line data
  • Progress reviews

Evidence artefacts are used for choice of intervention, detailed appraisal and evaluation e.g.

  • Randomized control trials
  • Systematic reviews
  • Cost-effectiveness analysis
  • Option appraisal
  • Social return on investment
  • Business case
  • Impact evaluation

We may become so accustomed to using one or more of these artefacts that no external control is required to ensure our compliance. It is also common for an organisation to voluntarily adopt one of these artefacts in the absence of any mandatory donor requirement. Or even when their use is mandatory, a grant receiving organisation may be more exigent and controlling in how they are used than may had ever been envisaged by the artefacts’ originators.

Thus artefacts can take on a life of their own, independent of the authority that had initially required their use. Whether we find an artefact useful in our endeavours will influence how we feel about it. Our personality, experience and kind of job – including for example if we are independent consultants working for different organisations – may also influence our response. But the emotional and power effects such artefacts have on us also depends on the kind of organisation we work for, not only its position in the aid chain but also on its institutional culture and leadership. For example, two recipients of a similar grant from the same funding agency may differ widely in their attitudes and response to identical mandatory requirements.

Results and evidence artefacts are increasingly shaping our work experience in the international development sector. They frame what we are trying to achieve, shape how we spend our time and influence how we relate with our colleagues, partners, grantees and donors. What has been your experience? Please send us your case studies of the effects of any one of the artefacts listed above (or of another results or evidence artefact of which you have personal experience.

The Big Push Forward is looking for…

  • A case of how you have observed a specific artefact’s effects in practice (Note that we are not looking for a commentary on an artefact’s theoretical strengths and limitations but rather for stories of how you have observed the effects in practice.
  • Between 400-700 words would be good;
  • Firsthand experience (not hearsay) and from the development sector.

We need your email address but won’t share that with anyone else if you tell us you want to stay anonymous. If you do, the organisations and individuals described in your story must be given pseudonyms. Please contact us directly with your submissions.

Rosalind Eyben is a Research Fellow in the Participation, Power and Social Change research team at IDS and her Twitter account is: @rosalindeyben

Previous blog posts by Rosalind Eyben: