Perverse Payment by Results: frogs in a pot and straitjackets for obstacle courses

Robert ChambersRobert_Chambers200

Perverse  adj. deliberately deviating from what is regarded as normal, good, or proper (Collins English Dictionary)

The Department for International Development (DFID) claims to be a world leader in developing results-based aid, and now Payment by Results (PbR).  PbR means that recipients will have to show results before they are paid. It will help to share risk, we are told, and radically re-balance accountability.  It makes sense for DFID ‘to take a tougher, more business-like approach by requiring results up front before payment is made.  Better sharing of risk in this way will drive value for money as partners become more incentivised to deliver’.

The drift of the past two decades away from participation and towards top-down controls and upwards accountability has been continuous and gradual, a heating of the water in the pot. The logframe, results-based management, upwards accountability, delivering value for money, business cases… these are motherhood and apple pie words with their mantras and procedures.  They have to be good.  More of them has to be better.  But what has been happening to the frogs in the pot as these procedures intensify and heat it up?

Inside the DFID pot, am I right? More managers have been recruited who have no experience of development. Total staff numbers have been reduced while budgets have increased! (Before he became Secretary of State Andrew Mitchell said in public on two separate occasions at the University of Sussex that to do this would be ‘ridiculous’). But it is worse than this.  The staff to funds ratio has declined while in the name of accountability and value for money internal procedures have become more and more laborious and staff intensive.  Not least of these is making the business case for a project, which is much the same time-consuming effort for £500,000 as for £50 million, 100 times as much.  So staff are more tied than ever to their headquarters and offices, have less and less time for ‘the field’ and actually grounding and learning the truth for themselves of what is going on, at a time when aid is going out in bigger dollops.  Value for money? The DFID frogs in the heating pot have forgotten (or the recent tadpoles never knew) what it used to be like. The frogs already there socialise them into acceptance. And how can they jump out?  They need the water – work, careers, and (unspoken) the power that comes with increasingly stringent upwards accountability.

Realities beyond the pot

And outside the DFID pot, the recipients or would-be recipients are facing obstacle courses of complexity, emergence and unpredictability.  This applies markedly with NGOs and with research – NGOs work in highly risk-prone environments, research would not be needed if the results were known – but  DFID funding has increasingly become a dysfunctional demoralising nightmare with escalating risks and transaction costs. Goals and targets have become more rigid, measurement has taken over from judgement, linear Newtonian thinking and action denies and obliterates non-linear realities, motivation has to be ‘incentivised’ by carrots and now with PbR the sticks have no carrots at all, relationships have become distant, trust, discretion and flexibility have gone out of the window and reporting becomes more of a misleading nightmare.  And as aid monies go in larger lumps to consortia so transaction costs are transferred from inside to outside DFID, and unsuccessful bidders suffer disappointment, distress and costs…  Am I wrong?  I invite rebuttal.  Let’s be evidence-based.

Take rural sanitation.  We have learnt a lot in the last decade:  the top-down building of toilets for people doesn’t work as CLTS (Community-Led Total Sanitation) has shown – people have to want to end open defecation, have to want toilets, and have to use them, keep them clean, and maintain and improve them; the astonishing spread of CLTS in African countries and elsewhere and its adoption by over 20 governments is evidence enough – despite problems common to all programmes that go to scale it is dramatically more cost-effective than earlier approaches, and this is well known in DFID.  But we all know that with good and enduring participation, you cannot predict speed.  Perversely there are evidently now targets, paid by results, for toilets.  Forced to achieve targets for payment after results, organisations may have to choose: abandon participation and do what they know won’t work, or go bankrupt, or lie (well, massage facts, gloss realities, mislead, report on what was planned not learn from what happened…).  I know a participatory organisation with an excellent track record which is being bankrupted through one of these contracts: the senior management have had no remuneration for six months.  If it goes under, this will be a result of PbR, evaluators please note, and please ensure, DFID, that negative externalities like this will be identified, costed and reported in all evaluations of PbR.

The rise of PbR – to what end and at whose cost?

Standing back, let’s ask:  what is going on?  How far will this go?  At what gratuitous cost to people living in poverty, and diminished cost-effectiveness of our tax payers’ money?  Is this what we had in mind when we campaigned for 0.7 per cent?   For the obstacle course of development, are straitjackets now to be de rigueur?  Brave new world.   And who is accountable to whom?  There isn’t the faintest whiff of accountability to poor people.  Has that been utterly forgotten?  And how many aid managers have development field experience (and by field I don’t mean in other capital cities)? Do they confuse accountability – the words are similar – with accountancy?  DFID is proud of meeting its spending targets!  Who in DFID recognises that not spending budgets can be an indicator of participation and empowerment?  That it is good to save money? Who rewards those who save money or spend it slowly because they are empowering people through participation?

The perversities of PbR and related approaches are set to diminish value for money:

  • Misfit with complexity. PbR and fixed goals, targets, milestones misfit complexity, unpredictability, flexibility, adaptability. Strait jackets for obstacle courses? Less that is good will be achieved. The Secretary of State has claimed that ‘DFID is … becoming a world leader in pioneering innovative Payments by Results programmes for tackling complex development problems’ (my italics) It is precisely with complex development problems where PbR’s misfit is most stark and tragic, and worst value for taxpayer’s money.
  • Misleading reporting.  Incentives can be perverse incentives to report results. Shoddy work: do what the numbers demand, never mind sustainability, ownership, empowerment – you can’t measure those so easily.  Bias to the measurable. DFID  ‘ …being paid by results… (drives up) performance standards, management and measurement {sic]’.  Yet Andrew Naitsios, former head of  USAID said ‘ …those development programs that are most precisely and easily measured are the least transformational, and those programs that are most transformational are the least measurable’ (No surprise that infrastructure is so much the rage – it can be measured)
  • View of human nature.  ‘Incentives’, ‘incentivise’ – in PbR there is an unspoken implication that people working in ‘aid’ are doing it for the money  (an insulting view of what drives people in development NGOs and research).  That is not what motivates the vast majority of the people I know.  They work in development because they actually care and want to make a difference
  • Disbursement targets.  It is a matter of pride that DFID is ‘meeting its spending targets’. Spend the budget by the end of the financial year.   Get the money out of the door in time.  Tick.  But too much aid too fast is a widespread complaint from recipients.

I could go on. In the world of NGOs and research at least, and probably much more widely, the effects of PbR seem set to be cruelly perverse.

In any case, where is the evidence that PbR is better than alternatives? Until PbR is soundly evidence-based, while DFID may  be a world leader in payment by results, for aid to be cost-effective, and for the sake of people living in poverty, let there be no followers.  If DFID want to be a world leader, perhaps this could be in honest, insightful evaluation of PbR which goes deep into its externalities and  the realities of all who are affected.

Alternative paths to the PbR highway

And let us note that outside the DFID pot, there is another, thrilling, world evolving.  Last month I was at the biennial Development Cooperation Forum in New York, convened by the Economic and Social Council of the UN.  There was talk there of ‘trust-based inclusive partnerships’ ‘horizontal relationships’, ‘South-South collaboration’ and the sense of a new dawn, a re-recognition of the vital significance of relationships which are not distorted by power.

So in PbR, where is the understanding of people, of communities, of participation, of complexity, where is the listening, where are the relationships in this world sanitised of humanity, and of people, of the personal?  Read the wonderful book Time to Listen: hearing people on the receiving end of international aid (Anderson et al 2012) whose researchers listened to almost 6,000 recipients of aid.  And what did they say? They said they appreciated aid but would prefer not to need it.  They said it comes too much and too fast.  They said they want and need relationships with donors, face-to-face, but contact is rare, if ever, and then rushed. They said they had to divert time and energy from action to demoralising and misleading reporting and upward accountability.  Again and again, they point to hidden transaction costs….  And the Time to Listen researchers found that wherever there was a good initiative or project there was always a person.

To conclude:  To give better value for money, please DFID

  • radically simplify and streamline your internal procedures and free up staff time,
  • recruit many more staff with grassroots field experience and who understand development realities, and free them to be in touch with the rapidly changing realities outside their pot, and
  • put PbR on hold until a few examples can be evaluated in depth over time.

If you did these three things, you would indeed, truly, be leading the world.  And there is one more thing.  To deliver astronomical value for money (and without a business case) the Secretary of State could authorise the purchase of 2000 copies of Time to Listen and send them to all DFID staff.  As a starter, I am mailing her a copy.

Robert Chambers is a Research Associate in the Participation, Power and Social Change research team at IDS.

Read other blog posts from Robert Chambers:

 

17 Responses to Perverse Payment by Results: frogs in a pot and straitjackets for obstacle courses

  1. Edwin Phiri says:

    Once again Robert your analysis is spot on. I agree totally.

  2. Robert Stone says:

    Yes, spot on indeed, Robert. And the PbR perspective on the need to incentivise is not only insulting to those involved in NGOs and research. Most of those I work with in development consulting firms have a range of motives and drivers, of which money is only one, and for most of us, not the strongest one.

  3. Mansoor Ali says:

    While there are a number of important points in Roberts blog, I feel this note is very much putting all the blame on DFID. Have worked extensively with those who produce so called results and decide priorities, I think NGOs and professionals are equally responsible.
    Any thoughts?

    • Marc says:

      Agree completely. DFID has earned plenty of blame. NGOs have choices. NGOs have a responsibility to maintain a degree of independence and integrity. False reporting of results? Even a handful of examples of sham aid and ghost achievements will tarnish the entire sector.

  4. GB Adhikari says:

    People are only accountable to those who give money but not to those who were served by the money mean people at the receiving ends. Local knowledge, wisdom, skills and participation always get ignored and never counted in the development.

  5. Daniel says:

    Are not Anderson et al the tadpoles to a great old frog I know who wrote about how being accountable to those aid supports means listening, listenign well and observing – ” Listen to the People: Participant-Observer Evaluation of Development Projects by Lawrence Salmen (Apr 16, 1987) and others soon after? I learnt much about how, for example, listening to and observing how folk respond to a process stimulated by direct matrix ranking provides insights that helps you learn through holding yourself accountable. Doing so gives you opportunity to change and improve in ways that try and be attuned and responsive to those that legitimise us and our industry.

    As for DFID, I move elsewhere in my work, for now anyway. We all have options to exit.

    Not sure that sending a book on humanitarian support that mimicks a message, better and more succinctly put from 25 years ago will help, but hope it will.

    • Hello Daniel and others — great discussion here prompted by Robert’s thought provoking post. I offer a clarification on the Time to Listen book, its scope and purpose. The book does not prescribe a particular methodology for listening or for participatory evaluation of development results. There are plenty of such texts and Robert has written some of these back in the early 80s. Time to Listen book captures the experiences and feedback of people on the receiving end of (different forms of) international aid, not just humanitarian. People we listened to in 20 countries reflected on decades of assistance (e.g. Ethiopia) and more recent experiences (e.g. Aceh) and overwhelmingly expressed that international aid is a good thing and is appreciated. They also, consistently and overwhelmingly, called for a major change to the way aid is conceived, “delivered” and evaluated. The book calls for a shift from externally driven aid system to a more collaborative aid system. Listening is only one step towards this shift. The change requires attitudinal, structural and systemic shifts in incentives, priorities, agendas, procedures, and working methods. We would not have had to write Time to Listen if the manual published by the World Bank in 1987 that you mentioned (and others) were fully put to use and taken up as standard operating procedure. We hope this will change. Thank you for reading it and for working on this important issue!

      • Daniel says:

        Dear Isabelle,

        Quite right: the issue is not much why and how to; rather why those shifts are not happening. My point was that books and articles appear to have little bearing on those who are responsible to make those shifts happen. It would be useful for books like yours to mention Lawrence salmen’s work around the time of Mr. Chamber’s in the early 1980’s if only to explain the reasons why it did not become standard practice. In this respect, I very much hope your book succeeds.

  6. […] Extraordinary broadside from Robert Chambers accusing DFID of losing the development plot, and Payment by Results as proof. (Haven’t seen any responses yet – let me know if I’ve missed them) […]

  7. Annie says:

    Absolutely spot on.

  8. James says:

    Of course PBR is not just about DFID’s relationship with NGOs. Many organisations that provide development services are private businesses motivated by something other than social impact or development results. To me, it seems a good thing for DFID to use a mechanism like PBR because it helps to ensure that such partners will deliver what they say they are going to deliver. Perhaps the key point is that you need different tools to hold different types of partner to account.

  9. Daniel says:

    Dear Rob,

    You mean that NGOs, in their relationship with DFID, are solely interested in social impact or development results (and not “something other” like private businesses)? Or did you just mean that both should be treated equally by DFID in this arrangement?

    Aside from this, I think Robert is right to make the point about how NGOs or consulting outfits being paid to deliver against pre-determined types and amounts of support that may or may not be relevant to those is riddled with problems. (That is even assuming DFID can differentiate between outputs and the consequences of these ‘results’, the outcomes, in defining results.) All aid interventions are theories incarnate are they not?

    My view? NGOs and consulting companies should be paid on the basis of assessments made by those it supports (its clients) and not by the donors (the shareholder).

  10. Pete Vowles says:

    Hi Robert,

    It is difficult reading. However much I might disagree, it is important that we read and reflect. Personally, I found your books in the 80s and 90s really inspiring, which makes your blog even more challenging to read.

    You are right that development assistance is getting more complex, scrutiny is increasing, and accountability to tax payers has (rightly) increased. Over the past few years we have been working to strengthen our impact on global poverty, through adding rigour into the design process, better use of evidence, focusing on options that achieve the best value for money and focusing on delivery.

    Over the past year, our reform programme (https://dfid.blog.gov.uk/2013/10/21/adaptive-programming/) has been about the very things you set out: creating the space for staff on the front line to use professional judgement, adapt and flex according to different contexts and be open and honest about when things go wrong, fix and learn from them. You clearly don’t think we are there yet?

    In response to your 3 recommendations:

    1. You say we should radically simplify and streamline your internal procedures and free up staff time: The Smart Rules, published in June (https://www.gov.uk/government/publications/dfid-smart-rules-better-programme-delivery) provide a clear coherent operating framework to give people the space to flex and adapt. In publishing this we stripped out old rules and replaced them with 37. I’d welcome views on whether these go far enough? We are planning meetings at BOND and with contractors in coming weeks to hear how this is going.

    2. You say we should recruit many more staff with grassroots field experience and who understand development realities. I am not sure DFID staff have less than in the past or that they have less development experience – between 2006-07 and 2012-13 DFID’s overall staff complement rose by 222, largely consisting of technical specialists.

    3. You ask DFID to put PbR on hold until a few examples can be evaluated in depth over time: We are clear that PBR is one of a range of tools we have and we will continue to experiment and learn about how PBR can be used to provide the space to innovate, adjust and adapt. Our PbR strategy commits us to: “Establishing rigorous, independent, and comparable evaluations of Payment by Results.” Our staff guidance strongly recommends that all PbR projects should be independently evaluated and we are clear that PbR will not be feasible in all circumstances.

    4. Finally, you suggest we should purchase of 2000 copies of Time to Listen and send them to all DFID staff. Even better, we have circulated the link for those who missed it last year, and it is free to download so no need to purchase!

    I would be happy to discuss what more you think DFID can do as we rely on expert challenge and critique, however challenging, and by listening we hope we can continue to improve the way we work and the impact we have.

  11. Glen Fisher says:

    Reblogged this on Canada Diaries and commented:
    This is such an interesting blog, and certainly finds an echo in my (admittedly limited) experience, that I really had to share it….

  12. It will be interesting to see how the Smart Rules filter down and get applied. Like most organisations, DFID is not homogenous in views and approaches; just compare the (frighteningly wide-eyed) PbR strategy to the more measured Clist/ Dercon 12 Principles and this week’s Guidance on PbR for evidence of that.

    When you combine some uncertainty about DFID’s views with (a) the often distant relationships that exist between DFID and its suppliers/ grantees (often now mediated by fund managers with their own interpretations of rules and good practice) and (b) Mansoor’s spot-on point about differences in how NGOs choose to respond and make use of available wriggle-room, you can get a range of experiences from the “dysfunctional demoralising nightmare”, to ticking donor boxes while getting on with what you believe is right, through to a happy working relationship with common views.

    Better engagement, relationships and communication seem key, but I doubt many NGOs would feel things are moving in the right direction with DFID on that front (given pressures on staff time, use of fund managers, etc.).

  13. idsppsc says:

    Hi Pete,

    Thank you for the spirit of your comment and the sources you give. Apologies for my delayed response. I have been travelling and hope to consult them when I am back.

    I do understand, and regret, that my blog was painful to you, and probably to others in DFID. My hope was to help not hurt. I have gone on record in the past about my respect for what DFID and its predecessors have achieved. I admire and applaud the work of many of those I know and have known in DFID. And my dream is that UK Aid will make a major contribution to a better world especially post-2015 and be something we can all be proud of.

    Let me make two points.

    First, on technical staff numbers, it is not just total numbers, but the development staff to total budget ratio. What has happened to this over the years? Beyond this, various factors have multiplied and intensified to trap staff in capital cities and distance them from the field, from projects, from realities, from people – and I believe this has been a point made by the independent evaluation of UK Aid. To me this is a tragedy, diminishing the cost-effectiveness of our tax payers’ money.

    Second, on applications of PbR, as I said, I hope there will be a moratorium on all future applications until there have been extended evaluations which go deep into the effects on many actors in many places – only then will they be rigorous. It is good that evaluations are planned. But in any case pending the outcome of such evaluations, there are two areas in which PbR would be quite exceptionally damaging, where it would be deeply dysfunctional – NGOs and research.

    NGOs are dealing with unpredictable environments. They need to be flexible and adaptive. The first PPAs gave those selected valuable flexibility. Because of its flexibility, money was worth more than its face value. But with time the money was less useful (and in my view less value for taxpayers) because of higher transaction costs and tighter conditions (all in the name of accountability and of value for money). It would be wonderful to revive those earlier arrangements and relationships of trust.

    In the case of research, we embark on it because we do not know what the results will be or usually where it will take us. If anything has to be measured, what? Journal articles? Participation in events? These may or may not be significant. Counting them would be largely meaningless. Most of the time you cant predict with any assurance what will make a difference. My life experience of research has been exploration, and the surprises and the unanticipated. Of ahha!s. And of needing to change direction. Had I had to specify results in advance I would quickly have been frustrated and hamstrung and unable to follow up on promising leads. Much good research simply isn’t predictable. So I appeal to DFID, earnestly, not to hobble it. We need it to run and to run freely. Pick good organisations, good people, good projects, and give them freedom. Trust pays.

    Regards, Robert

  14. […] Perverse Payment by Results: Frogs in a Pot and Straitjackets for Obstacle Courses Robert Chambers strait up nipping the Payment by Results (PbR) development fad in the bud. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: