Democratic evaluation for the 21st Century

Guest blogger Robert Picciotto, from King’s College, London explores whether the democratic evaluation model from half a century ago is still fit for purpose in a world of rapid internationalization, economic inequality and social fragmentation.

The International Year of Evaluation (2015) is based on the notion that evaluation is a tool of good governance. According to its sponsors “evaluation can enable leaders and civil society to develop and support better policies, implement them more effectively, safeguard the lives of people, and promote well-being for all”

The underlying idea is that evaluation contributes to (and benefits from) an enabling environment where authority is accountable to citizens and valid evidence is sought to guide decision making, i.e. that evaluation favours democracy and democracy favours evaluation. Yet according to Ernest House “the capture of evaluation by its sponsors is the greatest threat the evaluation community has faced for some time. In fact, the credibility of the field is at risk”.

What then is to be done in a world where democracy is absent or in retreat? Are current evaluation approaches adapted to the challenges of rapid internationalization, economic inequality and social fragmentation? Is the traditional democratic evaluation model designed half a century ago still fit for purpose?

The evaluation models in widespread use today leave a lot to be desired. To be sure ethical commitment to freedom, equality and democracy is central to the vision of such eminent evaluation thinkers as Michael Scriven, Jennifer Greene, Karen Kirkhart, and Donna Mertens. But in practice democratic evaluation that was pioneered decades ago by Barry MacDonald in the United Kingdom and by Ernest House and Ken Howe in the United States is hardly practiced today. It needs renewal.

Both for MacDonald and for House/Howe expert facilitation was the main way to help ensure that evaluation is effective. However the effectiveness of a scrupulously neutral approach hinges in large part on the quality of the governance framework. It falls short in environments that do not tolerate dissent or for evaluation assignments that are closely controlled by decision makers. Where evaluation governance has been captured by vested interests it is simply not enough to provide neutral information services, broker debate and facilitate deliberative decision-making processes. 

Looking at the evaluation landscape today it is hard to avoid the conclusion that at one end of the spectrum utilization oriented doctrines dominate the discipline. The resulting evaluation products are constrained by a repressive architecture of evaluation protocols and terms of reference. When the pursuit of knowledge acquisition is the main goal, the evaluation approach is akin to a value free scientific approach so that such questions as what knowledge is being sought, why and who benefits from it are rarely asked. At the other end of the spectrum, critical evaluation approaches lack leverage because they stand aloof from decision making structures and processes which they view as immune to change short of a comprehensive revolution in world views and mindsets.

It has therefore become necessary to design and promote an alternative evaluation model (independent democratic evaluation) that embraces the vision of evaluation as a morally engaged, value driven occupation. It would be tailor made to diverse governance environments. It would assert its autonomy in shaping evaluation agendas and selecting evaluation methods. It would privilege the public interest, emphasize analytical integrity and involve fulsome engagement with citizens. This activist and independent approach would be grounded in professional autonomy. It would be reliant on independent funding sources and break the chains of fee dependence. It could even pave the way for a democratic wave of evaluation diffusion in the 21st century in reaction to the currently dominant market based model.

This blog is based on a recent Sage publication: R. Picciotto (2015) Democratic evaluation for the 21st century Evaluation April 2015 21: 150-166


Views expressed by guest bloggers may not reflect the views of What Works Scotland.