REGIONAL SEMINAR INTRODUCTION TO THE IMPACT EVALUATION OF PUBLIC POLICIES AND PROGRAMMES
August  23 to 25, 2017

BACKGROUND

Assessment is an essential component of accountability in a democratic State and an imperative of modern public administration. Assessment exercises are aimed at determining if the purposes of government intervention are being achieved and at evaluating the adequacy of mechanisms and instruments for the resolution of a public problem. Insofar as any public policy implies a causal theory or model of social transformation, the burden of proof on the effectiveness and impact of programmes shall lie, in the first instance, with the public authority, which is forced to provide evidence and channels for citizens to know, question and validate the Government’s work. While impact evaluation is related to scientific criteria for testing hypotheses (expected impact of the programme), as any assessment exercise, it is carried out in a political and organizational context that determines its link with decision-making processes, continuous improvement of programmes and the ability of citizens to demand public policies that add value to citizens and maximize the rate of return on public investment. Unlike clinical analyses, impact evaluation operates in complex political and social contexts, which pose unique challenges for the rigorous and methodologically sophisticated identification of outcomes of interventions. For this reason, a “good” impact evaluation is expected not only to respond to the most robust methodological standards, but also to make a stated commitment to public ethics and provide a clear guidance on the use. Assessment is expected to be transparent and methodologically rigorous, but also to make its intention clear (what and why to assess, what the evaluation will be used for and what results are expected). It should be timely, accessible to decision makers and, finally, applicable.

The Ibero-American Charter of Quality in Public Administration stated two fundamental pillars for quality in public administration: 

i) Public administration must be aimed at the satisfaction of citizens, either as a user or a beneficiary of public programmes and services, or as a legitimate participant in the process of formulation, implementation and control of policies under the principle of social co-responsibility; and ii) public administration must be based on results, and therefore it should be subject to different controls, assuming, among other modalities, accountability for the exercise of public authority through social control and regular accountability (2008: 21-26). 

According to the Ibero-American Charter provisions and the urgency to build a public administration capable of facing the new challenges of society – which, while ensuring the compliance with social and economic contracts, should be a guarantor of social rights – the evaluation exercise becomes especially relevant. This is due to the fact that the proper functioning of policies requires learning from mistakes and successes in different experiences, which can only be identified through the measurement and review of public policy instruments. Moreover, a quality public administration requires a permanent, internal and external assessment, based on the principle of efficiency. In other words, it must be focused on results and must create monitoring systems that verify the proper development of public programmes. 

A results-based public administration is unthinkable without the generation of credible and reliable evidence and information on the social change/effect strictly attributable to the presence of programmes: the counterfactual. Only from an impact evaluation, it is possible to determine what would have happened to beneficiaries of a programme if it had not existed. 

In this context, a robust capacity-building mechanism for the use of impact evaluations as an instrument for accountability and as a tool for continuous improvement of programmes involves a series of different training needs that are based on the technical and methodological knowledge, but not limited to it. For impact evaluations to be demanded and properly used, decision-makers, without being experts in measuring impact, must have sufficient elements to i) identify the type of response offered by this type of assessment and its limitations; ii) know the prerequisites of management and information imposed by this type of evaluation; iii) perform a cost-benefit analysis of this type of evaluation compared to its less costly alternatives. Once the decision to evaluate has been made, decision-makers should iv) assess the soundness of the methodological proposal and the validity of conclusions arising from it.