As per CEES objectives, practising energy solidarity should move people towards having access to sufficient, affordable energy that enables ‘human flourishing’.
Putting energy solidarity mechanisms into practice aims to improve the lives of people living in energy poverty. For this reason, it is important to plan – at the outset – ways that ECs can evaluate whether their actions are delivering the intended impacts to targeted communities.
Also, as noted in the series of blogs associated with the CEES Energy Solidarity Toolkit, practising energy solidarity can be relatively easy and low-cost or extremely time- and resource-intensive. Well-planned evaluation can help project managers to understand the impacts relative to the effort that has been put in.
What is evaluation?
Evaluation is a systematic study of a project (or programme, practice, procedure, intervention or initiative). It aims to help assess whether predetermined objectives were achieved and to understand what factors helped or hindered along the way.
As implementing energy solidarity mechanisms is new to many ECs – and may require new skills, different business models and additional sources of financing – being able to demonstrate clear impacts is critical on many levels. Ultimately, comprehensive evaluation strategies can help ECs to constantly develop and improve their processes and projects.
To help ECs get started, this series of blogs provides an overview of options and things to consider, rather than an exhaustive ‘how-to’ methodological text. Noting that energy solidarity projects can take many forms, the approaches described in this blog series may be more relevant to some than to others. Also, many other types of evaluations are valid.
Why evaluate?
Evaluation is an increasingly important tool to support decision making across virtually all sectors of society, including in relation to the just, clean energy transition. When properly planned for from the beginning, the information, data and learning that comes from evaluation can be used by multiple actors in vital ways.
- To constantly improve processes and impacts so that projects can meet their objectives.
- To demonstrate credibility and build trust among target groups and/or potential collaborators.
- To help other ECs implement projects that achieve stated goals more effectively and efficiently.
- To strengthen future bids for funding by showing both clear impacts and the organisation’s capacity to constantly learn, develop and adapt.
- To have solid evidence to include in communications materials and reports.
How to plan for evaluation
Evaluation plans should align closely with a strong project plan, which sets out clear and coherent project objectives. Many project managers (regardless of the fields in which they operate) find it helpful to apply the SMART acronym when establishing project objectives.
Evaluators also often find it useful to construct a Theory of Change or Logic Model. Ideally, this one-page summary will show, in linear-graphical way, how specific actions should lead to the desired change (examples of different formats can be easily found online).
Ideally, the project and evaluation plans will be developed in parallel, with close collaboration among individuals or teams that hold responsibility for different aspects.
Timing is critical to effective evaluation. Thorough planning is needed up front and sufficient time must be allocated at the end – after the project is completed– for data analysis and report writing. Including funds for that time in project budgets is also vital. Many researchers suggest earmarking 10-15% of the total budget.
A strong evaluation plan will start with the fundamental elements of background and context; project objectives; and corresponding evaluation objectives. Building on the stated evaluation objectives, it should then set out a series of evaluation questions and attach key ‘indicators’ to each. In energy solidarity projects, for example, indicators of energy poverty typically include things like not being able to afford energy bills or being too cold in winter or too hot in summer. But other aspects should also be considered, such as associated negative impacts on health and well-being. The plan should also outline the overall evaluation design and specific evaluation methods to be applied.
Finally, a timetable should demonstrate how evaluation aligns with the project itself. In fact, evaluation should be allocated additional time after the formal end of the project. This is vital for final data collection, analysis and reporting. This point is particularly important when preparing timetables in funding bids.
Evaluation carries an inherent challenge, particularly for organisations that are small and have limited budgets – which is the case for many ECs. The desire to establish evaluation processes and procedures to collect the necessary data must be balanced in relation to the burden they place on participating households, the project delivery team and the evaluators.
Click through to read more about basic principles of evaluation