Please note that this research was conducted before gas prices increased at the end of 2021.The analysis is based on energy prices and installation costs at the time the research was undertaken, in late 2020.

The Scottish Government intends to develop new building standards to ensure all new homes use zero emissions heating at the point of use from 2024. Similar requirements are also due to be phased in for non-domestic buildings.  

This report looks at the costs of delivering zero emissions heating in domestic and (as far as possible) non-domestic new buildings. It identifies the factors that influence these costs and how they are split between different actors, including building developers, building owners and building users over the lifetime of a technology. 

We used a literature review and stakeholder interviews to inform a cost analysis model, which was used to analyse six new build scenarios: Scenario 1: Private housing development; Scenario 2: Mixed-use build-to-rent development; Scenario 3: Social housing development; Scenario 4: Small-scale rural development; Scenario 5: Student accommodation; Scenario 6: Primary school.

The cost analysis considered six zero emissions heating technology options within the cost analysis: air source heat pumps (ASHPs); ground source heat pumps (GSHPs); on-demand direct electric heating (dry system); direct electric heating (wet radiator system); new district heating network; and connection to an existing district heating network. It also considered building-level solar PV as an additional electricity source to feed into the selected electric heating system.

Key findings

Cost analysis

  • In all six scenarios, the use of zero emissions heating technology options represented lifetime cost increases ranging from 25%-231% compared to the equivalent lifetime cost of heat supply using gas boilers.
  • There is a significant difference in the cost optimum zero emissions heating solution, depending on whether it is considered in terms of capital expenditure (CAPEX), electricity running costs or lifetime costs.  
  • Individual ASHPs appeared cost optimum on a lifetime cost basis in the scenarios with less dense developments. Lifetime costs were significantly lower in the scenarios where it was assumed that new developments could connect to an existing district heating network. A new district heating network also appeared cost optimum in the high-density mixed-use development.
  • Since grid constraint costs were excluded from the analysis, wet and dry electric heating options offered a significantly lower capital cost, but with higher electricity running costs. 

Stakeholder analysis

  • The stakeholder interviews highlighted how the choice of which zero emissions heating technology to use in developments was driven by more than just cost considerations. Commercial delivery models and the role that a developer played in a development after construction  were also key factors.
  • Delivering zero emissions heating was perceived as a significant change in existing development processes for some interviewees; design and delivery processes were still being optimised and refined. There was greater evidence of innovation in the social housing sector. 
  • This study highlights a potential gap in the sector for energy service organisations to deliver technology options with higher capital costs but lower running costs (i.e. optimising use of lowest lifetime cost).  

Ruth Bush and Niall Kerr, ClimateXChange Research Fellows looking at retrofit policy in Scotland, set out lessons on how evaluation can play a role in helping Scotland to meet its climate targets.

The Energy Efficient Scotland Route Map was launched in May 2018, setting out the Scottish Government’s plans for large-scale energy efficient retrofit of Scotland’s building stock alongside heat decarbonisation. Chapter 7 of the route map details plans to establish a monitoring and evaluation framework that will demonstrate the programme’s progress and “will allow [Scottish Government] to adapt and flex the programme where necessary”.

As ClimateXChange Research Fellows we use different types of evaluation to inform energy efficiency policy in Scotland. Experiences in our projects offer lessons about how to ensure evaluation can play a meaningful role in helping Scotland to meet its climate targets in the future.

At the European Environmental Evaluators Forum (EEEN) conference at the end of 2017 we came together with environmental agencies, consultancies and academic colleagues from across Europe to hear how other countries are using evaluation in policy development and to discuss Scotland’s approach.

From left to right: University of Edinburgh researchers Ruth Bush and Niall Kerr, Jack Causley from Scottish Government and Mark Winskel a Chancellor’s Fellow at UoE and the session chair at the EEEN. 

Evaluation approaches

Niall Kerr’s research:

  • Evidence use in energy policy-making: decision making under social, technological and political uncertainty
  • Systematic evidence reviews: systematically and transparently gathering potentially contradictory evidence
  • Multiple criteria for policy effectiveness: efficient, equity and institutional fit
  • Dedicated method of energy policy effectiveness in Scotland
  • Pilot evidence review findings: encouraging households to invest in home energy efficient retrofit

Ruth Bush’s research:

  • Learning lessons from local pilot projects, led by local authorities delivering energy efficiency retrofit in domestic and non-domestic buildings
  • Using social surveys: To understand investment drivers and barriers in different contexts
  • In-depth interviews: to understand programme design and management approaches
  • Building capacity amongst local authorities – Sharing lessons from local pilots with other local authorities and Scottish Government

The importance of embedding a culture of evaluation into policy and delivery.

As retrofit programmes start to focus on some of the more challenging areas of energy retrofit like private non-domestic buildings, or moving away from gas central heating, we’re not going to get things right first time.

Embedding this type of learning culture takes time. Starting this process of embedding culture of evaluation at national and local levels will help us to innovate and learn from when things go wrong.

In the local SEEP pilots, many of the pilot leads had little or no experience of evaluation. They were in charge of coordinating technical monitoring and survey data collection within their projects, and we did training and workshops to develop a way of working that would deliver good quality and consistent data that would allow us to compare across the projects as well as draw local lessons.

The pilot leads now have a much more developed understanding of evaluation; not only about how to make the practicalities of data collection work, but also about what they can learn to help their own work.

Whilst the process of national policy decision-making may be unpredictable, many of the findings from a systematic evidence review are likely to have relevance over medium to long-term time periods. Reviewing multiple sources of evidence on a particular topic in energy policy entails summarising the most relevant details often from a mix of qualitative and quantitative sources. Inevitably some detail from the reviewed sources will be excluded. Whilst seeking to offer insight by identifying trends in the assembled evidence our reviews will also act as something of a reference document for those interested in the details of the policy question.

For readers that have a particular interest in a sub-section of the review the gathered evidence will allow an initial introduction to a potentially much more comprehensive evidence base. Building a consolidated knowledge base on the particularly complex issues of energy system transition is critical so that lessons are learned from historical and international policies.

The importance of the timing of evaluation?

Policy processes don’t wait for pilot projects to finish and outcomes of an evaluation to be published. It was important for our evaluation team to create opportunities to feedback what we were learning to Scottish Government and local authorities on the local pilots on an on-going basis. We did that formally through interim reports, but also held workshops with everyone involved and facilitated discussions on key emerging themes from our data. National policy makers could get a more in-depth insight into how delivery was working at the local level, and they also provided an opportunity for local authorities and their partners to exchange experiences and learn from one another.

An on-going relationship is important to making sure the evaluation is contributing to delivery and policy processes – no matter the speed of policy development.

 Systematic evidence reviews do involve a trade-off between the rigour of evidence examination and the length of time taken; more time means more evidence gathering and more consideration of potentially complex and contradictory findings. Policy decisions are, however, often made in an unpredictable and dynamic environment. This mismatch has led to the use of Rapid Evidence Assessments (REA) for energy policy. REAs are designed to apply a comprehensive and un-biased approach to evidence gathering, and are intended to take place over a relatively brief, roughly 6-month period. Certain policy questions may be suitable to longer reviews of 12 months or more, while shorter scoping reviews of around 1 month can be applied to more urgent issues. Our project intends to trial different lengths of evidence review throughout the energy policy effectiveness project.

Presentations and more information about the EEEN Forum 2017 can be found on the European Environment Agency Website: https://www.eea.europa.eu/themes/policy/events/eeen2017/6th-european-environmental-evaluators-network-forum

Ruth Bush, Research Fellow CXC, University of Edinburgh and Professor Jan Webb, Heat and the City, University of Edinburgh

Our progress on emissions reductions over the next 10 – 20 years is critical for climate change mitigation. This is reflected in the ambition of the latest Scottish Government Draft Climate Change Bill which sets out draft targets for the domestic and service sectors to reduce emissions by 75% and 98% respectively by 2032.

To meet these targets, delivery programmes will need to radically increase in scale to bring about installation of building fabric energy efficiency measures, behaviour change campaigns, and low carbon heat solutions that will enable us to switch away from fossil fuel based natural gas and oil heating systems.

In this blog, researchers from the University of Edinburgh Heat and the City Research Group share their thoughts on their ClimateXChange research project, evaluating the 25 local authority-led pilot projects funded by Scottish Government to inform the design of Scotland’s Energy Efficiency Programme (SEEP).

The research project aims to bring academic best practice into evaluation of new local pilot delivery programmes, and draw lessons to inform both the local authorities coordinating delivery, and Scottish Government as they develop the national programme.

This blog was originally published as part of the Energy & Society Research Group blogseries on the Scottish Government Energy Strategy consultations in May 2017.

Policy and regulations are going to play a critical role in meeting the ambitious goals set out in the Scottish Government’s draft Climate Change Plan for the next 15 years. But the scale of change needed means we are unlikely to establish an interconnected suite of new regulations, standards, funding, financing, incentives, alongside supportive local and national governance structures, and get it right first time. We need to establish a constructive dialogue between local governments, the public sector, businesses and community organisations, that enables new approaches to be trialled – and sometimes to fail – without derailing the whole programme.

Lessons from the demise of the Green Deal

The politics of being in government mean that experimental approaches to developing new policy measures are susceptible to harsh critiques by opponents, resulting in loss of ‘political capital’ for incumbents and a correspondingly risk averse approach by officials. After the demise of the Green Deal at the UK level, post-mortems have highlighted the challenging political context that meant that “failure was not politically conceivable” (Rosenow & Eyre, 2016). Despite early evidence pointing to issues with the programme design, the political context did not allow space for adjusting problematic areas and subjecting proposals to a ‘reality check’.

By 2015, when the programme was eventually abandoned, the average delivery rate for loft insulation had dropped by 90%, cavity wall insulation by 62%, and solid wall insulation by 57%, compared to 2012 when the programme was launched (Rosenow & Sagar, 2015). This experience highlighted the importance of creating space for constructive dialogue and reflection as policies are developed. Perceived failure of one programme should not mean the end of a whole policy agenda, but present UK Government action for energy efficient buildings is stalling.

Embedding pilots and evaluation within policy development

It is great to see that the Scottish Government are not repeating these mistakes during the development of their ‘cornerstone’ policy – Scotland’s Energy Efficiency Programme(SEEP). They have already embedded a process for learning into policy development through supporting local pilot projects and embedding evaluation as an integral part of the projects and on-going delivery.

As the academic research team supporting the evaluation of the local SEEP pilots, we have a responsibility to contribute to establishing a constructive dialogue around the national development of SEEP. Research evaluation of such programmes is potentially influential in determining what is understood to be ‘successful’ more generally. However, choosing the criteria for robust evaluation is challenging given the diverse geographies and socio-economic circumstances within which programmes take place, alongside the differing contexts of skills and supply chains.

Collaborative evaluation can support lessons for delivery as well as policy development

There are currently 25 local authority-led pilots taking place across Scotland. As an evaluation team, we work collaboratively with both the local authority pilot leads and Scottish Government officials to understand their objectives and concerns during pilot design and delivery, and to determine the important dimensions of the local project from the local and national perspectives. This approach has highlighted common challenges and best practice, as well as areas where there is a need to gain more experience, such as approaches to targeting the commercial sector, or mixed-ownership tenement buildings.

As the number of pilot projects grows, local authorities are taking on an increasingly proactive role in shaping the evaluation process. This ensures that the evaluation speaks to the local context and plays a development role to support future programme delivery, as well as informing national policy design.

The hardest delivery challenges require political space for experimentation, and even failure

This use of evaluation within policy development is one way to provide a framework for constructive dialogue around the hardest delivery challenges we face in SEEP. It creates a formalised and evidence based way for local experiences to inform national policy design – rather than a blame culture where failed projects are seen only as a result of poor management and a waste of squeezed public resources.

But this won’t be enough on its own to create the political space for tackling the hardest delivery challenges for SEEP. Programmes that take on these hard issues will need political back-up from local and national politicians and senior officials, they will need time and space to develop and refine approaches, and they will need flexible resource allocation that can be shifted around different tasks and over time. A strong working relationship based upon trust and respect between national and local governments is critical here. We hope that this culture can continue to build over time to empower the officials and politicians supporting it to tackle the hard challenges posed for SEEP.

The SEEP pilot evaluation interim report (published May 2017) is now available on the Scottish Government’s website

Read more about the SEEP pilot evaluation project