Open Energy will modernise access to energy data across industry and address decarbonisation and the climate crisis via economic innovation.
This groundbreaking, forward-looking project addresses the organisational silos that work against information sharing. We have validated the need for Open Energy: to agree shared principles and practice to underpin our data infrastructure. Stakeholders repeatedly emphasised that there cannot be a ‘single platform’ in which ‘all data is put’ to address ‘all use cases’.
Our proposal tackles multiple aspects of the MEDA challenge and the articulated user needs. To deliver taxpayer value-for-money, address public sector needs and catalyse innovation in the SME sector, two programmes of work will build upon and reuse existing, proven processes and systems that enable data-sharing in a parallel sector:
- Stakeholder engagement
A three-month programme convening industry & the public sector to deliver a minimum viable Open Energy Standard based on needs. This will join existing work and best practice (such as the EDTF Data Best Practice Guidelines) with practical, testable examples to address data governance issues. - Open Energy Governance Platform (OEGP) prototyping & testing
Develop a prototype OEGP through which market rules can be applied in response to the needs articulated in the MEDA Phase One Discovery. In addition, we will work in concert with data platform providers (e.g. Siemens; Electron) to create active implementations.
OEGP will be underpinned by industry-defined, shared principles and practice agreed through stakeholder working groups. It will provide signposting to access (open and shared) data sets and enable accredited organisations to access the Open Energy ecosystem where accreditation is required to access data. The OEGP will actively monitor the ecosystem deployments and test availability and compliance.
Crucially, this enables many users to access and use the data without data owners having to create a custom integration with each one. It can automate the ‘presumed open’ ambition and augment it with preemptive licensing where shared data is needed.
OEGP will provide an integrated solution. Rather than a ‘data hosting platform’ into which organisations must continuously send data, the assets themselves (or organisations which manage them) should ‘enable the discovery, access, collection and use’ of the information they hold. This enables third-parties to seek and collect the information needed when they need it. It enables the asset owner to retain control over access to the data and ensures data is never stale. To ensure interoperability, data sources’ compliance with existing and evolving standards will be instrumented.
Building on the Open Banking Standard, the Open Data Institute and industry initiatives we will aggregate, repurpose and redeploy proven processes, practices and technologies which have received £millions of investment from industry and the taxpayer. Further, we will redeploy parallel work being led by Icebreaker One, funded by industry and UKRI. As with Open Banking this has the potential to position the UK as world-leaders for energy data.
Following engagement with ~200 stakeholders including the ONS, the ENA, Electralink—all validated our proposed approach. Existing MED initiatives (e.g. ENA Data Working Group) also expressed a need to connect the disparate pieces and that Open Energy is perfectly positioned to deliver. We propose a clear roadmap to transition from a fragmented data landscape to a robust, decentralised, federated data infrastructure.
The architectural approach to enabling domain-specific platforms, hubs, analytic networks, asset registries, catalogues, systems maps, etc. requires a shift in thinking from ‘push’ to ‘pull’. This will allow use of state-of-the-art data science techniques that drive £multi-billion savings and rapid decarbonisation.
Technical innovation
Energy data is highly diverse and is evolving too rapidly for a central, proprietary IT system to keep up. However, decentralised data and asset search is challenging and existing datasets are not strongly linked.
The critical innovation required is an Open Energy Governance Platform (OEGP) enabling a decentralised approach in which data and metadata is distributed, always up-to-date, and managed real-time on data custodians’ servers. Data will be searchable, accessible and available to agreed standards. This platform will provide the common rules, controls and processes needed for access, discovery, security, commercial applications, privacy and regulatory compliance. This proven approach, novel to energy, will form the Common Data Architecture enabling an energy data ecosystem.
This will be developed using the core functionality of the Open Banking Directory, enabling secure, automated, controllable machine to machine access to distributed data, supporting a participant identity service and list of trusted participants, while signposting the required data (API endpoints). It will use standards designed around end-user and market needs, enabling a wide range of solutions, with interoperability between sectors.
Repurposing existing infrastructure in this way substantially de-risks and reduces development cost. While energy datasets are not the same as banking datasets (in energy there is more open data, aggregated data and different consumer privacy issues), the core capabilities necessary are common and transferable. This ensures it is future-ready, able to accommodate complex provisioning chains, will evolve to support GDPR requirements and support future regulatory developments.
This approach will also address the challenge of effective data search and discovery, exceeding the capabilities of an asset register and data catalogue.
Technical deliverables will be:
- A prototype OEGP enabling secure, controllable data access. This will demonstrate a participant identity service, a list of trusted participants and access control to data/metadata (API endpoints). It will demonstrate implementation and integration with other MEDA partners, and enable creation of an end-to-end customer journey for a use case identified—localised resource allocation service using actual energy datasets.
- A prototype search and discovery function that illustrates capabilities including a ‘knowledge graph‘ with the ability to crawl energy metadata for both open and shared data. This approach enables searchable energy datasets and assets, and understands relationships and meaning. Furthermore, it will demonstrate how links can be created between related assets and datasets e.g. metadata about a dataset linking to metadata about the physical asset from which the dataset was recorded.
Workstreams
Two workstreams form the basis of our Phase Two approach which we propose take place in parallel to data platform implementation(s) with other MEDA participants, to
- Enable data publishing in machine-readable formats
- Enable machine-discovery of data
- Enable the ‘pull’ of specific data, subject to governance rules, by specific users for specific needs (e.g. a data platform for specific use-cases)
Workstream 1: Stakeholder engagement
Industry working groups will convene around the following focus areas to ensure stakeholder buy-in:
- User, market and societal needs – specific user needs, the market requirements to meet these, and the societal benefits that are sought;
- Policy, regulation and legal issues – regulatory requirements, participant certification as well as legal issues including data rights, IP and liability associated with the OE Standard;
- Operational requirements – the operational principles, procedures, processes and systems;
- Technical requirements –the technology architecture needed to support this work, including capability, authentication, data search and future API specifications.
Groups will bring together data sharing best practice to develop a minimum viable Open Energy Standard, with worked, testable examples.
Workstream 2: Open Energy Governance Platform (OEGP) and exploration of specific use cases, including search
The infrastructure is in place for a prototype. We will build and implement an OEGP through which a federated data infrastructure can be applied. Focus areas:
- Sandbox & production environment
- Directory implementation
- Automated, controllable access to distributed data
- Machine-certification
- Interoperability with data from outside the energy sector
Directly addressing articulated user needs, we will create a clear roadmap to transition from a fragmented data landscape to a robust, decentralised, federated data infrastructure. This will enable the development of platforms, applications (and potential app-stores) by defining and managing a pragmatic operational framework for cohesive interoperability.