Batteries and big data: developing shared storage and processing tools
26/05/21
< Project overview >
Introduction
Performance and lifetime testing of batteries requires considerable effort and expensive specialist equipment. A wide range of potentiostats and battery testers are available, but there is no standardisation of data exchange and storage between them.
Discussions between Oxford, Newcastle and Sheffield Universities identified the barrier this creates for battery researchers, including issues with data security and collaboration with partner organisations. Working with engineers at the Oxford Robotics Institute and Department of Computer Science, we developed Galvanalyser, a battery test database developed to manage the growing challenges of collating, managing and accessing data produced by multiple different battery testers.
Further improvements to the user experience have been made in collaboration with the international modelling community originating from the Oxford Battery Modelling Symposium.
Project aims
The project addressed the difficulties associated with managing increasingly large amounts of data in the battery research community.
We aimed to automate the labour-intensive processes involved with data de-cluttering and management, as well as standardise procedures. Resource access, including security, distributed hierarchies to manage IP issues and knowledge about the best methods to leverage IoT data, are major barriers to research and development in the energy sector.
This project aims to remove these barriers, facilitate research and collaboration between HEIs and industry, and speed up battery design and development. Ultimate beneficiaries of these developments include users of future electric vehicles and clean energy systems utilising battery storage.
What was done
The project originated from discussions between principal investigators at Oxford, Newcastle and Sheffield Universities on the barriers faced by the community in relation to the IoT, where data de-cluttering and management was identified as a major obstacle. Researchers at Oxford investigated the different analytical tools and software solutions available to tackle this problem.
Subsequently Professor Howey’s Battery Intelligence Lab worked with engineers at the Oxford Robotics Institute to build the ‘Harvester’ – software to parse data from battery testers – plus a PostregSQL database and web application. Working with the IT infrastructure team at Oxford’s Department of Engineering Science, a server was built and housed by the Oxford e-Research Centre.
Further work with research software engineers at the Department of Computer Science developed the Galvanalyser tool’s functionality; tightening security, facilitating ease of use and looking at the management of metadata. Finally, improvements to usability were begun following user feedback and dissemination amongst the battery modelling community.
Results
We have delivered a validated tool for the processing and storing of large data sets produced by different battery testers and potentiostats.
‘Galvanalyser’ is a database solution, which automatically collects data from various separate battery cycling equipment, parses it into a common format, then stores data in a single PostgreSQL database with suitable user access privileges. This allows for easy backups to be made and allows users to conveniently access data through a web application or directly using an API. As a result, data may be exchanged and analysed more easily, speeding up battery design and development work.
Galvanalyser has already received wide interest from members of the international battery modelling community.
Deliverables and other tangible outputs
We delivered on three core measures of success:
1. Delivery of validated tools/code for the processing/storing of large data sets.
Galvanalyser processes and stores large data sets from different battery testers and potentiostats. It consists of two components: a server, and a client. The Harvester is the client-side application, which scans selected file-system locations for changes and uploads these to the server.
The server itself has 3 parts:
The PostgreSQL database stores and manages data.
The web application enables querying and plotting of data.
The Nginx server handles the connection between browser clients and the webapp.
2. Demonstration of storage, access and processing of a battery data set in line with best practice in other sectors.
Galvanalyser is based on the popular PostgreSQL database system which is widely used across many sectors.
3. Publish an example of data storage and processing.
An introduction to Galvanalyser, together with an illustrative example has been published by the open access e-print repository service ArXiv, arXiv:2010.14959. Additionally, Galvanalyser has been discussed and reviewed by partners in the international battery modelling community with over 600 battery researchers, and Professor Howey is currently working on a high profile perspective article focused on a ‘Battery Data Genome’ together with a number of authors from the US and Germany.
As a stretch goal, we aimed for publication of common standards applied to Oxford and HEI partners for dealing with big battery data sets which can be advertised to others.
Following feedback, we are implementing improved features to improve the user experience, for example, ease of installation, user documentation, security, and metadata. This will facilitate wider sharing of data sets and common standards within the community.
Impact
As the commercial success of batteries continues to increase, the value of improving our understanding of these complex devices and accelerating efforts to learn from data goes up.
Galvanalyser solves growing data management issues by storing data from disparate sources in a standardised searchable format, ensuring data security and facilitating collaboration between research organisations and industry. By facilitating the transfer of knowledge from the battery modelling community via industry to the technology powering electric vehicles and grid-connected energy storage, the project contributes to improving access to clean, secure energy for users in the UK and abroad.
Next steps
Publication of the high-profile perspective article focused on a ‘Battery Data Genome’, a collaboration with a number of authors from the US and Germany.
The next steps to develop Galvanalyser focus on software developments to improve the user experience and dissemination amongst the wider community. The aim is to publish common standards for dealing with big data and a library of available data sets from HEIs, industry and open sources.
Examples of the software improvements include support for the customised function of the Harvester on different battery testing systems, improvements to the Python API, and the addition of an admin dashboard.
Lessons learned
What went well?
Extensive discussions leading to a common understanding of the issues around managing battery test data.
Development of a database solution for this that is in the process of being rolled out to beta testers.
What could have been done better?
Started with a more modest initial aim (eg realised that an extensive GUI was not necessary, and modified our aims accordingly).
Engaged a broader team of software engineers earlier in the process.
Access to deliverables, resources and content
Article – Galvanalyser: a battery test database
What has Pitch-In done for you
The ‘Batteries and big data’ Pitch-In project has enabled us to dedicate time and resource to understanding the issues around the management of big data within the battery research community; gaining knowledge and understanding, as well as engaging with partners.
We have been able to develop and implement ‘Galvanalyser’, a tool to address the barriers to research and collaboration posed by data management issues, solving these problems in our own lab. It has also enabled us to reach out to international partners in research organisations and industry to engage on these issues, discuss best practice and ultimately facilitate collaborative working and improved outcomes for battery design and development.
Project lead
Professor David Howey – Associate Professor in Engineering Science at the University of Oxford