Earth Compress will use AI to compress and unlock the world’s largest environmental datasets.
The world’s environmental satellites are generating data faster than science can use it. The Sentinel-2 Earth observation archive alone stands at 40 petabytes, which is enough storage for 8 million hours of HD movies, yet only a fraction of a percent has ever been processed. The problem is not the data itself, but the cost and complexity of accessing it.
A team led by the National Oceanography Centre (NOC), in partnership with deep-tech worker-cooperative Asterisk Labs, has been awarded £7 million by the Advanced Research and Invention Agency (ARIA) to tackle this challenge head-on. The project, called Earth Compress, will develop open-source AI infrastructure to compress, distribute and democratise access to massive environmental datasets, from ocean observations to atmospheric data, enabling novel research and a new generation of services that are currently out of reach for all but the best-resourced institutions.
Earth Compress grew out of Clouds Decoded, a £500,000 ARIA-funded seed project led by Asterisk Labs that set out to harness millions of cloud-obscured satellite images previously dismissed as unusable. The team rapidly proved that this ‘waste’ data held genuine scientific value, but the project’s impact was hampered by a long-standing, fundamental barrier in environmental data-science: the outputs were too large and costly to share openly. Rather than a project-by-project fix, they developed a bigger ambition: to solve the data accessibility problem at infrastructure level, once and for all.
The core idea is ‘petabytes of environmental data in your pocket’. Earth Compress will create AI software tools and infrastructure that enable any user, from a student on a laptop to a national meteorological agency, to work with entire environmental archives that currently require enormous computing resources and budgets. Processing one petabyte of earth observation data in a commercial cloud can easily cost over £1 million. Earth Compress aims to reduce this to under £1,000 for storage and between £10,000 and £100,000 for compute, figures that are much more feasible for smaller organisations and publicly-funded research.
Being part of the National Oceanography Centre and having previously led large projects for other world-leading environmental organisations and big technology companies, the potential for impact was clear to me, and the benefits of joining forces with Asterisk Labs were clear too. We not only shared a worldview but had complementary skills and experiences across data management, software engineering, AI, community building, and user-focused research.
Alberto Arribas Herranz, Head of Software Engineering and AI, NOC
Earth Compress will work across the full data chain, empowering data producers to create traceable, reproducible compressed versions of their datasets, and encouraging data users to build high-impact research and services from them. The entire stack will be open-source and co-designed with key partners including the European Centre for Medium-Range Weather Forecasts (ECMWF), EUMETSAT (whose Directors General have provided letters of support) and the Natural Environment Research Council (part of UK Research and Innovation), ensuring the technology is publicly owned, trustworthy and scalable across institutions.
For NOC, Earth Compress represents a significant step forward in its role as a national leader in digital environmental science. As the UK’s national centre for ocean science, NOC generates and relies upon vast quantities of observational data, from deep-sea autonomous vehicles and research ships to sustained oceanographic monitoring networks. Making this data more accessible and usable is central to NOC’s 2025–2035 strategy and to the broader ambition of translating world-class environmental research into real-world impact.
Jacqueline Campbell, Co-Founder, Asterisk Labs, commented: “The insight was that solving data accessibility at the infrastructure level, rather than on an application-by-application basis, would unlock not just our own research but an entire class of currently infeasible environmental science. We are genuinely excited to be developing this alongside NOC and our wider partners, to ensure it is publicly owned and openly accessible from day one.”
The ambition extends beyond the project itself. Earth Compress is designed to catalyse a wider ecosystem, accelerating the development of start-ups and public-private services built on environmental data, and deepening the relationship between UK and European scientific institutions. The longer-term vision is for the AI software infrastructure developed by Earth Compress to be managed and sustained by a multi-organisation foundation spanning UK and European institutions, building shared, publicly-owned capability for decades to come.
While environmental data volumes continue to grow, our supporting infrastructure has lagged behind. Earth Compress is aiming to solve this bottleneck and have secured the backing of key data producers and users across the sector. If successful, this work won’t just slash data management costs — it could fundamentally change the very questions scientists can ask about our planet.
Dan Giles, Science and Technology Lead for ARIA’s Forecasting Tipping Points project
Jon Blower, Director of Digital Science, NOC, comments: "Artificial intelligence is transforming what is possible in environmental science, but only if the data those models need can actually be reached and used. Earth Compress tackles that challenge at its root, by creating the shared infrastructure that will power a generation of AI-enabled research and services. NOC's leadership of this project reflects our commitment to being at the forefront of digital innovation for the benefit of the UK and the world."
Earth Compress is led by the National Oceanography Centre (NOC) in partnership with Asterisk Labs, and funded by the Advanced Research and Invention Agency (ARIA). Strategic partners and early adopters include ECMWF, EUMETSAT, and NERC Environmental Data Services centres.
Be part of the community helping our ocean thrive.
