In our series of posts celebrating the unsung heroes of space missions, we’ve been looking at a few of the behind-the-scenes contributions that are rarely talked about. This time, we’re shining the spotlight on the data archives that are the foundation of many of the important discoveries from the European Space Agency’s (ESA’s) science missions.
ESA’s Bruno Merín, Head of the European Space Astronomy Centre (ESAC) Science Data Centre (ESDC) in Spain, and Starion’s Héctor Pérez, Archives and SPICE Software Development and Operations Work Area Manager, give us a guided tour.
The ultimate goal of any space mission is to gather data that can be analysed and turned into useful information. To enable that process, the data needs to be stored and made accessible in useful formats. For ESA’s space science missions, this takes place at the European Space Astronomy Centre (ESAC) Science Data Centre (ESDC) in Spain, where a team of 50 scientists and engineers manage an exponentially growing volume of data.
“There’s a lot of focus on the very visible rockets and the hardware flying in space, which are technological marvels. But there are also technological marvels that we build and manage quietly on the ground – essential to the success of science missions – that might be less known because of their behind-the-scenes nature,” says ESA’s Bruno Merín, Head of the ESDC.
“Our overall remit is software development, operations and maintenance of the science data archives. We start by gathering the scientific community’s requirements and then implement them, but this isn’t as simple as it may sound because from a scientific point of view, different scientific communities use data very differently, with their own standards and tools, and the types of data analysis they do. It’s a lot more complex than people think.” Bruno Merín, ESA
The Archive team supports four scientific domains: astronomy, heliophysics (the study of the Sun, its influence on the solar system, and the space environment it creates), planetary science and, since 2018, human and robotic exploration. The latter includes aspects such as medicine and human biology on the International Space Station and material physics in low gravity. Each domain contains multiple individual archives.
‘Digital Library of the Universe’
“We call the archive ecosystem the ‘Digital Library of the Universe’ because it contains data of many varied missions and scientific domains all in one place.” Starion’s Héctor Pérez, Archives and SPICE Software Development and Operations Work Area Manager
“With these different domains, the team here needs to have a lot of technical knowledge to understand the data and process it, and make it available in a way that scientists can easily find and exploit it. There’s a huge effort going on to homogenise the look and feel of all the archives and web applications,” adds Héctor Pérez. Starion leads the development, operations and maintenance of the archives.
Not only do the archives contain data from a broad variety of missions, but they also keeps data for every single science mission ever launched by ESA. “We have data from missions from the 1970s onwards and in parallel we are preparing archives for missions that will be launched 10 or 20 years in the future,” says Bruno Merín, ESA
This leads to one of the biggest challenges that the team faces, which is the ever-growing volume of data: partly because of additional missions and also because satellite downlinks are improving, gradually removing a traditional limit on the data that can be captured from a spacecraft. One mission in particular is starting to make a huge difference in this respect. The archives currently hold around 2700 terabytes (2.7 petabytes) of data, whereas over the next 5 years, the Euclid mission will boost that to over 20,000 terabytes (20 petabytes).
“This massive increase means we are having to completely redesign our systems, including our data centre,” says ESA’s Bruno Merín. “It’s not just about storing all the extra data but designing it so that it is still failproof. We are about 80% of the way there, though there are still details to resolve.”
Starion’s Héctor Pérez adds: “Another issue is the amount of data that is going to be travelling across our network. Previously users would download a product from the archive and it may not even have been as big as 1 gigabyte. But if they want products for Euclid, they will be way bigger: too big to download to their machines. So it changes what we provide and means we have to change the mindset of users too, who will instead be able to analyse the data in cloud-based platforms.”
Viewing the universe through ESASky
What often goes unrecognised is that the archives are available for anyone to access, including via the graphical ESASky tool. And yet they lie behind an enormous number of scientific discoveries.
A recent analysis showed that between half and three-quarters of the scientific knowledge from ESA’s space missions came not from those who built the instruments flying on individual spacecraft or who requested observations or measurements, but from people accessing the archives, analysing the data and publishing papers showing their findings.
The analysis also shows that the archives remain highly scientifically productive for 20 or more years after the end of mission operations when the original instrument and science operations teams may no longer be operational.
“It’s important that people realise all this data that has been collected over decades is open for them to inspect, download, analyse and play with, if they want. Anyone can use ESASky to see images from missions such as the Hubble and James Webb telescopes, and any other science mission.” Bruno Merín, ESA
“There’s an important difference between hardware and software that most people don’t realise,” Bruno continues. “With hardware, once the human effort has been put in, the value really lies in the hardware itself. But the value of software lies with the people who made it and continue to support it, because the software evolves and grows almost like a living being.
“Rapid technological developments mean the way humans interact with information evolves quickly and we have to keep up with this. For example, 5 years ago, artificial intelligence [AI] was science fiction but now there’s a growing demand for us to understand and respond to the demand for scientists to use AI. And that’s just one example.”
Find out more
This is an extract from the latest issue of OpenSpace magazine. Subscribe to read other in-depth articles on space weather and civil security from space, plus an interview with the Director of the Spanish Space Agency.
Accessing ESASky
ESASky is a science-driven portal providing access to imagery observed by ESA’s space astronomy missions plus those of its partner agencies. Anyone can access ESASky at: sky.esa.int/esasky/
What’s in the ESAC science data archives?
Astronomy observation and surveys
Astronomy observation missions typically point at a target in the far distant sky to capture specific images. Survey missions scan the sky to create a map of a certain feature, such as our universe, the Milky Way (Gaia) or all the features (Euclid). Missions include: Cheops, Euclid, Gaia, Hubble Space Telescope, XMM-Newton, James Webb Space Telescope, Herschel, ISO, Planck, Exosat, Integral and LISA Pathfinder.
Planetary observation
Primarily missions studying planets, their moons and other solar system bodies such as comets. Missions include: BepiColombo, Juice, ExoMars, Mars Express, Rosetta, Huygens, Venus Express.
Heliophysics
Any missions studying the Sun directly or related phenomena such as the solar wind. Missions include: Solar Orbiter, SOHO, Ulysses, Cluster, Proba-2 and Double Star.
Human and robotics exploration
Investigations funded and co-funded by ESA’s Directorate for Human and Robotic Exploration performed on microgravity and ground-based facilities, plus a spectrometer on the International Space Station.
Main image © ESA
