UK astronomers and software developers have created an infrastructure to process and distribute petabytes (PB) of information from a digital camera that is taking images from 1.2 million light years in the past.
Using “the largest digital camera in the world”, the Vera C Rubin Observatory is carrying out an astronomical survey of the entire available southern sky every few nights to create a timelapse movie of the universe. It published its first images earlier this week (23 June 2025).
The observatory, named after the astronomer who first proposed the existence of dark matter in the universe, sits in Northern Chile, but the data it produces is being sent to UK datacentres, where it is processed and made accessible to scientists.
About 20 research software engineers and computational scientists, in various places, are involved in the UK part of the project, which includes cleaning data to make it science-ready, providing access to the data to astronomers and providing them with alerts.
By the end of the 10-year survey, the Legacy Survey of Space and Time (LSST) project will have produced about 200PB of science-ready data and have an entire archive of about 500PB.
Rubin Observatory stands on Cerro Pachón in Chile against a sky full of star trails in this long exposure night sky image
The observatory is primarily funded by US research agencies the National Science Foundation and the Department of Energy, said computational scientist George Beckett, programme manager at Edinburgh University, who is overseeing the tech infrastructure for the UK’s role in the LSST project.
World’s biggest digital camera
He told Computer Weekly: “They have basically built the world’s largest digital camera and mounted it on a very innovative mirror design for the telescope, to allow it to capture the sky very quickly.”
The camera can cover the whole southern sky in just three or four nights and get an image quickly, allowing repeat observations of each part of the sky and changes to be identified.
The LSST Camera installed on the Simonyi Survey Telescope at NSF–DOE Vera C Rubin Observatory
The project expects to image each patch of sky around 800 times over 10 years to make a “really deep sensitive image”.
“By accumulating lots of images and the light that’s captured by them, we can look back about 1.2 million light years, which is further than we have been able to delve before,” said Beckett.
It could potentially see things that have never been seen before, such as very small asteroids and similar objects in the solar system, and potentially various stars that are too dim to be seen by other telescopes or observation facilities.
“They are also hoping to catch lots of very exciting changes that happen around us. One of the most interesting ones is exploding supernovae, which are the death of certain stars,” added Beckett. “It’s designed to cover a whole range of different applications, everything from understanding the making of dark energy and dark matter, to cataloguing asteroids of around the 20- to 40-metre size, which are interesting in their own right, but also potential threats to the Earth.”
This illustration depicts NSF–DOE Vera C Rubin Observatory capturing light from supernovae, the explosive deaths of massive stars
The observatory can do this through a camera with extremely high resolution, effectively 3.2 gigapixels. “It might be the most high-resolution camera ever built,” said Beckett.
Over the 10-year project, the UK will host one of three international data facilities and process around 1.5 million images, capturing around 10 billion stars and galaxies. When complete, the full survey is expected to rack up as much as 500PB of data.
Computationally intense
The UK team is contributing to the software and computational science aspects of the project.
This includes cleaning the raw images that come from the telescope camera by removing, for example, atmospheric distortion and systematic noise from the telescope. This, according to Beckett, is “computationally intensive, taking millions of computing hours”.
External view of the LSST Camera on the vertical lift platform during its installation on the Simonyi Survey Telescope at the Vera C Rubin Observatory
The UK team is using the same computing systems that are used for the Large Hadron Collider. The US and France are also contributing to this part of the project.
Data is divided into two categories. “Hot data” is what the astronomers need to access on-demand, while “cooler data” is used for more strategic campaign-based processes, where a science community takes a large amount of data and moves it to a high-performance computer system, such as Archer2 in the UK.
Beckett said the team is currently building some “very large disk arrays” that will house data that requires on-demand access, while tape will be used for data that, although very important, isn’t being used every day, or is only used at certain times.
On-demand community access
In the UK, a science portal for the international community is capable of connecting around 1,500 astronomers with the UK Digital Research Infrastructure to support the exploitation of data.
“We’re running what’s called a data access centre,” said Beckett.
The team installing the LSST Camera on the Simonyi Survey Telescope
In the previous generation of computational astronomy, a scientist would typically download datasets they were interested in to their local computer. But now, via cloud platforms, the LSST project in the UK is providing access to the data, which is hosted in Edinburgh and the Rutherford Appleton Laboratory in Oxfordshire.
Astronomers want to be able to access all of the data for data mining or machine learning-type applications. “What we do is called ‘bring your computers to the data’, so the data is all hosted in Edinburgh or Rutherford Appleton, and we have cloud platforms that sit very close to it. Astronomers can log in, and they have all the tools and software that they need to access the data,” said Beckett.
The Edinburgh datacentre, which is called the Advanced Computing Facility, is the home of the Archer2 supercomputer. There are four machine rooms. In machine room one, the UK team is building a cloud system called Somerville, named after the Scottish astronomer Mary Somerville.
These exciting First Look images show that everything is working well and reassure us that we have a decade’s worth of wonderful data coming our way, with which UK astronomers will do great science
Bob Mann, University of Edinburgh/LSST:UK
“[Machine room one] will be both where we host the data and the compute resources that sit alongside that. And it’s very much an on-premise cloud. So, it looks like what you might get from AWS [Amazon Web Services] or Google, but it’s all running in a datacentre in Edinburgh,” said Beckett.
High alert
Astronomers will be provided with alerts by what the LSST team in the UK refers to as a “community alert broker”.
The observatory will aim to detect and report changes in the night sky, which could signify important scientific opportunities, within a minute of the most recent observation being taken. Each night, they’re expecting to produce around 10 million of these alerts.
“There is an application for artificial intelligence (AI) here, because handling 10 million alerts is not a trivial task, and many of them will either be uninteresting or potentially false positives,” said Beckett.
Things like AI and machine learning will deal with alerts based on the classifications they are given. Beckett said these alerts are sent to 10 different datacentres around the world, including the one in Edinburgh.
“We are particularly interested in what are called transient events. Things like exploding stars or quasars. We will consume these 10 million alerts a night and then, as quickly as possible, we will analyse and classify each one of them,” said Beckett.
Astronomers will sign up to certain types of alert or alerts in a particular area of the sky, or possibly alerts on certain galaxies, he explained. “If something happens within their sphere of interest, they will get an alert, by email, text message, or some kind of notification. They might want to go and look at it there and then, depending on what they’re interested in. Minutes could be significant.”
Following the publication of the first images taken by the Vera C Rubin Observatory LSST project, Bob Mann, professor of survey astronomy at the University of Edinburgh and LSST:UK project leader, said: “UK researchers have been contributing to the scientific and technical preparation for the Rubin LSST for more than 10 years. These exciting First Look images show that everything is working well and reassure us that we have a decade’s worth of wonderful data coming our way, with which UK astronomers will do great science.”
A total of £23m funding came from the Science and Technology Facilities Council (STFC) to support the UK part of the multinational project. The UK is the second-largest international contributor to the project.
Researchers combined astronomy and pathology to enable tumour imaging and develop a prediction model for targeted cancer immunotherapy, a new study reveals.
