French-led team wins global challenge to crunch the SKAO’s vast data
The Observatory's unprecedented field of view means astronomers will be working with crowded images of the sky that were previously far too faint to detect. To fully exploit the treasure trove of data to come, the radio astronomy community must develop accurate automated tools – especially given the processing power required to churn through SKAO’s enormous datasets.
The SKAO data challenges aim to prepare the community by giving them an opportunity to try tools and approaches that can tackle this step-change in astronomical data.
SKAO Science Data Challenge 2 (SDC2) saw 40 teams in 22 countries competing to identify and analyse 250,000 galaxies from a mock dataset comprising a series of stacked radio images, each representing a different frequency.
Radio astronomy is one of the top priorities of the Paris Observatory. The methods developed by the MINERVA project are innovative and promising and I look forward to their further development
Eight supercomputing centres around the world granted access to teams to analyse the resulting 1 terabit data cube, with Paris Observatory’s Team MINERVA declared the winners.
Both MINERVA, and close runner-up team FORSKA-Sweden employed machine learning, specifically deep learning.
Also referred to as neural networks, this method entails computers learning to recognise objects after being fed training data, similar to speech recognition software on a smartphone.
Other teams used or further-developed existing software to apply complex filtering algorithms to the data, making galaxies stand out from the instrumental noise.
The test’s use of supercomputers provided an interesting parallel with how SKAO’s data will eventually be crunched.
Eight terabits per second of data will be transferred from the SKA telescopes to Central Processing Facilities on onward to Science Data Processors in Cape Town and Perth.
To process this enormous amount of information, SKAO will use two new supercomputers – each among the five fastest on earth with a processing speed of ~135 PFlops.
In total, the SKAO will archive 710 petabytes of data each year – enough to fill the storage capacity on 1.5 million typical laptops
