Preparing for the ultimate Big Data challenge

The SKA project poses the ultimate big data challenge. It will archive more than 700 petabytes of data per year – equivalent to the data storage capacity of 1.5 million typical laptops. To manage the vast quantities of data, the SKAO will be supported by a global network of SKA Regional Centres (SRC), distributed around the world and hosted by member states.

Canada’s existing expertise makes it uniquely qualified to tackle this challenge. The country will be home to the only SRC in the Americas and will be hosted by the National Research Council of Canada’s (NRC) Canadian Astronomy Data Centre (CADC) in collaboration with the Digital Research Alliance of Canada and CANARIE.

Working with the Canadian SRC, Canada is also forming a national network of SKA experts  with the NRC’s Canadian SKA Scientist Program. The program supports outstanding early-career scientists at universities across Canada who will play a pivotal role in shaping SKA science in Canada, building international research networks and leading both scientific and technical initiatives.

The combination of accessible cloud computing, sophisticated data analysis tools, and a network of astronomical and digital experts will promote research inclusivity by ensuring that all Canadian researchers have effective and equitable access to the full potential of the SKA. This will enable cross-cutting research that will advance Canadian innovation in digital technology, AI and machine learning.

While focused primarily on astronomical data processing, the investment in the Canadian SRC will enhance overall digital and AI expertise in Canada. Its cutting-edge infrastructure and collaborative network provide an ideal environment for developing the technical expertise and systems-level understanding needed to build and maintain the digital foundations of Canada's AI future.

The skills developed handling the deluge of SKA data will prepare the next-generation of innovators who will shape the digital future.