D-MeerKAT Projects

The science goals of D-MeerKAT are designed to develop software and hardware support for the large data sets (tens of PetaBytes) from the MeerKAT, such as, large storage devices and data management, advanced imaging techniques and pipelines, spectral-line analysis, source characterization, time-domain astronomy, and many more. Below are the five major projects of D-MeerKAT.

Federated Science Data Centre and Big Data Challenges (RUB, FZJ & UBI)

Data Transfer, Storage and Computing
Astronomical observations using modern and future telescopes such as MeerKAT, ext-MeerKAT (MeerKAT Extension) and the upcoming Square Kilometre Array (SKA) produce a very large amount of data. It is a big challenge to handle the transfer and storage of such a large data volume. We will therefore:

develop data-compression techniques (for both general and astronomical applications)
research the fastest way to transfer data from the Karoo telescope site to Cape Town, South Africa, and then to Germany
establish a state-of-the-art data-archive system, with quick and easy data access through meta-data, on the high-performance computing (HPC) cluster in Jüich.

Pipeline Development and Scalability for Future Facilities
In order to maximize the efficiency and performance of astronomical analyses in such large data volumes, we will:

optimise the current data-reduction software and pipeline
compare new imaging and processing techniques for best performance and scientific results
implement a dynamic quality control system, which enables interruption of the process as needed

The newly developed pipeline will be able to handle increasingly large data sets and be ready for future applications on supercomputing facilities and computing networks.

MeerKAT Imaging and Signal Processing (UHH, TUM & MPA)

MeerKAT is the best available telescope to perform magnetic-field studies thanks to its wide bandwidth (770 MHz at L-band) and high spectral resolution (26 kHz). The upcoming MIGHTEE (MeerKAT International GHz Tiered Extragalactic Exploration) Survey will allow us to observe one of the first large samples of polarised astronomical sources. Through this survey, we can measure the evolution of the properties of Active Galactic Nuclei (AGN) as well as study large-scale magnetic fields. In order to fully exploit the potential of the MIGHTEE's polarisation surveys, we will address the following technical challenges.

Ionospheric Effects
When a radio wave passes through a magnetised plasma, the polarisation of the wave will be rotated due to the magnetic field. This phenomenon is known as the Faraday rotation effect and provides the main tool in astrophysics to infer the properties of magnetic fields. However, as the cosmic radio signal passes through the Earth's ionosphere, an additional time-varying Faraday rotation is introduced. To fulfll the goal of the polarisation surveys, it is essential to correct for this ionospheric contribution and minimise the uncertainty of the Faraday rotation measurements. We will evaluate and correct for ionospheric effects in MeerKAT data using two approaches:

monitor the total electron content (TEC) of the ionosphere using GPS on satellites
combine differential TEC and differential rotation measure (RM) measurements and assuming a model for the Earth's magnetic field

RM Synthesis Imaging
We can obtain a 3D reconstruction of the polarised sky by using a technique called RM synthesis, which employs the Faraday rotation effect. Ideally, this reconstruction should be directly obtained by performing an image deconvolution (CLEAN) of the entire bandwidth of the raw data. However, such a technique is extremely computationally intensive and, due to the computational limitations, we currently first reduce the data amount by imaging only in small frequency chunks instead of the entire bandwidth, which severely limits the sensitivity and the fidelity of the images. We will therefore test and compare two different approaches using WSCLEAN and RESOLVE to implement the direct reconstruction, and explore possibilities to accelerate the computational process and to include the pipeline into the MeerKAT infrastructure.

Direction Dependent Polarisation Calibration
The Information Field Theory (IFT) group at the MPI for Astrophysics is developing novel imaging and calibration algorithms based on IFT, which explore and exploit correlations of signals in spatial, temporal and spectral domains. For MeerKAT, an enhancement of the RESOLVE algorithms is in development, which provides spatial-spectal image cubes directly from the raw measurements. Performance of the RESOLVE algorithm on a real data set is shown in the figure. This project will enable direction-dependent phase calibration for issues such as ionospheric effects, varying antenna patterns and reflections as well as polarisation leakage and pointing errors.

Multi-frequency Polarimetric Imaging
We also aim to establish the first Bayesian imaging algorithm for polarisation imaging. Both polarisation calibration and imaging can thereby be unified into one single inference machinery, allowing us to quantify the uncertainty on the final data products.

Real-time Screening and Transient Identification (ZAH, UBI & MPIfR)

All MeerKAT observations allow for a commensal observing mode, where all incoming data are instantaneously screened for transient phenomena (ThunderKAT and MeerTRAP programmes). The goal of this project is to establish the system that identifies astrophysical transients in real-time and follows them up

by immediately changing the MeerKAT schedule and
by posting alerts in Virtual Observatory (VO) format for other facilities, which can then conduct multi-frequency follow-up observations.

This requires a real-time identification and mitigation of radio frequency interference (RFI) generated by different sources such as a rapidly increasing number of telecommunication satellites and Earth-based transmitters, which affects any observational data being acquired. We will further advance specialised techniques for so-called "spatial" filtering of RFI to remove such RFI as the data come off the telescopes.

On-the-fly Interferometry during Scanning Mode Observations with MeerKAT Screening (LMU & TLS)

The MeerKAT sensitivity is opening a new discovery space in extragalactic astronomy and observational cosmology. Neutral atomic hydrogen (HI), which is the most abundant atom in space, has its emission line at a wavelength of 21 cm in the radio spectrum, known as the HI line. Mapping the hydrogen line (HI intensity mapping) has not only been a primary technique in radio astronomy to study the dynamics of galaxies, but will also provide a unique cosmological probe of fundamental physics with the new generation of radio telescopes. The planned MeerKLASS (MeerKAT Large Area Synoptic Survey) is a deep and wide-area HI survey. In such large-scale observations, it is crucial to have a commensal observing mode that allows the same data to simultaneously serve different scientific goals. The MeerKLASS will simultaneously provide two different cosmological surveys: HI intensity mapping and a continuum radio galaxy survey, which will yield strong constraints on several key cosmological parameters (such as the nature of dark energy) as well as larger samples of rare objects. The challenge of such commensal observing lies in the fact that the intensity mapping needs to be carried out in scanning mode, wherein the entire antenna array scans across the sky at rates much higher than the sky's sidereal motion. Our goal is to develop such an observing mode on the MeerKAT array.

Calibrated HI and Continuum Imaging for Scanning Mode Observations
Our primary goals in this project are:

to establish an on-the-fly interferometry capability for MeerKAT that allows for continuum and HI imaging, and
to develop the calibration strategies for the interferometric data during scanning mode.

In order to achieve these goals, we will take the following steps:

adopt the established continuum and HI imaging techniques that are being used for the MIGHTEE survey
extend these techniques to enable an on-the-fly Mosaicing mode by following both the techniques developed by the VLA team and extensions of that work within SARAO
apply the techniques to MeerKAT data sets and extend the algorithms
build automated on-the-fly interferometry pipelines tuned to optimally deal with the MeerKAT data in preparation for large-scale surveys like MeerKLASS

Preparing the SKA-MPG Telescope for the SKA Era (UBI, TLS, MPIfR, ZAH & TUDO)

Enabling Science with the SKA-MPG Telescope
The 15-m SKA-MPG telescope is an excellent instrument to perform fast, sensitive, broad-bandwidth polarisation surveys of the entire southern sky. The prototype dish will provide spectro-polarimetric data (i.e., measurements of the polarisation state of the radio wave as a function of wavelength) in the frequency range of 1.7 to 3.5 GHz and thereby the opportunity to study the Galactic magneto-ionic medium (i.e., ionized gas permeated by a steady magnetic field). Some of the exciting scientific prospects being pursued are:

Milky Way (our Galaxy)
- separate the synchrotron and the free-free emission from the Milky Way (using the broad-bandwidth radio continuum data alone)
- reveal the nature of turbulence in the Milky Way's magneto-ionic medium
- study the origin and nature of large-scale polarised structures in the Milky Way
Cosmic microwave background (CMB)
- accurately measure and correct for the contribution of the foreground (synchrotron and free-free) emission from the Milky Way
- detect signatures of primordial gravitational waves and the reionization history of the Universe (imprinted in the polarized CMB)
Active galactic nucleii (AGN)
- monitor time-variability of polarisation of bright AGN for several years with roughly one month cadence
- gain insight into the connection between polarisation variability related to black hole accretion and jet launching mechanisms

Toward Robotic Operations
Automatic and robotic modes of operation play a key role in accomplishing highly efficient telescope operation and in high reliability of calibrations and observations. Such robotic operations are particularly beneficial in arrays of telescopes with similar hardware and similar observing schedules, even more so in radio astronomy where telescopes can run 24/7. Our goal in this project is to automate operations of the SKA-MPG telescope and turn it into the first fully robotic radio telescope. A proven system will be a prototype concept for future robotic operations of the entire ext-MeerKAT array and the upcoming SKA.

1. Automated Scheduler

The key element in a robotic system is a scheduler that optimises operations (calibrations and observations) of multiple observing programmes to maximise efficiency (scanning, tracking, imaging). An automated scheduler will:

increase the efficiency of scientific operations
enhance the flexibility of conducting multiple programmes
allow an optimum sharing of resources between continuous programmes and triggered observations of transient or time-domain studies.

In this project, we will build such an auto-scheduler based on our expertise in identifying and communicating meerKAT recorded transients and in developing and running the fully robotic ATOM optical telescope in Namibia (ZAH).

2. Data Quality Control from Metadata

Another key aspect in efficiently conducting survey observations (and robotic operations in general) is an autonomous system that can decide whether the data quality is sufficient to achieve the science goal of the observation. Such a system will enable, for example, autonomous decisions on repetitions, greatly enhancing the survey efficiency. Both the telescope performance and environmental/observational conditions are crucial factors to determine the data quality. These are monitored by sensors, which provide a stream of metadata. Building upon our extensive expertise in fast and efficient tracking of data quality and in analysis of metadata streams towards robotic operation of the FACT Cherenkov gamma-ray telescope on La Palma (TUDO), we will develop a new system to correlate sensor metadata with the observation data quality of the SKA-MPG telescope, which will then be fully integrated with the auto-scheduler and generalised for other radio telescope systems.

Members of the D-MeerKAT consortium: