Science projects of D-MeerKAT

Federated Science Data Centre and Big Data Challenges

Led by RUB, FZJ & UBI

Data Transfer, Storage and Computing: As a very large amount of data will come from the MeerKAT telescope and in a later stage ext-MeerKAT and the upcoming Square Kilometre Array (SKA), it is essential to develop new methods of data storage, compression and management. This project aims to achieve lossless compression techniques and incorporate them into the data reduction and storage software. The development of such compression techniques involves a new and more efficient format of file storage than the current standards. For this, we will adapt not only general data compression techniques, but also those specific to radio astronomy, for example, baseline-dependent averaging, compression after flagging, and on-the-fly calibration applications. Since the compressed data size is expected to remain still large, another crucial part of this project is to research into what is the fastest way to transfer data from the Karoo telescope site to Cape Town, South Africa, and then to Germany. In combination with the newly developed compression techniques, a further holistic approach will be taken to the data archive system, such that specific observations are quickly and easily accessible through meta-data. Such a new archiving system will be implemented as data center on the HPC cluster and incorporated into the data reduction pipelines.

Pipeline Development and Scalability for Future Facilities: Another goal of the project is to further optimise and develop the current pipeline and other software for data reduction as scalable software, which can then be applied to even larger datasets on super computers. To ensure the best possible data reduction and its products, new imaging and processing techniques will be incorporated and their performance will be compared. Another aspect of the pipeline development is the quality control: instead of a full report done at the end of a run, a more dynamic and scalable quality control system will be implemented, which enables interruption of the process as needed. These developments will ensure that these pipelines can be applied to increasingly large data sets from MeerKAT and the future SKA.


MeerKAT Imaging and Signal Processing

IFT image construction construction
IFT-based image re-construction. Radio interferometric image of radio galaxies in the galaxy cluster Abell 2219. The images were constructed by data back-projection (top), the CLEAN algorithm (middle), and the RESOLVE algorithm (bottom). Negative and therefore not physical fluxes are displayed in white.

Led by UHH, TUM & MPA

The MIGHTEE (MeerKAT International GHz Tiered Extragalactic Exploration) Survey will provide one of the first large sample of polarised sources (with several hundred to 1000 polarised sources per 1 degree2) to measure the evolution of the intrinsic and environmental properties of Active Galactic Nuclei (AGN) as well as a supremely-dense Faraday Rotation Measure (RM) grid to study large-scale magnetic fields. Furthermore, MeerKAT's wide L-band bandwidth (770 MHz) and high spectral resolution (26 kHz) makes it the best available telescope to perform RM synthesis. The goal of this project is to address the following technical challenges, in order to fully exploit the potential of the MIGHTEE's polarisation surveys.

Ionospheric Effect: The Faraday rotation effect, which is induced into a polarised radio wave passing through a magnetised plasma, is the main tool in astrophysics to infer the properties of magnetic fields. However, as the cosmic radio signal passes through the Earth's ionosphere, an additional time varying Faraday rotation is introduced. Although this effect has more impact at low frequencies, it is essential to correct for the ionospheric contribution to the Faraday depth even in L-band of MeerKAT, in order to minimise the uncertainty associated with measurements of the rotation measure (RM) and to fulfil the requirements for the polarisations surveys. In this project, we will evaluate and correct for ionospheric RM effects in MeerKAT data using two approaches: 1) by monitoring the total electron content (TEC) of the ionosphere using GPS on satellites and 2) by combining differential TEC and differential RM measurements and assuming a model for the Earth magnetic field.

Direction Dependent Polarisation Calibration: The Information Field Theory (IFT) group at the MPI for Astrophysics is developing novel imaging and calibration algorithms based on IFT. These explore and exploit correlations of signals in spatial, temporal and spectral domains. For MeerKAT, an enhancement of the RESOLVE algorithms to provide spatial-spectal image cubes directly from the raw visibilities is in development. Performance of the RESOLVE algorithm on a real dataset is shown in the figure. This project aims in particular to enable direction-dependent phase calibration such as ionoshperic effects, varying antenna patterns and reflections as well as polarisation leakage and pointing errors. 

Multi-frequency Polarimetric Imaging: Another aim of the project is establish the first Bayesian imaging algorithm for polarisation imaging. Both polarisation calibration and imaging can thereby be unified into one single inference machinery, allowing us to quantify the uncertainty on the final data products. Furthermore, by implementing the non-uniform Fourier transform, we will provide excellent valuable software for data reduction with excellent scaling capabilities.

RM Synthesis Imaging: A 3D reconstruction and representation of the polarised sky can be obtained with a technique called RM synthesis, which uses the Faraday rotation effect. The current approach first reduces the data amount by imaging the polarised intensity in small frequency chunks and then performs the RM synthesis imaging along the line of sight, which severely limits the achievable sensitivity and the fidelity of the images. Ideally, the Faraday spectrum should be directly reconstructed by performing an image deconvolution (CLEAN) of the entire bandwidth in (u-v) visibility domain. As such a technique is extremely computationally intensive, this project will test and compare two different approaches using WSCLEAN and RESOLVE to implement this, and explore the possibilities to accelerate the computational process and to include the pipeline into the MeerKAT infrastructure.


Real-time Screening and Transient Identifiation

Led by ZAH, UBI & MPIfR

All MeerKAT observations allow for a commensal observing mode, where all incoming data are instantaneously screened for transient phenomena (ThunderKAT and MeerTRAP programmes). The goal of this project is to establish the system that identifies astrophysical transients in real-time and follows them up 1) by immediately changing the MeerKAT schedule and 2) by posting alerts in Virtual Observatory (VO) format for other facilities, which can then conduct multi-frequency follow-up observations. This requires a real-time identification and mitigation of radio frequency interference (RFI) generated by different sources such as a rapidly increasing number of telecommunication satellites and Earth-based transmitters, which affects any observational data being acquired. We will further advance the specialised techniques for so-called "spatial" filtering of RFI to remove such RFI as the data come off the telescopes.


On-the-fly Interferometry during Scanning Mode Observations with MeerKAT Screening

Led by LMU & TLS

The MeerKAT sensitivity is opening a new discovery space in continuum and extragalactic HI science. To realise the scientific goal of a deep (down to 10μJ) and large solid angle (>1000 degree2) survey such as the planned MeerKLASS (MeerKAT Large Area Synoptic Survey), it is crucial to have a commensal observing mode that allows observations to simultaneously serve continuum and intensity mapping goals. Such an observing mode scheme enables continuum and HI science of much larger samples of rare objects (such as galaxy clusters out to z~1) and HI intensity mapping to study the clustering of galaxy populations out to z ~0.6 toward the goal of detecting baryon acoustic oscillations and using them to constrain the distance-redshift relation. The challenge of commensal observing lies in the fact that the intensity mapping needs to be carried out in scanning mode, wherein the entire array scans across the sky at rates much higher than sidereal. Our goal is to develop such an observing mode on the MeerKAT array by advancing techniques to address the challenge.

Calibrated HI and Continuum Imaging for Scanning Mode Observations: In this project, our effort focuses on bringing up an on-the-fly intereferometry capability for MeerKAT that allows for continuum and HI imaging. Our aim is to develop and demonstrate the calibration strategies for the intereferometric dataset during scanning mode. Specifically, we adopt the established continuum and HI imaging techniques that are being used for the MIGHTEE survey and extended these to an on-the-fly Mosaicing mode, by following both the techniques developed by the VLA team and extensions of that work within SARAO to intereferometric datasets and by extending these algorithms. Finally we will build automated on-the-fly interferometry pipelines tuned to optimally deal with the MeerKAT data in preparation for large scale surveys like MeerKLASS.

Preparing the SKA-MPG Telescope for the SKA Era

SKA-MPG telescope
SKA-MPG telescope, the first SKA prototype dish assembled on
the South African site (Credit: SKA Observatory)


Enabling Science with the SKA-MPG Telescope: The 15-m aperture SKA-MPG telescope is an excellent instrument to perform fast, sensitive, broad-bandwidth polarization surveys of the entire southern sky. The prototype dish will provide spectro-polarimetric data in the frequency range 1.7 to 3.5 GHz at 1 degree angular resolution and thereby the opportunity to explore an entirely new parameter space in studies of the Galactic magneto-ionic medium. Some of the exciting scientific prospects that are being pursued are:

      • Separate the synchrotron and the free-free emission from the Milky Way by using the broad-bandwidth radio continuum data alone.
      • Quantify Faraday depolarization through broad-bandwidth spectro-polarimetry and study the nature of turbulence in the Mikly Way's magneto-ionic medium.
      • Study the origin and nature of large-scale polarized structures in the Milky Way.
      • Pin down the contribution of the foreground synchrotron + free-free emission components to the cosmic microwave background (CMB) at sub-μK accuracy per 1 degree2, and the contribution of the polarized synchrotron emission at nK accuracy. This will be crucial in detecting signatures of primordial gravitational waves and the reionization history of the Universe imprinted on the E- and B-modes of the polarized CMB.
      • Monitor time-variability of bright active galactic nucleii (AGN) in all Stokes parameters for several years with roughly one month cadence. This will provide insights into the connection between AGN polarization variability related to black hole accretion and jet launching mechanisms.

Toward Robotic Operations: Automatic and robotic modes of operations following a standard protocol is a key element in improving reliability of calibration and observations as well as the efficiency of telescope operations. Such robotic operations are particularly beneficial in arrays of telescopes with similar hardware and similar observing schedules, even more so in radio astronomy where telescopes can run 24/7. Our ambition is to automate operations of the SKA-MPG telescope and turn it into the first fully robotic radio telescope,A proven system will be a prototype concept for future robotic operations of the entire ext-MeerKAT array and the upcoming SKA.
1) Automated Scheduler: The key element in a robotic system is a scheduler that optimises operations (calibrations and observations) of multiple observing programmes to maximise efficiency (scanning, tracking, imaging). An automated scheduler will not only increase the efficiency of scientific operations but also enhance the flexibility of conducting multiple programmes and allow an optimum sharing of resources between continuous programmes and triggered observations of transient or time-domain studies. In this project, we will build such an auto-scheduler based on our expertise on identifying and communicating meerKAT recorded transients and in developing and running the fully robotic ATOM optical telescope in Namibia (ZAH).
2) Data Quality Control from Metadata: Another key aspect in efficiently performing survey observations (and robotic operations in general) is an autonomous system, based on which the decision can be made if data quality is sufficient to achieve the underlying science goal of the observation. Such a system, interfaced with the auto-scheduler, will greatly strengthen the survey efficiency of the telescope, for example, by enabling autonomous decisions on repetitions. The observational data quality crucially depends on telescope performance and environmental/observational conditions, both monitored by sensors providing a stream of metadata. Building upon the extensive expertise in characterising telescope properties and optimising telescope performance by means of fast and efficient tracking of data quality and analysis of metadata streams towards robotic operation of the FACT Cherenkov gamma-ray telescope on La Palma (TUDO), we will develop a new system to correlate sensor metadata with the observation data quality of the SKA-MPG telescope, which will then be fully integrated with the auto-scheduler and generalised for other radio telescope systems.



Members of the D-MeerKAT consortium:




Logo Max-Planck-Institut für Radioastronomie, Bonn

Logo Max-Planck-Institut für Astrophysik (MPIA), Garching





Sponsored by







The D-MeerKAT page is maintained by: This email address is being protected from spambots. You need JavaScript enabled to view it. This email address is being protected from spambots. You need JavaScript enabled to view it. and This email address is being protected from spambots. You need JavaScript enabled to view it..