KalCal: a novel calibration framework for radio interferometry using the Kalman Filter and Smoother
- Authors: Welman, Brian Allister
- Date: 2024-10-11
- Subjects: Radio interferometers , Calibration , Kalman filtering , Bayesian inference , Signal processing , Radio astronomy , MeerKAT
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/467127 , vital:76818
- Description: Calibration in radio interferometry is essential for correcting measurement errors. Traditional methods employ maximum likelihood techniques and non-linear least squares solvers but face challenges due to the data volumes and increased noise sensitivity of contemporary instruments such as MeerKAT. A common approach for mitigating these issues is using “solution intervals”, which helps manage the data volume and reduces overfitting. However, inappropriate interval sizes can degrade calibration quality, and determining optimal sizes is challenging, often relying on brute-force methods. This study introduces Kalman Filtering and Smoothing in Calibration (KalCal), a new framework for calibration that combines the Kalman Filter, Kalman Smoother, and the energy function: the negative logarithm of the Bayesian evidence. KalCal offers Bayesian-optimal solutions as probability densities and models calibration effects with lower computational requirements than iterative approaches. Unlike traditional methods, which require all the data for a particular solution to be in memory simultaneously, KalCal’s recursive computations only need a single pass through the data with appropriate prior information. The energy function provides the means for KalCal to determine this prior information. Theoretical contributions include additions to complex optimisation literature and the “Kalman-Woodbury Identity” that reformulates the traditional Kalman Filter. A Python implementation of the KalCal framework was benchmarked against solution intervals as implemented in the QuartiCal package. Simulations show KalCal matching solution intervals in high Signal-to-Noise Ratio (SNR) scenarios and surpassing them in low SNR conditions. Moreover, the energy function produced minima that coincide with KalCal’s Mean Square Error (MSE) on the true gain signal. This result is significant as the MSE is unavailable in real applications. Further research is needed to assess the computational feasibility and intricacies of KalCal. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2024
- Full Text:
- Authors: Welman, Brian Allister
- Date: 2024-10-11
- Subjects: Radio interferometers , Calibration , Kalman filtering , Bayesian inference , Signal processing , Radio astronomy , MeerKAT
- Language: English
- Type: Academic theses , Master's theses , text
- Identifier: http://hdl.handle.net/10962/467127 , vital:76818
- Description: Calibration in radio interferometry is essential for correcting measurement errors. Traditional methods employ maximum likelihood techniques and non-linear least squares solvers but face challenges due to the data volumes and increased noise sensitivity of contemporary instruments such as MeerKAT. A common approach for mitigating these issues is using “solution intervals”, which helps manage the data volume and reduces overfitting. However, inappropriate interval sizes can degrade calibration quality, and determining optimal sizes is challenging, often relying on brute-force methods. This study introduces Kalman Filtering and Smoothing in Calibration (KalCal), a new framework for calibration that combines the Kalman Filter, Kalman Smoother, and the energy function: the negative logarithm of the Bayesian evidence. KalCal offers Bayesian-optimal solutions as probability densities and models calibration effects with lower computational requirements than iterative approaches. Unlike traditional methods, which require all the data for a particular solution to be in memory simultaneously, KalCal’s recursive computations only need a single pass through the data with appropriate prior information. The energy function provides the means for KalCal to determine this prior information. Theoretical contributions include additions to complex optimisation literature and the “Kalman-Woodbury Identity” that reformulates the traditional Kalman Filter. A Python implementation of the KalCal framework was benchmarked against solution intervals as implemented in the QuartiCal package. Simulations show KalCal matching solution intervals in high Signal-to-Noise Ratio (SNR) scenarios and surpassing them in low SNR conditions. Moreover, the energy function produced minima that coincide with KalCal’s Mean Square Error (MSE) on the true gain signal. This result is significant as the MSE is unavailable in real applications. Further research is needed to assess the computational feasibility and intricacies of KalCal. , Thesis (MSc) -- Faculty of Science, Physics and Electronics, 2024
- Full Text:
Systematic effects and mitigation strategies in observations of cosmic re-ionisation with the Hydrogen Epoch of Reionization Array
- Authors: Charles, Ntsikelelo
- Date: 2024
- Subjects: Cosmology , Astrophysics , Radio astronomy , Hydrogen Epoch of Reionization Array , Epoch of reionization
- Language: English
- Type: Academic theses , Doctoral theses , text
- Identifier: http://hdl.handle.net/10962/432605 , vital:72886 , DOI 10.21504/10962/432605
- Description: The 21 cm transition from neutral Hydrogen promises to be the best observational probe of the Epoch of Reionisation (EoR). It has driven the construction of the new generation of lowfrequency radio interferometric arrays, including the Hydrogen Epoch of Reionization Array (HERA). The main difficulty in measuring the 21 cm signal is the presence of bright foregrounds that require very accurate interferometric calibration. However, the non-smooth instrumental response of the antenna as a result of mutual coupling complicates the calibration process by introducing non-smooth calibration errors. Additionally, incomplete sky models are typically used in calibration due to the limited depth and resolution of current source catalogues. Combined with the instrumental response, the use of incomplete sky models during calibration can result in non-smooth calibration errors. These, overall, impart spectral structure on smooth foregrounds, leading to foreground power leakage into the EoR window. In this thesis we explored the use of fringe rate filters (Parsons et al., 2016) as a mean to mitigate calibration errors resulting from the effects of mutual coupling and the use of an incomplete sky model during calibration. We found that the use of a simple notch filter mitigates calibration errors reducing the foreground power leakage into the EoR window by a factor of ∼ 102. Thyagarajan et al. (2018) proposed the use of closure phase quantities as a means to detect the 21 cm signal, which has the advantage of being independent (to first order) from calibration errors and, therefore, bypasses the need for accurate calibration. In this thesis, we explore the impact of primary beam patterns affected by mutual coupling on the closure phase. We found that primary beams affected by mutual coupling lead to a leakage of foreground power into the EoR window, which can be up to ∼ 104 times and is mainly caused by the unsmooth spectral structure primary of primary beam sidelobes affected by mutual coupling. This power leakage was confined to k < 0.3 pseudo h Mpc−1. Lastly, we also proposed and demonstrated an analysis technique that can be used to derive a flux scale correction in post-calibrated HERA data. We found that after applying flux scale correction to calibrated HERA data, the bandpass error reduces significantly, with an improvement of 6%. The derived flux scale correction was antenna-independent, and it can be applied to fix the overall visibility spectrum scale of H4C data post-calibration in a fashion similar to Jacobs et al. (2013). , Thesis (PhD) -- Faculty of Science, Physics and Electronics, 2024
- Full Text:
- Authors: Charles, Ntsikelelo
- Date: 2024
- Subjects: Cosmology , Astrophysics , Radio astronomy , Hydrogen Epoch of Reionization Array , Epoch of reionization
- Language: English
- Type: Academic theses , Doctoral theses , text
- Identifier: http://hdl.handle.net/10962/432605 , vital:72886 , DOI 10.21504/10962/432605
- Description: The 21 cm transition from neutral Hydrogen promises to be the best observational probe of the Epoch of Reionisation (EoR). It has driven the construction of the new generation of lowfrequency radio interferometric arrays, including the Hydrogen Epoch of Reionization Array (HERA). The main difficulty in measuring the 21 cm signal is the presence of bright foregrounds that require very accurate interferometric calibration. However, the non-smooth instrumental response of the antenna as a result of mutual coupling complicates the calibration process by introducing non-smooth calibration errors. Additionally, incomplete sky models are typically used in calibration due to the limited depth and resolution of current source catalogues. Combined with the instrumental response, the use of incomplete sky models during calibration can result in non-smooth calibration errors. These, overall, impart spectral structure on smooth foregrounds, leading to foreground power leakage into the EoR window. In this thesis we explored the use of fringe rate filters (Parsons et al., 2016) as a mean to mitigate calibration errors resulting from the effects of mutual coupling and the use of an incomplete sky model during calibration. We found that the use of a simple notch filter mitigates calibration errors reducing the foreground power leakage into the EoR window by a factor of ∼ 102. Thyagarajan et al. (2018) proposed the use of closure phase quantities as a means to detect the 21 cm signal, which has the advantage of being independent (to first order) from calibration errors and, therefore, bypasses the need for accurate calibration. In this thesis, we explore the impact of primary beam patterns affected by mutual coupling on the closure phase. We found that primary beams affected by mutual coupling lead to a leakage of foreground power into the EoR window, which can be up to ∼ 104 times and is mainly caused by the unsmooth spectral structure primary of primary beam sidelobes affected by mutual coupling. This power leakage was confined to k < 0.3 pseudo h Mpc−1. Lastly, we also proposed and demonstrated an analysis technique that can be used to derive a flux scale correction in post-calibrated HERA data. We found that after applying flux scale correction to calibrated HERA data, the bandpass error reduces significantly, with an improvement of 6%. The derived flux scale correction was antenna-independent, and it can be applied to fix the overall visibility spectrum scale of H4C data post-calibration in a fashion similar to Jacobs et al. (2013). , Thesis (PhD) -- Faculty of Science, Physics and Electronics, 2024
- Full Text:
Accelerated implementations of the RIME for DDE calibration and source modelling
- Authors: Van Staden, Joshua
- Date: 2021
- Subjects: Radio astronomy , Radio inferometers , Radio inferometers -- Calibration , Radio astronomy -- Data processing , Radio inferometers -- Data processing , Radio inferometers -- Calibration -- Data processing
- Language: English
- Type: text , Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10962/172422 , vital:42199
- Description: Second- and third-generation calibration methods filter out subtle effects in interferometer data, and therefore yield significantly higher dynamic ranges. The basis of these calibration techniques relies on building a model of the sky and corrupting it with models of the effects acting on the sources. The sensitivities of modern instruments call for more elaborate models to capture the level of detail that is required to achieve accurate calibration. This thesis implements two types of models to be used in for second- and third-generation calibration. The first model implemented is shapelets, which can be used to model radio source morphologies directly in uv space. The second model implemented is Zernike polynomials, which can be used to represent the primary beam of the antenna. We implement these models in the CODEX-AFRICANUS package and provide a set of unit tests for each model. Additionally, we compare our implementations against other methods of representing these objects and instrumental effects, namely NIFTY-GRIDDER against shapelets and a FITS-interpolation method against the Zernike polynomials. We find that to achieve sufficient accuracy, our implementation of the shapelet model has a higher runtime to that of the NIFTY-GRIDDER. However, the NIFTY-GRIDDER cannot simulate a component-based sky model while the shapelet model can. Additionally, the shapelet model is fully parametric, which allows for integration into a parameterised solver. We find that, while having a smaller memory footprint, our Zernike model has a greater computational complexity than that of the FITS-interpolated method. However, we find that the Zernike implementation has floating-point accuracy in its modelling, while the FITS-interpolated model loses some accuracy through the discretisation of the beam.
- Full Text:
- Authors: Van Staden, Joshua
- Date: 2021
- Subjects: Radio astronomy , Radio inferometers , Radio inferometers -- Calibration , Radio astronomy -- Data processing , Radio inferometers -- Data processing , Radio inferometers -- Calibration -- Data processing
- Language: English
- Type: text , Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10962/172422 , vital:42199
- Description: Second- and third-generation calibration methods filter out subtle effects in interferometer data, and therefore yield significantly higher dynamic ranges. The basis of these calibration techniques relies on building a model of the sky and corrupting it with models of the effects acting on the sources. The sensitivities of modern instruments call for more elaborate models to capture the level of detail that is required to achieve accurate calibration. This thesis implements two types of models to be used in for second- and third-generation calibration. The first model implemented is shapelets, which can be used to model radio source morphologies directly in uv space. The second model implemented is Zernike polynomials, which can be used to represent the primary beam of the antenna. We implement these models in the CODEX-AFRICANUS package and provide a set of unit tests for each model. Additionally, we compare our implementations against other methods of representing these objects and instrumental effects, namely NIFTY-GRIDDER against shapelets and a FITS-interpolation method against the Zernike polynomials. We find that to achieve sufficient accuracy, our implementation of the shapelet model has a higher runtime to that of the NIFTY-GRIDDER. However, the NIFTY-GRIDDER cannot simulate a component-based sky model while the shapelet model can. Additionally, the shapelet model is fully parametric, which allows for integration into a parameterised solver. We find that, while having a smaller memory footprint, our Zernike model has a greater computational complexity than that of the FITS-interpolated method. However, we find that the Zernike implementation has floating-point accuracy in its modelling, while the FITS-interpolated model loses some accuracy through the discretisation of the beam.
- Full Text:
Parametrised gains for direction-dependent calibration
- Authors: Russeeaeon, Cyndie
- Date: 2021
- Subjects: Radio astronomy , Radio inferometers , Radio inferometers -- Calibration
- Language: English
- Type: text , Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10962/172400 , vital:42196
- Description: Calibration in radio interferometry describes the process of estimating and correcting for instrumental errors from data. Direction-Dependent (DD) calibration entails correcting for corruptions which vary across the sky. For small field of view observations, DD corruptions can be ignored but for wide fild observations, it is crucial to account for them. Traditional maximum likelihood calibration is not necessarily efficient in low signal-to-noise ratio (SNR) scenarios and this can lead to ovefitting. This can bias continuum subtraction and hence, restrict the spectral line studies. Since DD effects are expected to vary smoothly across the sky, the gains can be parametrised as a smooth function of the sky coordinates. Hence, we implement a solver where the atmosphere is modelled using a time-variant 2-dimensional phase screen with an arbitrary known frequency dependence. We assume arbitrary linear basis functions for the gains over the phase screen. The implemented solver is ptimised using the diagonal approximation of the Hessian as shown in previous studies. We present a few simulations to illustrate the performance of the solver.
- Full Text:
- Authors: Russeeaeon, Cyndie
- Date: 2021
- Subjects: Radio astronomy , Radio inferometers , Radio inferometers -- Calibration
- Language: English
- Type: text , Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10962/172400 , vital:42196
- Description: Calibration in radio interferometry describes the process of estimating and correcting for instrumental errors from data. Direction-Dependent (DD) calibration entails correcting for corruptions which vary across the sky. For small field of view observations, DD corruptions can be ignored but for wide fild observations, it is crucial to account for them. Traditional maximum likelihood calibration is not necessarily efficient in low signal-to-noise ratio (SNR) scenarios and this can lead to ovefitting. This can bias continuum subtraction and hence, restrict the spectral line studies. Since DD effects are expected to vary smoothly across the sky, the gains can be parametrised as a smooth function of the sky coordinates. Hence, we implement a solver where the atmosphere is modelled using a time-variant 2-dimensional phase screen with an arbitrary known frequency dependence. We assume arbitrary linear basis functions for the gains over the phase screen. The implemented solver is ptimised using the diagonal approximation of the Hessian as shown in previous studies. We present a few simulations to illustrate the performance of the solver.
- Full Text:
- «
- ‹
- 1
- ›
- »