Design patterns and software techniques for large-scale, open and reproducible data reduction
- Authors: Molenaar, Gijs Jan
- Date: 2021
- Subjects: Radio astronomy -- Data processing , Radio astronomy -- Data processing -- Software , Radio astronomy -- South Africa , ASTRODECONV2019 dataset , Radio telescopes -- South Africa , KERN (omputer software)
- Language: English
- Type: text , Thesis , Doctoral , PhD
- Identifier: http://hdl.handle.net/10962/172169 , vital:42172 , 10.21504/10962/172169
- Description: The preparation for the construction of the Square Kilometre Array, and the introduction of its operational precursors, such as LOFAR and MeerKAT, mark the beginning of an exciting era for astronomy. Impressive new data containing valuable science just waiting for discovery is already being generated, and these devices will produce far more data than has ever been collected before. However, with every new data instrument, the data rates grow to unprecedented quantities of data, requiring novel new data-processing tools. In addition, creating science grade data from the raw data still requires significant expert knowledge for processing this data. The software used is often developed by a scientist who lacks proper training in software development skills, resulting in the software not progressing beyond a prototype stage in quality. In the first chapter, we explore various organisational and technical approaches to address these issues by providing a historical overview of the development of radioastronomy pipelines since the inception of the field in the 1940s. In that, the steps required to create a radio image are investigated. We used the lessons-learned to identify patterns in the challenges experienced, and the solutions created to address these over the years. The second chapter describes the mathematical foundations that are essential for radio imaging. In the third chapter, we discuss the production of the KERN Linux distribution, which is a set of software packages containing most radio astronomy software currently in use. Considerable effort was put into making sure that the contained software installs appropriately, all items next to one other on the same system. Where required and possible, bugs and portability fixes were solved and reported with the upstream maintainers. The KERN project also has a website, and issue tracker, where users can report bugs and maintainers can coordinate the packaging effort and new releases. The software packages can be used inside Docker and Singularity containers, enabling the installation of these packages on a wide variety of platforms. In the fourth and fifth chapters, we discuss methods and frameworks for combining the available data reduction tools into recomposable pipelines and introduce the Kliko specification and software. This framework was created to enable end-user astronomers to chain and containerise operations of software in KERN packages. Next, we discuss the Common Workflow Language (CommonWL), a similar but more advanced and mature pipeline framework invented by bio-informatics scientists. CommonWL is supported by a wide range of tools already; among other schedulers, visualisers and editors. Consequently, when a pipeline is made with CommonWL, it can be deployed and manipulated with a wide range of tools. In the final chapter, we attempt something unconventional, applying a generative adversarial network based on deep learning techniques to perform the task of sky brightness reconstruction. Since deep learning methods often require a large number of training samples, we constructed a CommonWL simulation pipeline for creating dirty images and corresponding sky models. This simulated dataset has been made publicly available as the ASTRODECONV2019 dataset. It is shown that this method is useful to perform the restoration and matches the performance of a single clean cycle. In addition, we incorporated domain knowledge by adding the point spread function to the network and by utilising a custom loss function during training. Although it was not possible to improve the cleaning performance of commonly used existing tools, the computational time performance of the approach looks very promising. We suggest that a smaller scope should be the starting point for further studies and optimising of the training of the neural network could produce the desired results.
- Full Text:
- Date Issued: 2021
Ecological assessment of a temperate river system using biomonitoring techniques: a case study of the Bloukrans River system, South Africa
- Authors: Magadze, Tinotenda
- Date: 2020
- Subjects: Bloukrans River , Ecological assessment (Biology) -- South Africa , Stream ecology -- South Africa , Rivers -- Environmental aspects -- South Africa , Benthic animals -- Effect of human beings on -- South Africa , Water quality biological assessment -- South Africa , Diatoms -- South Africa -- Data processing
- Language: English
- Type: text , Thesis , Doctoral , PhD
- Identifier: http://hdl.handle.net/10962/142933 , vital:38177
- Description: Escalating pressures from growing human populations and environmental impacts increasingly imperil freshwater ecosystems. The Bloukrans River, which drains an urbanised and agricultural catchment in the Eastern Cape province of South Africa, is no exception. Detailed knowledge of the structure and function of the aquatic ecosystems is required in order to create models and matrices that predict, guide assessment and direct intervention on ecological integrity and water quality management in these systems. The main objectives of this research were to: examine the effects of urbanization on the benthic macroinvertebrate functional feeding guild structure among different stream orders; determine if benthic diatoms can be used as effective and reliable indicators of ionic composition and conductivity in different stream order categories and finally; to evaluate the applicability of the South African Diatom Index (SADI) and other indices in the Eastern Cape region of South Africa. Field studies were carried out to explore temporal patterns in community structure (macroinvertebrates and diatoms) and ecosystem function related to land-use patterns, instream habitat availability, and water quality parameters in the Bloukrans River system across four study periods: February (summer) and July (winter) 2016 and February (summer) and May/July (winter) 2018. The study was conducted along a gradient of impacts from less impacted agricultural headwaters to highly impacted urban sites located immediately downstream of the city of Makhanda. Macroinvertebrates were separated into functional feeding groups (FFGs) (i.e. collector-gatherer, collector-filterer, scraper, shredder, and predator) which were then used to assess the effects of selected physico-chemical variables and riparian zone condition on FFG organization. Collector-gatherers were the most abundant in the Bloukrans River and represented 71.3 % of the macroinvertebrate assemblages. Stream order 1species such as Nitzschia palea, Gomphonema parvulum, Tryblionella apiculata, Diploneis vulgaris and Staurosira elliptica. Multivariate analysis (Canonical correspondence analysis (CCA)) indicated that differences in diatom community assemblages were best explained by calcium, magnesium, pH, phosphate, nitrate, dissolved oxygen, sediment nitrate, conductivity and salinity. These results indicate that diatoms can be used as bioindicators for monitoring highly impacted river systems and to also further examine pollution gradients and impacts of specific/point pollution sources. In order to further test the application of diatom indices, nine sites with contrasting water quality were sampled along the length river system in February, May and July 2018. Diatom-based indices incorporated in OMNIDIA software were applied to assess the integrity of the water quality as indicated by diatom communities. For comparative purposes, several foreign indices (e.g. the trophic diatom index (TDI), the percentage pollution-tolerant valves (%PTV), biological diatom index (BDI)) and the South African Diatom Index (SADI) were used in the study. From the results, the Percentage Pollution-Tolerant Valves (%PTV) of most urban sites in the Bloukrans River was above the 20% limit indicating the presence of organic pollutants. Although the foreign diatom indices were applicable in the study, the SADI had significant correlations with most water quality variables (p < 0.05) compared to other indices such as Watanabe Index (WAT), Biological Index of Water Quality Trophic Index (BIWQ) and Trophic Index (TI)). These results support wider use of the SADI as an indicator of water quality conditions in South African river systems. Finally, the observed variations in diatom community structure and composition as a result of changes in water quality were broadly in agreement with the results of macroinvertebrate FFG structure indicating that the two biological indicators can, and should, be used as complementary techniques in the biomonitoring of rivers and streams in South Africa.
- Full Text:
- Date Issued: 2020
Determining the hydrological functioning of the palmiet wetlands in the Eastern and Western Cape South Africa
- Authors: Smith, Caitlin
- Date: 2019
- Subjects: Wetlands -- South Africa -- Eastern Cape , Wetland ecology -- South Africa -- Eastern Cape , Wetland management -- South Africa -- Eastern Cape , Prioniaceae -- South Africa -- Eastern Cape , Prionium serratum -- South Africa -- Eastern Cape
- Language: English
- Type: text , Thesis , Masters , MSc
- Identifier: http://hdl.handle.net/10962/95956 , vital:31218
- Description: Wetlands provide a range of supporting, regulating and provisioning ecosystem services, including hydrological benefits such as flood attenuation and sustaining base flows. Despite their value, wetlands are one of the most vulnerable ecosystems in South Africa. Palmiet wetlands in the Eastern and Western Cape are particularly threatened. Palmiet (Prionium serratum) is a robust perennial plant that is endemic to wetlands and rivers located in the sandstones and quartzites of the Table Mountain Group (TMG), in the Eastern and Western Cape as well as the Natal Group sandstones in KwaZulu-Natal. Palmiet is described as an ecosystem engineer because of its ability to alter its environment and create large valley-bottom wetlands. The Krom River is an important water source for the city of Port Elizabeth and there has been a decline in palmiet wetlands along the Krom River as a result of alien vegetation invasion, agricultural activity, and gully erosion. Working for Water has been clearing alien vegetation and Working for Wetlands has been installing rehabilitation structures in the Krom River catchment for a number of years. There are, however, serious knowledge gaps in the understanding of palmiet wetland structure and function, particularly in respect of the hydrological functioning of these wetland systems. The aim of this study was to investigate the hydrology (surface and groundwater) behind these wetland systems. The investigation focussed on small-scale dynamics of the palmiet wetland system in order to increase general understanding of the surface water and groundwater processes of these wetland systems. Field work was concentrated on the Kompanjiesdrif and Krugersland palmiet wetlands in the upper K90A Krom River catchment. The investigation involved the installation of piezometers, water quality and stable isotope sampling and analysis, an Electrical Resistivity Tomography survey, and hydrological and mixing cell modelling. The results of the investigation indicate that the hydrological functioning of palmiet wetlands is closely linked with high sub-surface discharges typically associated with TMG aquifers. It is proposed that the palmiet wetlands are sustained by significant amounts of sub-surface water (both groundwater and interflow) moving through preferential flow paths in the alluvial fans and tributaries, which are in turn sustained by groundwater discharge from the surrounding sandstones and quartzites of the Nardouw Sub-group and Peninsula Formation. The palmiet wetlands clearly retain a significant amount of water, leading to the maintenance of prolonged flows, and a larger baseflow. However, it is hypothesised that the occurrence of palmiet as the dominant species in these wetlands is due to the sustained low flows related to catchment geology and high hydrological connectivity between the catchment and the wetland that is enabled by flow paths that allow the free flow of water from the catchment to the wetland. It is further proposed that palmiet is possibly more reliant on a consistent water supply for its existence and survival than it is on acidic nutrient-poor water and soils as stated by other authors.
- Full Text:
- Date Issued: 2019
Social movements and economic development in post apartheid South Africa: lessons from Latin America
- Authors: Makoni, Tinotenda Charity
- Date: 2019
- Subjects: South Africa -- Economic conditions -- 1991- , South Africa -- Politics and government -- 1994- , Social movements -- South Africa , Social movements -- Latin America , Economic development -- South Africa
- Language: English
- Type: text , Thesis , Masters , MCom
- Identifier: http://hdl.handle.net/10962/76420 , vital:30561
- Description: The aim of this research is to bring the literature on political agency and economics together in an analysis of whether social movements can play an important role in economic development in post-apartheid South Africa. The entrenched discourse of sluggish growth and high inequality in post-apartheid South Africa can largely be attributed to the political decision to implement a neoliberal economic development orthodoxy. On the one hand, there is an urgent need to shift the economic development model to an alternate developmentalist model. However, no clearly articulated alternative developmental model has emerged. As a result, economically, South Africa is seemingly stuck. On the other hand, the selection of an economic development model and change in macroeconomic policies requires a political shift. Politically, formal politics has assumed the form of neoliberal democracy, characterised by a largely centralised state and the usurpation of the state and institutions by a national bourgeoisie. Social movements have emerged in response to the failure of neoliberalism to fulfil the promises of early post independent periods. They have been largely successful at highlighting the injustices and the inequalities in the country. However their ability to influence structural economic development has come into question. Firstly, social movements and their “politically destabilising distributive demands” have faced repression from the state as the state and institutions are aligned behind the interests of capital under a neoliberal democracy. Secondly, social movements in South Africa have been largely ideologically under-developed. They have been largely fragmented and tended to contest specific single issues rather than aiming to shift the deeper underlying systemic drivers behind the symptomatic immediate discomforts. The economic dimensions of such a shift are particularly unclear. This fragmentation and apparent lack of economic pragmatism make management or suppression of disruptive movements by the state relatively easy. The research uses a contrast between the Latin American social movements against a South African background in order to see what lessons South Africa can draw from social movements in Latin America. The Latin American case is cautiously more positive and provides comparably more sanguine lessons. In this way, this research seeks to construct a more comprehensive framework for the further study of social movements in South Africa and their potential impact on economic development in South Africa.
- Full Text:
- Date Issued: 2019
Defining governance in Uganda in a changing world order, 1962-94
- Authors: Kintu-Nyago, Crispin
- Date: 1996
- Subjects: Uganda -- Politics and government , Uganda -- Economic conditions , World politics , International relations
- Language: English
- Type: Thesis , Masters , MA
- Identifier: vital:2788 , http://hdl.handle.net/10962/d1002998 , Uganda -- Politics and government , Uganda -- Economic conditions , World politics , International relations
- Description: This study argues that much of early post colonial Uganda's political developments, had its roots in the colonial patterns of governance. It was, however, the imperative of Uganda's early post colonial rulers to have formulated and maintained conditions for legitimate and orderly governance. Largely, this required a coherent political class with a mass based and mobilising political movement, that moreover had a political programme that catered for the interests of its support base. Indeed, their opting to negate these very prerequisite conditions, contributed greatly to Uganda's subsequent political disorder, and it's further marginalisation in the International Political Economy. This study suggests that since the impact of colonialism in Uganda, its governance policies have closely been linked to the broader dictates of the International Political Economy. A reality that the policy makers in post colonial Uganda should have realised, and in the process attempted to advantageously adapt to the Ugandan situation. Their was a qualitative improvement in Uganda's governance from 1986. This study illustrates that this was a result of the emerging into power of a political class, whose policies deliberately and strenuously attempted to fulfil the above mentioned criteria. Their is need to link Uganda's foreign and governance policies. Consequently a conscious and deliberate effort has to made by its policy makers, to ensure that the two are amicably adapted to each other, so as to derive the best possible benefits. For instance what Uganda needs in the existing New World Order are development, domestic and foreign investments and export markets for its produce. All of which can only be obtained if political order through a legitimate political system and government exists. With a leadership, that moreover, deliberately attracts foreign investments and creates the enabling conditions for competitive economic production. The onus is upon Ugandans to ensure that they institutionalise conditions for their appropriate governance and foreign policies. For this thesis argues that the International Political Economy is dynamic, and Uganda was never predestined to be at its margins.
- Full Text:
- Date Issued: 1996
A study of the personal literature written in the Eastern Cape in the nineteenth century
- Authors: Young, Cheryl Ann
- Date: 1995
- Subjects: Arkwright, Robert Wigram, 1822 -- Diaries , Armstrong, Frances -- Diaries , Bowker, Thomas Holden, 1807-1885 -- Diaries , Brownlee, James, 1824-1851 -- Diaries , Crealock, John North, 1837-1895 -- Diaries , Flanagan, Arthur Stephen -- Diaries , Hall, Henry, 1815 -- Diaries , Merriman, Nathaniel James, 1809-1881 Diaries , Pigot, Sophia, 1804-1881 -- Diaries , Stretch, Charles Lennox, 1797-1882 -- Diaries , Griffith, Patrick Raymond -- Diaries , Clergy -- South Africa -- Diaries , Diaries -- History and criticism , Soldiers -- South Africa -- Diaries , South African diaries -- History and criticism , Women -- South Africa -- Diaries
- Language: English
- Type: Thesis , Masters , MA
- Identifier: vital:2231 , http://hdl.handle.net/10962/d1002274 , Arkwright, Robert Wigram, 1822 -- Diaries , Armstrong, Frances -- Diaries , Bowker, Thomas Holden, 1807-1885 -- Diaries , Brownlee, James, 1824-1851 -- Diaries , Crealock, John North, 1837-1895 -- Diaries , Flanagan, Arthur Stephen -- Diaries , Hall, Henry, 1815 -- Diaries , Merriman, Nathaniel James, 1809-1881 Diaries , Pigot, Sophia, 1804-1881 -- Diaries , Stretch, Charles Lennox, 1797-1882 -- Diaries , Griffith, Patrick Raymond -- Diaries , Clergy -- South Africa -- Diaries , Diaries -- History and criticism , Soldiers -- South Africa -- Diaries , South African diaries -- History and criticism , Women -- South Africa -- Diaries
- Description: The evidence of these diaries, all written in the nineteenth century, reveals the heterogeneous nature of early settler society in the Eastern Cape. Generalizations can only be of the most tenuous kind in such a small sample; but women tend to dwell on the domestic, the men on their public lives, the most reticent about their private lives are the soldiers. There is one diary which can be described as personal; the diarists did not regard their diaries as appropriate repositories of their personal triumphs and failures. The perceptions formed in Britain about the land and people of Africa are not drastically modified upon arrival unless the diarist experiences a prolongued contact with either.
- Full Text:
- Date Issued: 1995