Identifying critical assumptions and risks in air quality management planning using Theory of Change approach

Governments across the


Introduction
Air is one of the most important environmental parameters in which all forms of life on Earth depend for survival in that it plays a vital role in several key processes that support life such as providing oxygen (O 2 ) for humans and organisms respiration (Imray et al., 2003).However, due to the human activities that introduce pollutants into the atmosphere, the ambient air quality has been gradually deteriorating over the years and has now become a major global human health, climate change, and environmental concern (Cohen et al., 2017;Roomaney et al., 2022;Sinha, 2018).Urban air pollution, and air quality impacts in particular, are a worldwide problem that affects different regions in different manners depending upon various aspects such as politics, economy, and technological landscapes of each particular region or country as well as on the nature of the available energy sources (Sinha, 2018).Rapid population growth, urbanisation, and industrialisation in recent decades have increased the introduction of air pollutants in the atmosphere which are associated with a range of acute and chronic effects on human health (Afros et al., 2003).Studies have found a direct correlation between emissions of carbon monoxide (CO), sulphur dioxide (SO 2 ), nitrogen oxides (NO X ), ozone (O 3 ), toxicants, and particulate matter (PM) with public health implications in both developing and developed countries (Dugard, and Alcaro, 2013;Henneman et al., 2016;Sinha, 2018).PM 10 is all particulate matter less than 10 micrometres in diameter, within which fine particulate matter PM 2.5 is a subcategory of less than 2.5 micrometres in diameter.According to Southerland et al., (2022), majority of the world's population was found to be living in territories with harmful PM 2.5 concentrations between 2000 and 2019 translating to noncommunicable disease burdens.According to the World Health Organization (WHO) 2016, approximately 91% of the world's population is living in areas where air quality levels exceed WHO limits.More to this, WHO indicates that seven million people die prematurely each year because of polluted air of which 600 000 are children under 5 years old.Poor air quality is the third leading cause of death after heart disease and smoking globally.
As a means of regulating air pollution and related impacts, governments in many countries have been developing legislation and policy implementation instruments to regulate and manage activities that contribute to air pollution (Kuklinska, et al., 2015;Mukwevho et al., 2022;Sheikh, 2020).
One of the widely adopted policy instruments is air quality management plans (AQMPs) (Feng et al., 2019;Gulia et al., 2015;Sivertsen and Bartonova, 2012;Gokhale, and Khare, 2007;Engelbrecht, and VD Walt 2007;Lieu, and Treyz, 1992;Miranda et al., 2015, Moreoane et al., 2021;Park, and Bae, 2006).These plans are defined as strategic instruments that describe past trends, the current state of air quality in a city or region and stipulate goals and objectives and describe short and long-term strategies, policies and controls to improve air quality within a city or region (Sivertsen, and Bartonova, 2012;DEA, 2012).Different countries use different terms to describe AQMPs such as air quality plans (AQP) (European Union member states), urban air quality management plans (UAQMPs) (e.g.India, UK, USA), air pollution prevention and control action plan (the action plan) (e.g.China) (Feng et al., 2019;Gulia et al., 2015, Sivertsen, andBartonova, 2012;Gokhale, and Khare, 2007;Engelbrecht, and VD Walt 2007;Lieu, and Treyz, 1992;Miranda et al., 2015;Moreoane et al., 2021, Park, andBae, 2006).The use of these terms depends on the spatial scale of a region which varies from national level, city level and site-specific level.An AQMP, therefore, provides measures, strategies, or interventions that will achieve air quality goals and objectives of a particular geographical area and for it to succeed it requires the involvement of various stakeholders from government, business, industry, non-profit or non-government organisations (NPOs or NGO's) (Sivertsen, and Bartonova, 2012).Governments must therefore prioritise air quality management and the development and implementation of efficient AQMPs as a key policy tool is needed to protect public health and preserve the environment.
Many developed countries such as USA, UK, and Australia have managed to develop and implement their AQMPs at different scales in accordance with their regulatory management frameworks (Gulia et al., 2015;Gulia et al., 2020;Sivertsen, and Bartonova, 2012;EEA, 2008;Tonne et al., 2008;Hasheminassab et al,. 2014).Recent studies suggest an improvement in some of the air quality parameters such as PM 2.5 and PM 10 concentrations in countries such as China, Korea, Japan, Spain, United Kingdom (UK) and Europe particularly at the regional or city level (Colette et al., 2020;De la Campa et al., 2018;Ito et al., 2021;Munir et al., 2021;Sicard et al., 2021).These improvements are the results of various policy interventions associated with their AQMP implemented by those countries in recent years.Cities including Paris, Sao Paulo, Mexico City, and New York have managed to address their vehicular emissions by implementing their AQMPs focusing on circulation restriction, fuel initiatives, technological advancements such as emission abatement technologies as well as fiscal incentive approaches in targeting fuel and technology initiatives (de la Campa et al., 2018;Slovic, and Ribeiro, 2018;Molina et al., 2019).Some trends emanating from megacities of these countries show that urban air quality does show signs of improvement as a result of proper implementation of AQMPs (Gulia et al., 2015;Gulia et al., 2020;Naiker et al., 2012).This was seen in the European Environment Agency (EEA) countries where vehicular emissions were shown to be significantly reduced from 1990 to 2009 for SO 2 , NO X , PM 10 and PM 2.5 (Gulia et al., 2015).Significant reductions of NO 2 were observed in Cardiff and Norwich cities after a successful implementation of UAQMP (Moorcroft, and Dore, 2013).Correia et al., (2013), found that efficient and effective implementation resulted in the reduction of PM 2.5 which led to an increase in life expectancy in 545 counties in the USA.
In contrast, low and middle-income countries have typically not yet been able to efficiently and effectively implement their AQMPs, particularly at regional or city level such as Beijing , Shanghai, Guangzhou, and Chongqing in China, and New Delhi, Mumbai, Calcutta in India, as well as in Priority Areas in South Africa.This is mostly due challenges such as poor regulator and stakeholder commitment and participation, weak legal framework and poor quality of air quality data and emission inventories among other things (Tshehla, and Wright, 2019;Gulia et al., 2015;Gulia et al., 2020;Naiker et al., 2012;Groundwork, and VEM, vs. minister of environmental affairs case, 2022).This has since prompted several studies to be conducted globally over the past few years to evaluate the effectiveness of environmental policy regulations and instruments including air quality interventions (Feng et al., 2019;Shakil, and Ananya, 2015;Wang et al., 2014;van Erp 2008;Retief et al., 2011;Jorquera 2021).

South Africa case example
Like many other countries, an AQMP in South Africa is a legal requirement in terms of the National Environmental Management Air Quality Act (NEM: AQA) no 39 of 2004.This Act requires each national department, province, and municipality to prepare and implement an AQMP.The Act further requires that those organs of state that prepare the AQMP must also report on the plan's implementation.The NEM:AQA further requires that once the AQMP is developed, the national and provincial environmental departments must incorporate the AQMP into other sector plans such as environmental management plans (EMPs) and environmental implementation plans (EIPs) which are required in terms of the National Environmental Management Act (NEMA) framework legislation (Naiker et al., 2012).Municipalities are also required in terms of the Municipal Systems Act 32 of 2000 to include an AQMP in the Integrated Development Plan (IDP) which is the municipal strategic planning document.
Despite having established AQMP as an approach within one of the world's most mature and complex air quality management legal framework (Mukwevho et al., 2022;Nel, and Alberts, 2018), South Africa is still facing serious air quality challenges relating to air pollution, especially in those areas that have been declared as priority areas (September 2012;Tshehla, and Wright, 2019).In addition, according to the Department of Forestry, Fisheries and Environment (DFFE, 2021a) since the promulgation of NEM: AQA in 2004, not all municipalities and provinces have successfully developed and implemented AQMPs with 34 of 44 district municipalities, 7 out of 8 metropolitan municipalities, and 7 out of 9 provinces having developed AQMPs in 2020 (DFFE 2021a;Tshehla, and Wright, 2019).Given this lack of AQMP development and implementation, a better understanding is required of the underlying challenges.This is typically achieved through policy instrument evaluations and identifying constraints including underlying assumptions and risks that underpin the design and implementation of the particular policy instrument (Mason, and Barnes, 2007;Vogel, 2012;Alberts et al., 2020).The latter has not been done in relation to AQMP in the South African context which then makes it a good case study for global lessons.This paper, therefore, aims to identify critical assumptions and risks as a basis to evaluate the effectiveness of air quality management planning system through the application of the Theory of Change (ToC) approach.Like in other governance instruments, identifying and evaluating assumptions and risks is crucial to ascertain whether the system operates as intended or if there are flaws in its design (Moolman et al., 2022).If the latter scenario occurs, it becomes imperative to re-evaluate the system's design by means of policy and legislative reform (Moolman et al., 2022).We propose using ToC approach in identifying critical assumptions because it will provide the basis for further performance evaluation on the effectiveness of AQMPs which is a crucial step toward understanding the issue of poor development and implementation of AQMPs in South Africa and perhaps other countries.

Theory of Change (ToC) method of evaluation
Evaluation research under the term theory of change evaluation has been applied previously in assessing the effects of various social programmes and has also recently been applied to governmental programmes including important evaluations of job training programmes, compensatory education, mental health centres, community health services, community action, law enforcement, corrections, and other government interventions (Alberts et al., 2021;Amundsen, and D'Amico, 2019;Archibald et al., 2016;Biggs et al., 2017;Connell, and Kubisch, 1998;Jackson, 2013;Thornton et al., 2017;Weiss, 1995).According to Biggs et al., (2017), ToC can be defined as an instrument that supports decision making by identifying the causal relationships and sequences of the events required for a programme to reach its intended outcomes and describing the key assumptions in each step of the process.ToC evaluation's interest is to apply logical thinking to reconstruct a causal model based on several sources to unpack how a specific intervention (programme) is intended to achieve its outputs and outcomes based on its inputs and activities (Stame, 2004;Rogers, 2008).
Theory of change approach exposes the assumptions underpinning how policy instruments are expected to function (Alberts et al., 2022;Alberts et al., 2021;Moolman et al., 2022;Amundsen, and D'Amico, 2019;Connell, and Kubisch, 1998;Vogel, 2012).Connell, and Kubisch, (1998), earlier described ToC as a systematic and cumulative study of the links between activities, outcomes, and contexts of the policy instrument by specifying how activities will lead to interim and longer-term outcomes and identifying the contextual conditions that may affect them.Furthermore, through ToC approach, assumptions are identified through a systematic process that also requires agreement between relevant stakeholders and practitioners in the particular field of interest and this approach has been recognised as a best practice performance evaluation method (Allen et al., 2017;Alberts et al., 2021;Connell, and Kubisch, 1998;Davidson, 2005;Stein, and Valters, 2012).
Internationally, several programmes including communitybased development programmes have also used ToC, with many international agencies considering ToC as a best practice evaluation method (Allen et al., 2017;Archibald et al., 2016;Biggs et al., 2017;Connell, and Kubisch, 1998;USAID, 2015).In addition, ToC approach was selected for this research due to its wide adoption as a policy evaluation method and its status as a mandated policy evaluation method in South Africa (Moolman et al., 2022;Alberts et al., 2022;Alberts et al., 2021;DPME 2011).
According to Brookfield (1995), assumptions are perceived as beliefs, expectations, or considerations taken for granted about how the world works.Assumptions play a vital role in the success of any programme and so in order to design a project with a good chance of success, there is a need to have consensus among the project designers on what they expect to be true in order to determine any gaps in the logic of the project and assess whether an assumption will, in fact, turn out to be valid or true (Nkwake, 2013;Kaplan, andGarrett, 2005, Weiss, 1995).Assumptions may be valid or invalid and there may only be a few that are critical among the many assumptions underpinning an intervention or a programme (van Es, and Guijt, 2015).This means that if these assumptions are not valid or true, the intervention will probably not work as planned and therefore it is essential to identify which assumptions are most critical to the success of the intervention (van Es, and Guijt, 2015).These assumptions also pose risks to the effectiveness of the policy instrument as well as the extent to which it can be expected to achieve its intended outcomes (Moolman et al., 2022;Archibald et al., 2016;DPME 2011).Accordingly, this paper aims to identify critical assumptions and risks to the effectiveness of AQMP development and implementation in South Africa through the application of ToC.

Overview
This study adopted a method used in several international and local studies in applying the ToC approach to identify critical assumptions and risks to the AQMP system as a policy instrument in South Africa (Alberts et al., 2022;Durant et al., 2022;Moolman et al., 2022;Alberts et al., 2020;Alberts et al., 2019;Archibald et al., 2016).The application of the ToC approach allows for an existing policy instrument to be broken down by critically analysing and understanding the causal linkages of all the steps involved in the development and implementation of that policy instrument (DPME,211).This is then presented in the form of a causal narrative and visual ToC map (Figure 2).The South African AQMP development and implementation system is broken down into six different ToC components (i.e.design, inputs, activities, outputs, outcomes (immediate and intermediate), and impacts following the ToC approach of other key studies (Alberts et al., 2022;Moolman et al., 2022;Alberts et al., 2022;Thorton et al., 2017;DPME 2011;Weiss, 1995;Stein, and Valters, 2012;Romero, and Putz, 2018).These ToC evaluation components are generally conceptualised and designed based on the so-called results-based pyramid (Alberts et al., 2022;Moolman et al., 2022;Alberts et al., 2021;DPME, 2011).
The results-based pyramid illustration provides the key questions asked in each of the six components of the ToC framework and this results-based pyramid is adapted in this study for the evaluation of the AQMP policy instrument as shown in Figure 1.The ToC developmental steps (outlined in steps 1-7 below) were conducted based on the results-based pyramid questions which then informed the conceptualisation and development of the ToC map and their causal narrative.

Development of the ToC map and causal narrative:
Theory of Change for any policy instrument needs to be done through agreements with relevant stakeholders and practitioners in that field to unpack the complexity of such an instrument (Allen et al., 2011;DPME 2011).This was achieved in this study through workshops with stakeholders in which presentations were made by the researchers and questions from the results-based pyramid (Figure 1) were posed.This was followed by feedback sessions where stakeholder participants provided inputs on the ToC map and identified assumptions.Criteria for stakeholder selection were based on experience, knowledge, and direct involvement in at least one of the South African AQMP process.The following steps were followed (full inclusion criteria and workshop details given in Appendix 1): Step 1: South African Specialist Workshop: The very first ToC conceptual map (see Figure 2) and key assumptions were developed in a workshop with five internal specialists at the North-West University (NWU) who are well-established in the application of ToC methodology to various sectors of environmental management.The initial conceptual ToC map was developed by brainstorming how the AQMP development and implementation process works as required by the legal framework of South Africa.Each step of the AQMP process was critically analysed, broken down and aligned with the six components of the ToC approach (i.e., design, inputs, activities, output, outcome and impact).Once the map had been agreed upon, the key underlying assumptions were identified for each of the six components.The AQMP ToC map and key assumptions were identified and developed by brainstorming based on the specialists' understanding of the South African air quality management and overall environmental legal framework.These participants were expert researchers and consultants in the application of ToC approach and other methodologies to policy evaluations in different disciplines of environmental sciences and management, including environmental impact assessments (EIAs), environmental auditing, water quality, and waste management.Together they had a combined experience of more than 50 years in the field of environmental management and sciences (Alberts, 2020;Alberts et al., 2019;Alberts et al., 2021;Moolman et al., 2022).
Step 2: Academics, consultants, and industry practitioners' workshop: Following the internal specialist workshop, three separate workshops were held during which the conceptual ToC map and the key assumptions were presented to different consultants and practitioners, including scientists, researchers, and practitioners in the industry (consulting firms, state-owned entities, mining, etc.) as shown in detail in Appendix 1.The following workshops were held: (1) Academics workshop with two senior lectures from well-recognised institutions in South Africa, (2) Specialist/consultant workshop with five senior environmental consultants from private consulting firms, (3) Industry workshop with five environmental practitioners/ officers from well-recognised organisations.The attendees were asked questions from the pyramid and were also asked whether they agreed with the assumptions presented to them and if they had anything to add.These participants were selected based on their knowledge, experience, and/or involvement in some part of AQMP development and/or the implementation process, either from a research, consultation, or monitoring and compliance point of view.The comments and additional assumptions from these workshops were then used to further refine the ToC map and assumptions.Some of the comments and assumptions raised in these workshops included issues related to the implementation of the AQMPs.
Step 3: Regulator workshop: Following the consultants' and practitioners' workshops, the ToC map and assumptions were presented to regulators represented by 11 government officials for refinement and to obtain a regulator's input.Representatives from various spheres of government were invited, including national, provincial, and local departments and municipalities (including the Department of Forestry, Fisheries and the Environment (DFFE), Gauteng Department of Agriculture and Rural Development (GDARD), City of Johannesburg; Knysna local municipality, Limpopo Department of Economic Development, Environment and Tourism (LEDET).Each of these officials had experience in the development and/or implementation of an AQMP as a regulator and many of the comments and assumptions raised in the workshop included concerns about the transparency of the AQMP legal framework and issues related to development and implementation.
Step 4: General stakeholder discussions: These engagements were conducted following the input from the previous workshops.Participants in these discussions included three members of non-profit organisations (i.e., Vaal Environmental Justice Group (VEJA) and Vukani Environmental Justice Movement in Action).These organisations are active and influential environmental justice organisations in the Vaal Triangle Airshed Priority Area (VATAPA) and Highveld Prioty Areas (HPA) which are among the air quality priority areas in South Africa.
Step 5: To ensure a broader perspective on the history of air quality legislation in South Africa and from a global point of view, a meeting was held with an ex-government official who had been involved in the initial development of air quality legislation and guidelines in South Africa and is now based abroad.They provided some background on the development of NEM:AQA.This provided valuable context for fine-tuning the design component of the ToC as it relates to how the South African air quality legislation is structured and its intent for AQMP development and implementation.
Step 6: National Association for Clean Air (NACA) Conference.An earlier version of this paper was submitted and presented at the NACA conference on 6-9 October 2021.Several comments and questions were used to further refine the ToC map and assumptions.The paper was also accepted and formed part of the conference papers published online (Mukwevho, and Burger, 2021).
Step 7: UK Air quality professional perspective.In October 2022 a meeting was held with air quality official from the Newcastleunder-Lyme Borough Council to discuss the assumptions and to get a UK perspective on the effectiveness of a UK's air quality action plans.
A ToC can be explained using different methods, such as plain narrative description, causal loop diagrams, logic models and results chains (Senge, 1990;Knowlton and Phillips, 2012).After following the seven steps the development of the ToC map, narrative and identification of critical assumptions, this study went on and conducted a literature review to identify existing literature that corresponds to the identified assumptions.This was done following a similar data collection approach by Mukwevho et al., (2022); and Olagunju et al., (2019), through a broad systematic search of popular academic electronic databases such as Google Scholar, and Scopus.Other nonpeer-reviewed literature such as relevant legislations and reports were located from general Google searches, textbooks, and specific databases such as government databases, the Centre for Environmental Rights (CER) library database etc.In doing so keywords were used in the search such as "air quality management plans", "AQMP Assumptions", "South African air quality management", "air pollution risks in south Africa".
At the end of conducting the above steps, the following were the outcomes of the workshops: • ToC conceptual framework or map (Figure 2); • causal narrative description as they relate to the 15 key assumptions and; • their associated risks (see Table 2) underpinning the AQMP development and implementation which are discussed in this section.

ToC map, causal narrative, and critical assumptions
Based on this causal logic this study suggests that the following narrative underpins the AQMP development and implementation process in South Africa which is also summarised in Table 1 and discussed in detail under the Causal Narrative description section: "The South African AQMP development and implementation process is fundamentally prescribed in legislation (design component).However, for any sphere of government to be able to develop and implement a plan, various resources are required as inputs to the process (input component) and these include time, money, data and information, infrastructure, skills and competencies.In this process, there are six steps to be followed which are prescribed in the manual for air quality management planning guidelines (activity component).An output from the six steps is a detailed final AQMP report (output component) which must be approved, gazetted (only for priority areas) and be incorporated into other sector plans including the IDP, EMP and/ or IEP (immediate outcome).Through effective implementation of the intervention strategies in the plan, it is expected that the specific geographic area should be able to meet the National Ambient Air Quality Standards (NAAQS) requirements as well as give effect to the NEMA: AQA requirements (intermediate component).Ultimately over the long run, the AQMP should be able to progressively give effect to the environmental right contained in Section 24 of the Constitution (impact component)."

Causal narrative description
This section provides a further detailed description of the ToC causal narrative of Table 1 and Figure 2 relating to the AQMP development and implementation system:

Design Component
In agreement with various stakeholders, the following critical assumption was identified relating to the design component of AQMP process for South Africa: • Assumption 1: It is assumed that the legislative framework provides guidance towards AQMP development and its implementation.
The ToC map begins with the design component which in terms of the results-based pyramid (Figure 1) is guided by the question "What is the legal mandate of the AQMP development and implementation process?".This component refers to the manner in which the AQMP development and implementation process is prescribed in South Africa by (a) the constitution, (b) NEMA framework legislation, (c) NEM: AQA sectorspecific legislation, (d) the national framework for air quality management, (e) technical guiding documents: the manual for air quality management planning, (f) other legislations directly or indirectly informing the process such as Municipal Systems Act, PAJA and PAIA, and (g) Municipal by-laws (see design component in Figure 2).a.Based on inputs from stakeholders during workshops and a review of existing literature (Naiker et al., 2012;Scorgie, 2012;DEA, 2012, and2018) this study suggests that the legal and other conditions that inform the process of AQMP development and implementation begin with section 24 of the constitution which provides that the government must use legislative and other means to ensure a human right which is the progressive realisation of an environment that is not harmful to health and wellbeing improve air quality and progressively ensure that ambient air is not harmful to health and well-being by preventing pollution (including air pollution) and ecological degradation.Section 24(b) requires the enhancement of ambient air which will enable an environment that is not harmful to the people of South Africa (Scorgie, 2012).b.The constitution then translates into the National Environmental Management Act (NEMA) framework legislation in which its purpose is the provision of cooperative environmental governance and provides principles for decision-making on environmental matters (DEA, 2018).Some of the key principles in this act that relate to air quality matters include polluter pays principle, pollution prevention or minimisation as well as the  promotion of participation of all interested and affected parties in environmental governance.Furthermore, NEMA regulations outline the EIA process including public participation that must be followed during the application of atmospheric emissions licences for the listed activities in terms of NEM:AQA c.The next tier in the design component is the sector-specific NEM: AQA which is the main legislation governing air quality in South Africa.In terms of chapters 3 and 4 of this act, each national department, province, and municipality must prepare an AQMP which is the document that sets out what will be done to achieve the prescribed requirements of NEM: AQA as well as the air quality standard.The objectives of this act are:

AQMP Process Activities
There are generally six steps that should be followed to develop and implement an AQMP as prescribed in the 2012 manual for air quality management planning (see Figure 3) According to the Manual, six steps need to be followed in the development and implementation of the AQMP as discussed later in the activity component (see Figure 3).f.
There are several other pieces of legislation that directly or indirectly impact on the implementation of the AQMPs (Mukwevho et al., 2022)

Input component
The following are the critical assumptions relating to the input component: • Assumption 2: Resources are available to develop and implement the AQMP.
The input component describes the resources required for the process to be effectively executed (Moolman et al., 2022;Alberts 2020;Alberts et al., 2021;Romero, and Putz, 2018;Thorton et al., 2017;Weiss, 1995).In terms of the results-based pyramid, the key question for the input component, in this case, asks the question "What inputs are required to implement the AQMP?".Once again from the stakeholder engagements, it was evident that key resources required for the development and implementation of an efficient and effective AQMP include data and information, human capacity, skills and competencies, time, money/budget, and infrastructure.Time: Section 3.4.1 of the Manual (Task 1 and 2: Intervention Strategies and Action Plan) requires the AQMP to propose generic achievable timeframes for achieving the set intervention strategies ranging from short-term (1-2 years), medium-term (3-5 years) and long-term (5-10 years).In addition, the Manual requires that the AQMP once developed needs to be revised every 5 years.

Skills and competencies:
Financial/Budget Capacity: This is the description of the budgetary needs to see through the entire AQMP development and implementation process.According to section 4.1.2 of the Manual, the AQMP must include an estimation of the expected costs and benefits of the intervention strategies.
The infrastructure: this is required for the development and implementation of AQMP as stipulated by the Manual includes the South African Air Quality Information System (SAAQIS) database in which relevant monitoring information can be obtained (i.e., location of monitoring stations, monitoring data, emission inventory, atmospheric dispersion modelling, and site access).

Activity component
Critical assumptions relating to the Activity component identified are: • Assumption 4: Stakeholders are established and are actively involved in the assessment or AQMP process.
• Assumption 5: A thorough baseline air quality assessment is done using current and relevant information and is sufficient to inform the gap and problem analysis.
• Assumption 6: Gap and problem analysis is done.
• Assumption 7: The intervention strategies and action plans are technically and economically feasible and are indeed implemented.
• Assumption 8: The intervention strategies are sufficient to achieve ambient air quality standards.
• Assumption 10: Monitoring, reporting and evaluation of the AQMP are done.
The activity component is the third component of the ToC map and in terms of the results-based pyramid, the key question asked for this component is "What are the steps required to develop and implement the AQMP?".There are generally six steps that should be followed to develop and implement an AQMP as shown in Figure 3 (DEA, 2012;Sivertsen, and Bartonova, 2012).These steps are: (1) Establishment of stakeholder groups, defining the boundaries of the AQMP geographic area and the establishment of a baseline; (2) Gap and problem analysis; (3) Establish air quality goals; (4) Develop interventions and a plan to achieve air quality objectives; (5) Implementation of the intervention strategies; and (6) Monitoring, reporting and evaluation as outlined in the national framework for air quality management planning (DEA, 2012(DEA, , 2018): • (1) Establishment of stakeholder groups, defining the boundaries of the AQMP geographic area and the establishment of a baseline assessment report.This activity entails establishing the different committee groups as well as the establishment and issuing the draft air quality baseline report.
• (2) Gap and problem analysis: Stakeholders and the technical committees are consulted to evaluate the degree to which the baseline assessment is complete and allows for a clear understanding of air quality and impacts.Once the gaps are identified, the committee should initiate a problem analysis to determine the problems, associated causes of the problems and the effects.The gap and problem analysis should be documented as part of the AQMP, building on the baseline assessment section.
• (3) Establishing air quality goals: Based on the draft report and problem analysis, the project steering committee (PSC) and technical committee / advisory forum must meet and ratify a vision, mission as well as air quality management goals for the AQMP.Then a meeting should be scheduled through an invitation with the broader air quality stakeholder group.Thereafter a draft AQMP document as well as a decision on goals being met is made available to stakeholders.
• (4) Development of interventions and a plan to achieve air quality objectives: Intervention strategies for each of the problems identified is formulated by the PSC and technical committee / advisory forum.Once these intervention strategies have been identified, an action plan noting the implementation schedule should be tabled with the buy-in from stakeholders.Once agreed upon, the implementation plan is documented as part of the AQMP and submitted to relevant stakeholders for comments.
• (5) Implementation of the intervention strategies: After this stakeholder consultation and once comments have been incorporated and the document is finalised, an internal evaluation/review of the AQMP should be undertaken by the PSC.Once finalised, the AQMP is included in the IDP/ EMP/EIP.Implementation of the AQMP is implemented in a systematic manner based on the rules developed in the implementation strategy. •

Output component
Critical assumptions identified relating to the output component are: • Assumption 11: The AQMP report will address the gaps and problems identified, ensure successful implementation of intervention strategies, and ultimately ensure improvement of AQ in the airshed.
The output component in air quality management planning provides tangible results usually in reports and documents and the key question asked here is "What outputs do AQMPs produce?".The type of reports differs depending on the nature in which each plan was developed and documented.The following are some of the documents that are developed: • Information material for stakeholder involvement (Background Information Document, relevant announcement, advertisements, media releases, etc.)this document may be useful for public dissemination and posting on a web page.
• A comprehensive stakeholder database, comments, and response document (public participation document) could be helpful in effectively managing the public participation and communication process.
• Draft baseline assessment -this report covers amongst other the geography of the area (geographical boundaries, population, climate and other geographic information) and a description of the meteorology and climate; collecting and evaluating existing air quality information; Identifying sources and listing pollutants of concern; development of air quality management system (emissions inventory and dispersion modelling, and monitoring); and evaluation of current management and tools available.
• Gap and problem analysis document -the gap and problem analysis should be documented as part of the AQMP, building on the baseline assessment section.
• Draft AQMP -this document comprising the baseline assessment, gap and problem analysis, goals and implementation plan should be submitted as the draft AQMP.
• Final AQMP -after the stakeholder consultation and once comments have been incorporated and the document is finalised, the PSC should undertake an internal evaluation/ review of the draft AQMP to become the final AQMP.

Output components
Critical assumptions identified relating to the outcome component are: • Assumption 12: The AQMP is gazetted or included in the IDP/EMP/EIP and influences decision-making.
• Assumption 13: The goal of an AQMP is to bring ambient air into compliance with the ambient air quality standards.
The outcome component in AQM planning is divided into immediate and intermediate outcomes.The immediate outcome depends on the sphere/level at which the AQMP is done.In declared priority areas, the AQMP must be approved and gazetted by the relevant minister/MEC.AQMPs for municipalities must be included in the IDP and for other government departments it must be included in EMPs and or EIPs.The question asked in this component is "what immediate and intermediate outcomes does the AQMP deliver?".
According to Euripidou et al., (2022), the objective of NEM:AQA read in line with NEMA is to develop means to avoid air pollution and enhance air quality which then ultimately gives effect to several constitutional rights.To achieve this objective, one of the key regulatory instruments mandated by the act is the NAAQS which set out ambient pollutant limits for eight pollutants, including NO 2 , SO 2 , PM 10 and PM 2.5 (DEA, 2009, DEA, 2012;Euripidou et al., 2022).The objective of the AQMP as one of the instruments is therefore to give effect to NEM:QA and the constitution.The medium to long-term objectives stipulated in the act are therefore to:

Impact component
The critical assumption identified relating to the impact component is: • Assumption 15: AQMP enables a progressive realization of the environmental right contained in section 24 of the constitution.
The question asked in this component is "what impact does AQMP aim to achieve?" and that is the realisation of core human rights contained in section 24 of the Constitution (1996), which states that: "Everyone has the righta.to an environment that is not harmful to their health or wellbeing; and b. to have the environment protected, for the benefit of present and future generations, through reasonable legislative and other measures thati.
prevent pollution and ecological degradation; ii.secure ecologically sustainable development and use of natural resources while promoting justifiable economic and social development."

Discussion
In evaluation studies, assumptions can either be valid or invalid and can either be a resource or risk to the success of a programme, therefore, it is essential to determine assumptions underpinning an intervention or programme such as the AQMP (van Es et al., 2015;Archibald et al., 2016).If these assumptions are flawed or unfounded, the intervention/programme will probably not work as planned and it is thus important to do a risk analysis to determine which assumptions are most critical to monitor (van Es et al., 2015).Critical assumptions are therefore a good basis for risk management in that monitoring these assumptions allows for a timely response to new information, planning, and strategizing as well as deciding on the best strategic reactive steps to take in complex contexts/processes (van Es et al., 2015).
As a first step towards evaluating the effectiveness of AQMPs in South Africa, the aim of this study was through the application of the ToC approach, to identify critical assumptions and underpinning AQMP development and implementation in South Africa.This study is limited only to the identification of the assumptions and did not evaluate or test the validity of these assumptions.Further to this, this study conducted a literature review to identify risks associated with these critical assumptions as shown In Table 2.The study resulted in the identification of 15 underlying critical assumptions within different components of the ToC map (i.e.design, input, activities, output, outcome, and impact components, see Figure 2).These assumptions are considered to be critical ones although this study acknowledges that they are not the only ones underpinning AQMP development and implementation in South Africa.However, the ones identified in this study are found to be the fundamentally important ones that may have a significant impact on the success of developing and implementing AQMPs as a policy instrument in South Africa.
Table 2 shows the 15 assumptions identified and their associated risks toward the successful development and implementation of AQMP in South Africa.In addition, the table also provides a link between the identified assumptions and risks from existing local and international literature in the air quality sector.The literature review references, therefore, supplement the argument in this study that these assumptions and risks identified are indeed critical and could be having a significant impact on AQMP development and implementation.It is important to highlight a.To protect the environment by providing reasonable measures fori.
the protection and enhancement of the quality of air in the Republic; ii. the prevention of air pollution and ecological degradation; and iii.securing ecologically sustainable development while promoting justifiable economic and social development; and b.To give effect to section 24(b) of the Constitution in order to enhance the quality of ambient air for the sake of securing an environment that is not harmful to the health and wellbeing of people.DFFE, 2021a;DFFE, 2021b Engelbrecht, andvan der Walt, 2007;Engelbrecht, and van der Walt, 2007;Gulia et al., 2015;Moreoane et al., 2021;Naiker, 2007;Naiker et al. 2012;Scorgie, 2012;Sivertsen, and Bartonova, 2012); Tshehla, and Wright, 2019;Roomaney et al., 2022.Molman et al., 2022;Alberts et al., 2022 3 Cooperation exists: Cooperative governance exists between government stakeholders Poor communication and coordination within municipalities and departments resulting in poor implementation of the AQMPs.
In assumption 1, it is assumed that the current legislative framework in South Africa as described in the design component of the ToC provides sufficient guidance toward AQMP development and implementation.However, literature shows that not all spheres of government have developed and implemented their AQMPs since the promulgation of NEM:AQA in 2004.Two provinces, 34 district municipalities and many more local municipalities still have no AQMPs in place (DFFE, 2021a and2021b;Tshehla, and Wright, 2019).There is, therefore, a need for further research relating to the evaluation or testing of this specific assumption.
Literature review shows that assumption 2 which states that resources are available to develop and implement the AQMP, is already proving to be flawed in practice in that many municipalities in South Africa and elsewhere still cannot effectively implement AQMPs due to the lack of budget, skills and competencies, stakeholder commitment, weak policies, standards and regulations, as well as the absence of air quality data and emission inventories (DFFE, 2021a;DFFE, 2021b, Engelbrecht, andVan der Walt, 2007;Gulia et al., 2015;Moreoane et al., 2021;Naiker et al., 2012;Scorgie, 2012;Sivertsen, and Bartonova, 2012;Tshehla, and Wright, 2019).It is also clear from existing literature that there is currently a skills and competency gap especially in government towards to AQMP system in South Africa (DEA, 2018;Engelbrecht and van der Walt, 2007;Engelbrecht, and van der Walt, 2012;Naiker, 2007;Naiker et al., 2012).In addition, for an AQMP to be effectively and efficiently developed and implemented, the gap in technical capacity and skills such as engineering control, air quality modelling, identification of sources and emission quantification as well as air pollution risk assessment more so in local municipalities is a matter that requires urgent prioritisation (Naiker et al., 2012;Engelbrecht, and van der Walt, 2007;Engelbrecht, and van der Walt, 2012).
Moreover, according to an international research-based practical guide to the principles and steps to developing AQMP, the AQMP development and implementation process should include inputs from various role players including industry, groups and individuals (Sivertsen, and Bartonova, 2012).Naiker et al., (2012), also contend that partnerships and cooperative governance are important contributors to South Africa's current governance setup and should be encouraged to drive air quality management interventions.This study, therefore, suggests that the assumptions that cooperative governance exists between government (assumption 3) and that other stakeholders (such as the public) are established and are actively involved (assumption 4) are important to monitor and may also require further evaluation.An earlier study, however, found that air quality management in South Africa generally faces challenges such as a lack of political will, consultation, and communication as well as non-utilisation of existing planning tools (Naiker et al., 2012).A similar situation is also found in the water governance sector where it is found challenging to establish effective intergovernmental relations and co-operative governance in South Africa (Moolman et al., 2022;Meissner et al., 2016;Bourblanc, and Blanchon, 2014;Colvin et al., 2008;Bourblanc 2012).Considering at the manner in which South Africa's air quality governance is structured with the delegation of key functions to local government, there is therefore a need for further research to be conducted to explore the validity of these assumptions at a bigger scale pertaining to the AQMP policy instrument in South Africa.This will give a clear picture of the situation within the different spheres of government in South Africa.
As an output of the ToC component, good quality baseline assessment and AQMP reports should have clear objectives, factual and truthful information, policies, public participation as well as be clear in the implementation and monitoring interventions (Berke, and Godschalk, 2009;Lyles, and Stevens, 2014).This then suggests the importance to evaluate and monitor the assumptions (5, 6, and 11) that in the current AQMP system, thorough baseline air quality assessments are done using reliable, current, and relevant information to inform the gap and problem analysis.This could lead to a risk that intervention strategies and implementation plans in those AQMPs could be based on inaccurate and misleading information which then makes them unrealistic to implement.This study suggests that similar to the EIA reports, quality becomes an important component to effectiveness since the extent to which the plan achieves its objectives is based on adherence to its procedural requirements and substantive purpose (Sadler, 2012;Sandham et al., 2013).However, it has been found in South Africa's water use licence application (WULA) system that poor quality reports based on weak impact assessments have played a role towards an ineffective system in South Africa (Moolman et al., 2022).This study argues that evaluating the completeness and substance quality of AQMP reports can help in providing clarity on whether these plans operational mechanisms are realistic or not towards meeting their own objectives.
Assumptions 7 and 9 assume that the intervention strategies and action plans are technically and economically feasible and implemented.The risk associated with this is that poor intervention strategies and implementation plans may not be implementable thus no improvement in ambient air quality.A similar study that evaluated the quality of the first and second Vaal Triangle Airshed Priority Area Air Quality Management Plans has found that there are some gaps in the intervention strategies despite the reports being found to be satisfactory overall (Moreoane et al., 2021).
Furthermore, it is assumed that effective monitoring, reporting and evaluation is done in the AQMP development and implementation process (assumption 10).The risk associated with this is a poor or no understanding of the AQMP performance on (1) the extent to which the AQMP is implemented; (2) air quality management initiatives; (3) compliance of the AQMP to the applicable standards; and (4), how the area performed in achieving the targets.A recent study by Roomaney et al., (2022), however, suggests that reducing the levels of air pollution emissions is directly proportional to the reduction of noncommunicable and infectious diseases and there is a need to increase efforts to have a comprehensive system of monitoring stations measuring ambient air quality to gather accurate and reliable information in South Africa.
Moreover, once the AQMP has been developed it needs to be gazetted if it is at the priority area level, or included in the EIP and/or EMP if it is a provincial level, and included in the IDP if it is at the municipal level as required by NEM: AQA and the Municipal Systems Act as per assumption 12 (DEA, 2018).Engelbrecht, and van der Walt, (2007), however, earlier found that only a few municipalities had included their AQMPs in their IDPs and 16 years later it remains unclear the extent to which the existing AQMPs have been incorporated in their IDPs.Again, further research is required to address this assumption.
In the end, the goal of AQMPs as a policy instrument in South Africa is to give effect to chapter 2 of NEM: AQA which ultimately seeks to realise the human environmental right stipulated in section 24 of the constitution (assumptions 13, 14 and 15).Such objectives and targets aim to reduce air pollution by identifying and mitigating anthropogenic activities that cause negative impacts on human health and the environment through compliance with the NAAQS.The risk associated with these assumptions is the ineffectiveness of the AQMP as a policy instrument in achieving its mandated objectives to bring the ambient air into compliance with the ambient air quality standards, to help realise the intended objectives of the NEM: AQA as well as lead to a progressive realisation of the environmental right prescribed in section 24 of the constitution.
A similar study has found the protected areas system policy instrument to be effective in contributing positively to an environment that is not harmful to our health and well-being, as well as realising the rights encapsulated in Section 24(b) in South Africa and this remains to be tested for AQMPs in South Africa (Alberts et al., 2022).

Conclusion
In conclusion, this study adopted the ToC approach to identify 15 critical assumptions underlying the AQMP development and implementation system in South Africa.A further literature review was conducted to identify the risks associated with these critical assumptions.This study was only limited to identification and did not evaluate each of the assumptions and risks.This can be regarded as a first step towards a better understanding of the effectiveness of air quality management plans in South Africa.Although not the only assumptions, those that have been identified through the Theory of Change approach are considered to be fundamental to the development and implementation of AQMPS as a policy instrument in South Africa.
The findings in this paper suggest that the current AQMP development and implementation process could be fundamentally based on flawed assumptions which may be the underlying factors for current challenges in municipalities being unable to develop and implement their AQMP since the promulgation of the NEM:AQA in 2004.In addition, evidence is found in the literature proving that some of these assumptions are indeed flawed and already manifesting in practice and are therefore a risk to the effectiveness of the current air quality management planning system as a policy instrument.It is however suggested that to get a clearer view of these challenges, future research should be conducted to test the validity of all these assumptions in order to fully provide a fundamental understanding of how AQMP process works in South Africa and elsewhere.This will also provide solutions for avoiding or mitigating risks associated with these assumptions.Some of the assumptions and risks identified in this study such as resource availability, public participation and cooperative governance have also been identified in other environmental governance systems.With the ongoing increase in air pollution challenges globally and locally, the assumptions and risks associated with new and existing air quality policy instruments such as the AQMP must be effectively managed for these policies to effectively achieve their intended objectives which are ultimately aimed toward realising an environment that is not harmful to the health and well-being of humans.

Note
An earlier version of this paper was presented at the National Association of Clean Air (NACA) Conference in October 2021 and was published in its Proceedings.

Theory of Change approach limitations
A limitation of the ToC approach is that the method is based on the causal-effect assumption.Meaning that the method assumes that if one of the components is correctly executed then the following step or component will also happen.

Figure 1 :
Figure 1: ToC results-based pyramid for the AQMP process as adapted from the DPME, 2011 and Alberts et al., 2021.

Figure 2 :
Figure 2: This figure hows that there are six components that are linked as shown by the black dotted arrows.The red dotted circles indicate the 15 critical assumptions relating to the AQMP development and implementation process in South Africa.These critical assumptions are shown by the red dotted lines and explained in detail in the sections below.A summary of the causal narrative is provided inTable 1 and discussed in detail in the causal narrative description section.

Table 1 :
Summary of the 6 ToC components as well as what informs each component pertaining to the AQMP development and implementation process in South Africa.
. Establishment of stakeholder groups, defining the boundaries of the AQMP geographic area and the establishment of a baseline; 2. Gap and problem analysis; 3. Establish air quality goals; 4. Develop interventions and a plan to achieve air quality objectives; 5. Implementation of the intervention strategies; and 6.Monitoring, reporting and evaluation as outlined in the national framework for air quality management planning(DEA, 2012(DEA,  , 2018) ) of the Constitution in order to enhance the quality of ambient air for the sake of securing an environment that is not harmful to the health and well-being of people.d.In addition to the air quality management plans being prescribed by NEM:AQA, a further detailed description of the development and implementation of AQMPs are outlined in the 2017 national framework for air quality management in the Republic of South Africa which was developed by then the Department of Environmental Affairs (DEA; now DFFE) (DEA 2018).The national framework is considered as the national air quality management plan for the republic and its mandate is to achieve the objectives of NEM:AQA by providing mechanisms, systems, and procedures for the management of air quality in a holistic and integrated manner.This includes the provision of guiding norms and standards relating to all technical aspects of air quality management.e.The national framework further provides that the development of AQMPs by the various spheres of government must be done in accordance with the process stipulated in the Manual for Air Quality Management Planning published in 2008 and 2012 (henceforth referred to as "the Manual" in this study)(DEA, 2012).The manual for the AQMP development in South Africa was developed and published by then the DEA to guide all spheres of government to establish best practice guidelines on the definition of objectives, strategies, plans, and procedures to meet the requirements of the NEM:AQA on air quality management planning and reporting(DEA, 2012).The Manual aims to improve and harmonise the quality of AQMPs produced by various spheres of government.

Table 2 :
Critical assumptions, key risks and implications on the effectiveness of air quality management planning as well as the relevant existing literature. .