Rapid Review: What is the effectiveness of public health interventions on reducing the direct and indirect health impacts of wildfires?

Executive Summary

Background

Exposure to wildfires and their smoke has a well-established impact on the health of populations, from direct physical and mental health implications to more indirect environmental and socioeconomic consequences (Hadley et al. 2022; Liu et al. 2015; Reid et al. 2016). The role of public health in responding to and mitigating these risks varies across jurisdictions, but typically involves multi-sectoral collaboration, community engagement, and the use of diverse sources of evidence.

Public health’s role in the emergency management of wildfires falls within the components of prevention and mitigation, preparedness, response, and recovery (Government of Canada 2023). Smoke forecasting and air quality assessment may be used to identify risks and inform public health’s decision making in issuing recommendations; effective public messaging is then critical for communicating these risks and recommendations, leading to intended behaviour change (Yao 2014; Fish et al. 2017).

Specific actions and interventions to respond to and reduce the negative impacts of wildfire smoke events have included establishing air quality monitoring systems and cleaner air shelters (i.e., indoor areas where the concentration of contaminants from outdoor smoke is reduced due to limited infiltration and by using air filtration devices), providing masks or recommending their use, issuing evacuation orders, and offering mental health and socioeconomic supports and resources (Government of Canada 2023; Maguet 2018). The evidence on the effectiveness of these interventions, however, is less understood (Cascio 2018).

As the frequency, severity, and duration of wildfire events increases worldwide (WHO 2023), the burden on public health organizations to implement interventions and measures to mitigate impacts will undoubtedly increase. This rapid review seeks to identify, appraise, and summarize available research evidence to support evidence-informed decision making in public health. It was produced through a collaboration between the National Collaborating Centre for Methods and Tools (NCCMT) and the National Collaborating Centre for Environmental Health (NCCEH).  

This rapid review includes evidence available up to July 27, 2023 to answer the question:

What is the evidence for the effectiveness of public health interventions, and their potential unintended consequences, to reduce the direct and indirect health impacts of exposure to wildfires, including wildfire smoke and combined heat-wildfire smoke events?

Key Points

• Interventions for reducing the health impacts of wildfires and wildfire smoke included in this review comprise executing evacuation orders, staying indoors and improving indoor air quality, and communicating about outdoor air quality. The effectiveness of these interventions was assessed using both direct health (e.g., physical, mental) and indirect (e.g., smoke exposure reduction) outcomes.

• Community-wide evacuation orders may negatively impact mental health, including stress, anxiety, and depression. Access to care (e.g., healthcare, prescription medications, mental health supports) and information about the wildfires and evacuation orders may also be negatively impacted, potentially resulting in greater unmet need and uncertainty among evacuees. The certainty of evidence is very low (GRADE, see Methods for details); findings are likely to change as new evidence emerges.

• Evacuation orders have had varied impacts on different individuals: female evacuees may experience higher rates of anxiety and depression, and female evacuees and those with mental illnesses may have more confusion around and difficulty finding information related to the evacuation. The evacuation destination may also have a differential impact on anxiety. The certainty of evidence is very low (GRADE); findings are likely to change as new evidence emerges.

• The findings from qualitative studies corroborate the quantitative results. Negative consequences of evacuation experiences (e.g., fear, uncertainty/confusion, anxiety, financial losses) are reported more frequently than positive outcomes. These consequences may have enduring and varied impacts on different populations, stressing the importance of communication and preparedness. Relocating evacuees together may offer opportunities to strengthen cohesion, altruism, and support within the community. The confidence of evidence is moderate (GRADE-CERQual); findings are likely a reasonable representation of the phenomenon of interest.

• Most of the studies reporting on indoor air quality included air filtration interventions, highlighting findings for both unfiltered and filtered conditions. Indoor air quality is likely impacted by outdoor air quality: when outdoor air quality is poor, unfiltered indoor air quality is also likely poor. The amount of infiltration may depend on factors related to the indoor environment (e.g., building type, number of windows, air exchange rate, occupant behaviours). Remaining indoors, without air filtration, likely does not provide sufficient protection during smoke events, particularly for high-risk populations. The certainty of evidence is low (GRADE); findings may change as new evidence becomes available.

• Filtering, or cleaning the air (e.g., by using filters on central heating, ventilation, and air conditioning (HVAC) systems or portable air cleaners), likely reduces indoor particulate matter (PM₅) concentrations. There is some evidence that high-efficiency particulate air (HEPA) filter use may have a greater impact on reduction. The certainty of evidence is low (GRADE); findings may change as new evidence emerges.

• Wildfire smoke forecasting and air quality health indices may be predictive of respiratory health indicators, with implications for use as an early warning system and indicator for other interventions (e.g., evacuations). The certainty of evidence is very low (GRADE); findings are likely to change as new evidence emerges.

• The evidence is very uncertain about the impacts of other interventions on respiratory symptoms. In one multicomponent intervention – involving HEPA use, mask provision, public health messaging, and evacuation support – only HEPA use and recalling a public service announcement related to wildfire risk were associated with decreased odds of reporting worsening respiratory symptoms. In another mobile app intervention, combining air quality, smoke, wildfire, and symptom monitoring, it was unclear whether app use was more effective than control in improving asthma outcomes. The certainty of evidence is very low (GRADE); findings are likely to change as more evidence becomes available.

Overview of Evidence and Knowledge Gaps     

• Despite wildfire smoke events being a well-established and growing concern, there is limited quantitative evidence that rigorously evaluates the effectiveness or unintended harms of public health interventions to reduce the impacts of exposure. More research is required, taking a realist approach (i.e., paired with expert opinion, evaluation studies), to establish which interventions are most effective, under which circumstances.

• This review includes 19 single studies of moderate to high quality, describing the effectiveness of public health interventions to reduce the health impacts of wildfires and their smoke. The evaluated interventions were related to executing evacuation orders (n=7), improving indoor air quality (n=7), and communicating outdoor air quality (n=4), with one study evaluating a multicomponent intervention (e.g., masking, evacuation, air filtration, and public service announcements).

• There is considerable variation across studies with respect to the setting, population, and outcomes - even within similar intervention groupings – as well as inconsistency in the level of detail provided for each. These inconsistencies may limit the generalizability of any findings, highlighting further the importance of context.

• A limited number of studies provide evidence for the experiences of populations who live with social and structural inequities, such as Indigenous, racialized, or rural communities, and those for whom gender and/or mental health considerations apply. These factors were mainly addressed by studies evaluating the impact of evacuation interventions. Further research is required to ensure representation of these populations for decision making related to all interventions.

• While the search was comprehensive (i.e., databases, reference lists, subject matter expert), it is limited in that a jurisdictional or grey literature scan (e.g., program evaluation data, regional data) was not conducted.

Methods

A description of the development of the NCCMT’s Rapid Evidence Service, including an overview of the rapid review process and rationale for methodological decisions, has been published (Neil-Sztramko et al., 2021).

Research Question

What is the evidence for the effectiveness of public health interventions, and their potential unintended consequences, to reduce the direct and indirect health impacts of exposure to wildfires, including wildfire smoke and combined heat-wildfire smoke events?

Search

On July 27, 2023, the following databases were searched using key terms wildfire* OR "forest fire*" OR "bush fire*" OR "wildland fire*" OR "smoke event*" OR "wild fire" OR "brush fire*" OR conflagration OR "rural fire*":

• MEDLINE database
• Global Health Database
• Political Science Database
• MedRxiv preprint server
• Web of Science

A copy of the full search strategy is available in Appendix 1.

In addition to the databases searched above, the reference lists of included studies were searched, as well as the reference lists of reviews and reports that did not meet all inclusion criteria themselves but were relevant to the general topic. A subject matter expert from the NCCEH, with air quality and wildfire smoke expertise, was also consulted to identify additional studies for consideration and confirm study inclusion/exclusion criteria.

Study Selection Criteria         

The search results were first screened for recent guidelines and syntheses. When available, findings from syntheses and clinical practice guidelines are presented first, as these consider the available body of evidence and, therefore, can be applied broadly to populations and settings.

Single studies were included if, 1) no (or only low quality) syntheses were available; 2) they were published after the search was conducted in any included syntheses; or 3) the synthesis in its entirety was not relevant to this rapid review’s question (i.e., relevant single studies from syntheses were included, but not the synthesis itself).

Peer-reviewed sources and sources published ahead-of-print before peer review were included. Surveillance sources and mathematical modelling studies that exclusively used estimated data were excluded.

Data Extraction and Synthesis

Data relevant to the research question (including study design, setting, location, population characteristics, interventions, and outcomes) were extracted when reported in the included studies. The results were synthesized narratively due to the variation in methodology and outcomes for the included studies. A subject matter expert from the NCCEH reviewed and provided feedback on the synthesized results; these considerations were incorporated into the final review.

Appraisal of Evidence Quality

The quality of included evidence was evaluated using critical appraisal tools, as indicated by the study design below. Quality assessment was completed by one reviewer and verified by a second; conflicts were resolved through discussion.

Completed quality assessments for each included study are available on request.

This review includes both quantitative and qualitative research evidence. The approach to assessing the certainty and confidence in the quantitative and qualitative findings, respectively, is described below and the results of the assessments are summarized in the Summary of Evidence Certainty and Confidence table.

The Grading of Recommendations, Assessment, Development and Evaluations (GRADE) (Schünemann et al., 2013) approach was used to assess the certainty in the findings in quantitative research based on eight key domains. 

In the GRADE approach to quality of evidence, observational and experimental studies (e.g., randomized controlled trials), as included in this review, provide low and moderate certainty evidence, respectively, and this assessment can be further reduced based on:

• High risk of bias
• Inconsistency in effects
• Indirectness of interventions/outcomes
• Imprecision in effect estimate
• Publication bias

and can be upgraded based on:

• Large effect
• Dose-response relationship
• Accounting for confounding.

The overall certainty in the evidence for each outcome was determined, considering the characteristics of the available evidence (observational studies, some not peer-reviewed, unaccounted-for potential confounding factors, different tests and testing protocols, lack of valid comparison groups). A judgement of ‘overall certainty is very low’ means that the findings are very likely to change as more evidence accumulates.

The Grading of Recommendations, Assessment, Development and Evaluations - Confidence in

Evidence from Reviews of Qualitative Research (GRADE-CERQual) approach was used to assess the confidence in the findings in qualitative research based on four key domains.

In the GRADE-CERQual approach to quality of evidence, qualitative research, as included in this review, provides high confidence evidence, and this assessment can be reduced by one or more levels based on:

• Methodological limitations
• Relevance
• Coherence
• Adequacy

The overall confidence in the evidence (expressed as either high, moderate, low, or very low) for each prominent theme was determined considering the characteristics of the available evidence. A judgement of ‘overall confidence is moderate’ means that it is likely that the finding is a reasonable representation of the phenomenon of interest (Lewin et al., 2018).

Findings

Summary of Evidence Certainty and Confidence

This review includes 19 single studies.

What is the evidence for the effectiveness of public health interventions, and their potential unintended consequences, to reduce the direct and indirect health impacts of exposure to wildfires, including wildfire smoke and combined heat-wildfire smoke events?

¹ In the GRADE approach to certainty of evidence, observational studies, as included in this review, provide low certainty evidence; this was downgraded to very low due to inconsistency and imprecision in effect estimates.

² In the GRADE-CERQual approach to confidence of evidence, qualitative studies, as included in this review, provide high confidence evidence; this was downgraded to moderate due to concerns with data adequacy and coherence.

³ In the GRADE approach to certainty of evidence, experimental and observational studies, as included in this review, provide moderate and low certainty evidence, respectively; the overall certainty here is low due to inconsistency.

⁴ In the GRADE approach to certainty of evidence, experimental and observational studies, as included in this review, provide moderate and low certainty evidence, respectively; this was downgraded to very low due to risk of bias, inconsistency, and imprecisions in effect estimates.

*Values exceed the total number of studies (n=19) as some studies involved multiple outcomes.

Table1: Summary of findings

See the PDF for full table

Image

References

Barn, P., Larson, T., Noullett, M., Kennedy, S., Copes, R., & Brauer, M. (2008). Infiltration of forest fire and residential wood smoke: an evaluation of air cleaner effectiveness. Journal of exposure science & environmental epidemiology, 18(5), 503–511.

Canadian Council of Ministers of the Environment (CCME). (2023). Canada’s Air.

Cascio, W.E. (2018). Wildland fire smoke and human health. Science of The Total Environment, 624, 586-595.

Cherry, N., & Haynes, W. (2017). Effects of the Fort McMurray wildfires on the health of evacuated workers: follow-up of 2 cohorts. CMAJ open, 5(3), E638–E645.

Christianson, A.C., McGee, T.K. & Whitefish Lake First Nation 459. (2019). Wildfire evacuation experiences of band members of Whitefish Lake First Nation 459, Alberta, Canada. Natural Hazards (98), 9–29.

Dodd, W., Scott, P., Howard, C., Scott, C., Rose, C., Cunsolo, A., & Orbinski, J. (2018). Lived experience of a record wildfire season in the Northwest Territories, Canada. Canadian journal of public health, 109(3), 327–337.

Fish, J.A., Peters, M.D., Ramsey, I., Sharplin, G., Corsini, N., Eckert, M. (2017). Effectiveness of public health messaging and communication channels during smoke events: a rapid systematic review. Journal of Environmental Management, 15(193), 247-256.

Government of Canada. (2023). Wildfires in Canada: Toolkit for Public Health Authorities.

Hadley, M.B., Henderson, S.B., Brauer, M., Vedanthan, R. (2022). Protecting cardiovascular health from wildfire smoke. Circulation, 146(10), 788-801.

Henderson, D.E., Milford, J.B., & Miller, S.L. (2005). Prescribed burns and wildfires in Colorado: impacts of mitigation measures on indoor air particulate matter. Journal of the Air & Waste Management Association, 55(10), 1516–1526.

Jenkins, J.L., Hsu, E.B., Sauer, L.M., Hsieh, Y.H., & Kirsch, T.D. (2009). Prevalence of Unmet Health Care needs and description of health care-seeking behavior among displaced people after the 2007 California wildfires. Disaster medicine and public health preparedness, 3(2 Suppl), S24–S28.

Krstic, N., Henderson, S.B. (2015). Use of MODIS data to assess atmospheric aerosol before, during, and after community evacuations related to wildfire smoke. Remote Sensing of Environment, (11), 1-7.

Lewin, S., Booth, A., Glenton, C., Munthe-Kaas, H., Rashidian, A., Wainwright, M., ... & Noyes, J. (2018). Applying GRADE-CERQual to qualitative evidence synthesis findings: Introduction to the series. Implementation Science, 13(1), 1-10. 

Liu, J.C., Pereira, G., Uhl, S.A., Bravo, M.A., Bell, M.L. (2015). A systematic review of the physical health impacts from non-occupational exposure to wildfire smoke. Environmental Research, 136, 120-132.

Maguet, S. (2018). Public Health Responses to Wildfire Smoke Events. BC Centre for Disease Control, National Collaborating Centre for Environmental Health.

May, N.W., Dixon, C., Jaffe, D.A. (2021). Impact of Wildfire Smoke Events on Indoor Air Quality and Evaluation of a Low-cost Filtration Method. Aerosol and Air Quality Research, (21)7.

McLean, K.E., Yao, J., & Henderson, S.B. (2015). An evaluation of the British Columbia asthma monitoring system (BCAMS) and PM2.5 exposure metrics during the 2014 forest fire season. International journal of environmental research and public health, 12(6), 6710–6724. 

Mott, J.A., Meyer, P., Mannino, D., Redd, S.C., Smith, E.M., Gotway-Crawford, C., & Chase, E. (2002). Wildland forest fire smoke: health effects and intervention evaluation, Hoopa, California, 1999. The Western journal of medicine, 176(3), 157–162.

Neil-Sztramko, S.E., Belita, E., Traynor, R.L., Clark, E., Hagerman, L., & Dobbins, M. (2021). Methods to support evidence-informed decision-making in the midst of COVID-19: creation and evolution of a rapid review service from the National Collaborating Centre for Methods and Tools. BMC Medical Research Methodology 21(231).

Nguyen, P.D.M., Martinussen, N., Mallach, G., Ebrahimi, G., Jones, K., Zimmerman, N., & Henderson, S.B. (2021). Using Low-Cost Sensors to Assess Fine Particulate Matter Infiltration (PM2.5) during a Wildfire Smoke Episode at a Large Inpatient Healthcare Facility. International journal of environmental research and public health, 18(18), 9811.

Postma, J.M., Odom-Maryon, T., Rappold, A.G., Haverkamp, H., Amiri, S., Bindler, R., … Walden, V. (2022). Promoting risk reduction among young adults with asthma during wildfire smoke: A feasibility study. Public health nursing (Boston, Mass.), 39(2), 405–414.

Reid, C.E., Brauer, M., Johnston, F.H., Jerrett, M., Balmes, J.R., Elliott, C.T. (2016). Critical review of health impacts of wildfire smoke exposure. Environmental Health Perspectives, 124(9), 1334-1343.

Schünemann, H., Brożek, J., Guyatt, G., & Oxman, A. (2013). Handbook for grading the quality of evidence and the strength of recommendations using the GRADE approach.

Stauffer, D.A., Autenrieth, D.A., Hart, J.F., & Capoccia, S. (2020). Control of wildfire-sourced PM_2.5 in an office setting using a commercially available portable air cleaner. Journal of occupational and environmental hygiene, 17(4), 109–120.

Tally, S., Levack, A., Sarkin, A.J., Gilmer, T., & Groessl, E.J. (2013). The impact of the San Diego wildfires on a general mental health population residing in evacuation areas. Administration and policy in mental health, 40(5), 348–354.

Thériault, L., Belleville, G., Ouellet, M.C., & Morin, C.M. (2021). The Experience and Perceived Consequences of the 2016 Fort McMurray Fires and Evacuation. Frontiers in public health, 9, 641151.

Wheeler, A.J., Allen, R.W., Lawrence, K., Roulston, C.T., Powell, J., Williamson, G.J., … Johnston, F.H. (2021). Can Public Spaces Effectively Be Used as Cleaner Indoor Air Shelters during Extreme Smoke Events?. International journal of environmental research and public health, 18(8), 4085.

World Health Organization (WHO). (2023). Wildfires.

Xiang, J., Huang, C.H., Shirai, J., Liu, Y., Carmona, N., Zuidema, C., … & Seto, E. (2021). Field measurements of PM2.5 infiltration factor and portable air cleaner effectiveness during wildfire episodes in US residences. The Science of the total environment, 773, 145642. 

Yao, J., Brauer, M., & Henderson, S.B. (2013). Evaluation of a wildfire smoke forecasting system as a tool for public health protection. Environmental health perspectives, 121(10), 1142–1147.

Yao, J. (2014). Evidence Review: Exposure measures for wildfire smoke surveillance. BC Centre for Disease Control.

Yao, J., Stieb, D.M., Taylor, E., & Henderson, S.B. (2020). Assessment of the Air Quality Health Index (AQHI) and four alternate AQHI-Plus amendments for wildfire seasons in British Columbia. Canadian journal of public health, 111(1), 96–106