The Role of Risk Management Prioritization

Decision Support Tools

An Overview of Barriers, Facilitators, and Recommendations

October 2020

Prepared for

USDA Forest Service

Southwestern Region, Region 3

Prepared by

Melanie Colavito, Melanie.Colavito@nau.edu

Ecological Restoration Institute

Northern Arizona University

PO Box 15017

Flagstaff, AZ 86001

NAU is an equal opportunity provider.

This report was funded by a grant from the USDA Forest Service.

Table of Contents

Executive Summary ..........................................................................................................1

Project Purpose and Need ................................................................................................3

Methods ...............................................................................................................................5

Findings ...............................................................................................................................7

Decision Support Tools Referenced in this Report ....................................................7

Decision Support Tool Definitions .............................................................................9

Decision Support Tool Development and Transfer .................................................10

Decision Support Tool Use ......................................................................................11

Decision Support Tool Barriers ...............................................................................14

Decision Support Tool Benefits and Facilitators .....................................................24

Recommendations ...........................................................................................................30

Conclusion ........................................................................................................................42

References .........................................................................................................................43

Appendix. Open-Ended Interview Questions ................................................................45

Acknowledgements

The Ecological Restoration Institute is grateful for the contributions and time of the interview

participants whose knowledge and perspectives informed this report.

Executive Summary

The purpose of this project was to analyze the use and adoption of wildfire risk assessment and

fuels treatment planning and prioritization analysis methods and products—broadly referred to

here as decision support tools (DSTs)—by federal land managers, especially in the US

Department of Agriculture (USDA) Forest Service. This project examines the efficacy of policy

direction articulated in frameworks and documents such as the National Cohesive Wildland Fire

Management Strategy, Forest Service Manual Chapter 5140, and Forest Service Shared

Stewardship Initiative that call for collaborative assessment of wildfire risk to inform fuels

treatment prioritization using the best available science. There is a need to demystify the topic of

spatial fire planning specifically with respect to assessing wildfire risk and determining areas for

fuels treatment prioritization to facilitate effective development and use of DSTs for pre-fire

planning. We acknowledge that the term DST may not adequately capture the complexity of the

different frameworks, methods, and tools referred to in this report, but we selected this term

intentionally while recognizing that new terminology may be warranted to describe the full realm

of approaches for wildfire risk assessment and treatment prioritization.

This report addresses the following key questions and the results are organized according to key

themes that emerged within each of these questions:

1. What decision support tools (DST) are commonly in use for assessing wildfire risk and

prioritizing areas for fuels treatment in the Forest Service?

2. How are DSTs defined by their developers and users?

3. How are these DSTs developed and transferred?

4. How are these DSTs used?

5. What are barriers to the use of these DSTs?

6. What are benefits and facilitators to the use of these DSTs?

7. How can the development and application of these DSTs be improved?

We used semi-structured interviews with key informants to identify common DSTs for assessing

wildfire risk and treatment planning and prioritization, approaches for the development and

transfer of DSTs, examples of DST uses, common barriers and facilitators in the development

and use of DSTs, and recommendations for facilitating the development and use of DSTs.

The audience for this report includes DST scientists and developers who develop analysis

methods and products, the end users of DSTs, and policy and decision makers who develop

guidance for using DSTs. There are recommendations for key audiences at the end of the report.

This report does not provide a comprehensive or technical summary of wildfire decision support

tools. Rather it is an effort to explore the social dimensions of developing and using DSTs for

pre-fire planning and fire response and to develop recommendations to enhance the use of

scientific and technical tools in pre-fire planning and fire response. This report can be used to

better understand the role of DSTs and support their effective use in wildfire risk assessment and

treatment prioritization planning.

1

Although there were many barriers identified to the effective development, integration, and use

of DSTs in pre-fire planning, interview respondents had numerous constructive

recommendations for improving this process, which we categorized into the following themes:

capacity, communication, implementation, question identification, testing, education and

training, and policy, guidance, and authorities. We hope these recommendations can help shape

the perspectives of science, management, and decision-making audiences for how to improve the

use of DSTs for wildfire risk assessment and treatment prioritization in order to effectively meet

the goals of national policies and frameworks.

2

Project Purpose and Need

The purpose of this project was to analyze the use and adoption of wildfire risk assessment and

fuels treatment planning and prioritization analysis methods and products—broadly referred to

here as decision support tools (DSTs)—by federal land managers, especially in the US

Department of Agriculture (USDA) Forest Service. This project examines the efficacy of policy

direction articulated in frameworks and documents such as the National Cohesive Wildland Fire

Management Strategy, Forest Service Manual Chapter 5140, and Forest Service Shared

Stewardship Initiative that call for collaborative assessment of wildfire risk to inform fuels

treatment prioritization using the best available science (Greiner et al. 2020, Stratton 2020).

There is a need to demystify the topic of spatial fire planning specifically with respect to

assessing wildfire risk and determining areas for fuels treatment prioritization to facilitate

effective development and use of DSTs for pre-fire planning. The desired outcomes of this

project are: 1) a characterization of common spatial fire risk assessment and treatment

prioritization planning analysis methods and products, and 2) actionable recommendations and

strategies developed with input from DST developers and users to facilitate the effective use of

DST outputs in decision making. We acknowledge that the term DST may not adequately

capture the complexity of the different frameworks, methods, and tools referred to in this report,

but we selected this term intentionally while recognizing that new terminology may be warranted

to describe the full realm of approaches for wildfire risk assessment and treatment prioritization.

Although there is policy direction to use DSTs for risk-informed, landscape-scale hazardous

fuels reduction treatments and fire response, the specifics for doing so are unclear. For example,

Forest Service Manual Chapter 5140 – Hazardous Fuels Management and Prescribed Fire –

includes two objectives: 1) “understand the role of fire on the landscape …”, and 2) “…

strategically plan and implement on a landscape scale, risk-informed, and cost-effective

hazardous fuel modification and vegetation management treatments …” (USDA Forest Service

2020). As Stratton (2020) notes, this guidance is consistent with broader frameworks such as the

National Cohesive Strategy, which articulates a cross-boundary, collaborative strategy to reduce

wildfire risk, as well as Shared Stewardship, which also promotes a collaborative, cross-

boundary approach to prioritizing hazardous fuels reduction treatments at landscape-scales.

However, none of these guidance or strategy documents include explicit requirements or

recommendations for what risk assessment and prioritization DSTs to use.

There are numerous analysis methods, products, and tools, and geographic and temporal scales

for modeling fire behavior, assessing wildfire risk, and prioritizing areas for treatment. This can

lead to confusion, among other barriers, about what kinds of information different frameworks,

models, and tools can provide for use in decision making (Rapp et al. 2020). A lack of effective

landscape-scale monitoring of completed restoration or fuels reduction treatments and limited or

incomplete data sets can also limit the development of effective risk assessment and treatment

prioritization DSTs (Esch and Waltz 2019). Nonetheless, DSTs can facilitate a risk-informed and

strategic approach to decision making in pre-fire planning, as well as in fire response (Calkin et

al. 2010, Thompson et al. 2019). And because land managers who are working to reduce the risk

3

of catastrophic wildfire and restore the ecological role of fire must balance many social and

ecological considerations, DSTs can provide an added level of support for complex decision-

making processes at different temporal and spatial scales that can allow for effective treatments

and the safe and effective use of fire (Caggiano et al. 2020, Thompson et al. 2019).

It is important to define key terms used in this report, given the need for a common language to

talk about risk in wildland fire management (Thompson et al. 2016) and confusion that exists

around the different approaches to characterizing and analyzing wildfire risk. The term used

most prominently in this report is “decision support tool” (DST). One of the first questions we

addressed with project participants was how to define a DST. Not surprisingly, responses varied

widely, and a summary of those responses comes later in this report. For the purposes of this

report, we use the term DST broadly to refer to the different frameworks, models, analysis

methods, and products and tools (i.e., software programs) that provide information about wildfire

risk assessment and treatment prioritization to support strategic, risk-informed pre-fire planning.

The focus in this report is on DSTs that support pre-fire planning, but certain DSTs referenced

by interviewees, such as the Wildland Fire Decision Support System (WFDSS), are largely

intended for fire response but may accept inputs and data layers developed from pre-fire

planning processes. In this report, the term DST refers to both specific tools, such as WFDSS

and the Interagency Fuel Treatment Decision Support System (IFTDSS), as well as analysis

methods and frameworks, such as Quantitative Wildfire Risk Assessments (QWRA) or Potential

Operational Delineations (PODs).

Other key terms used in the report include “wildfire risk assessment” and “treatment

prioritization.” While we did not ask participants to define these terms, it is important to provide

a frame of reference for how they are used in this report. Wildfire risk assessment refers to the

range of approaches for assigning values to risks and informing decision making (see Scott et al.

2013 and Thompson et al. 2016). The Thompson et al. (2016) definition, more specifically,

states, “A product or process that collects information and assigns values (relative, qualitative,

quantitative) to risks for the purpose of informing priorities, developing or comparing courses of

action, and informing decision making,” and is adopted here. Treatment prioritization refers to

approaches for determining where to treat hazardous fuels to reduce wildfire risk as determined

by a wildfire risk assessment or other prioritization process. It is explicitly referenced in this

report because we also examined DSTs that provide outputs specific to treatment prioritization.

This report addresses the following key questions:

1. What decision support tools (DST) are commonly in use for assessing wildfire risk and

prioritizing areas for fuels treatment in the Forest Service?

2. How are DSTs defined by their developers and users?

3. How are these DSTs developed and transferred?

4. How are these DSTs used?

5. What are barriers to the use of these DSTs?

6. What are benefits and facilitators to the use of these DSTs?

7. How can the development and application of these DSTs be improved?

4

The rationale for this project was informed by the “All-Lands Planning, Coordination, and

Wildfire Risk Reduction Project” conducted by the Ecological Restoration Institute (ERI)

(Colavito et al. 2018). One of the lessons learned from that project was that although DSTs can

provide robust, scientific information to inform land management planning, there are many

social barriers to realizing the effective use of DST outputs in decision making. This project

sought to identify the specific barriers and facilitators to the effective development and use of

DSTs for wildfire risk assessments and treatment planning and prioritization and to generate

actionable recommendations for DST developers, users, and policymakers to improve the use of

DSTs in land management.

The audience for this report includes DST scientists and developers who develop analysis

methods and products, the end users of DSTs, and policy and decision makers who develop

guidance for using DSTs. There are recommendations for key audiences at the end of the report.

This report does not provide a comprehensive or technical summary of wildfire decision support

tools. Rather it is an effort to explore the social dimensions of developing and using DSTs for

pre-fire planning and fire response and to develop recommendations to enhance the use of

scientific and technical tools in pre-fire planning and fire response. This report can be used to

better understand the role of DSTs and support their effective use in wildfire risk assessment and

treatment prioritization planning.

Methods

We used semi-structured interviews with key informants to identify common DSTs for assessing

wildfire risk and treatment planning and prioritization, approaches for the development and

transfer of DSTs, examples of DST uses, common barriers and facilitators in the development

and use of DSTs, and recommendations for facilitating the development and use of DSTs. Prior

to conducting the semi-structured interviews, we reviewed peer-reviewed literature, gray

literature, and other relevant sources to inform our research questions and interview protocol. We

also conducted preliminary, informal scoping interviews and conversations with key contacts

who did not participate in formal interviews to inform our study design.

In order to identify interview participants, we used a purposive sampling approach. We started

by asking contacts from the scoping conversations who they recommended we interview. The

requirement for our purposive sample was that respondents be familiar with the development or

use of the DSTs relevant to pre-fire planning. We asked interviewees to provide suggestions for

additional interviewees, as well as identified interviewees through our networks. We contacted

interviewees in multiple places and roles to ensure our data represented a range of perspectives

(Tables 1, 2, and 3). There were 27 interview participants (n=27). We did not geographically

constrain interviews and had respondents from Regions 1, 2, 3, 5, 6, and 9 of the USDA Forest

Service. Respondent locations included New Mexico (n=6), Colorado (n=5), Montana (n=5),

Oregon (n=4), Arizona (n=4), Idaho (n=1), California (n=1), Minnesota (n=1). We conducted

interviews until we reached information saturation.

5

The interviews averaged one hour in length, were all conducted over the phone, and were

recorded and transcribed for analysis. We used the qualitative data analysis software Nvivo 11

for Windows to organize and code interview transcripts for key themes. The analytical process

we used was a systematic approach called grounded theory, which allowed us to iteratively

generate themes both from prior knowledge and literature, as well as from the data itself. The

findings presented here represent a summary of themes across interviews. Interviewees are not

identified with quotes to ensure anonymity according to the requirements of the Northern

Arizona University Institutional Review Board that oversees this human subjects project.

Table 1. Respondent Demographics by Organizational Affiliation

Organizational Affiliation Number of Respondents Percent

USDA Forest Service 22 81%

External Stakeholder 5 19%

Total 27 100%

Table 2. Respondent Demographics by Title Classification

Title Classification Number of Respondents Percent

Agency Analysis/ Tech

Transfer 7 26%

Agency Researcher 2 7.33%

Fire Management 6 22%

Line Officer 2 7.33%

External Researcher 3 11%

Fuels Management 5 19%

Collaborative Partner 2 7.33%

Total 27 100%

Table 3. Respondent Demographics by General Classification (as assigned or self-identified during

interview)

General Classification Number of Respondents Percent

Boundary Spanner 8 30%

Scientist/ Developer 6 22%

Manager 13 48%

Total 27 100%

6

Findings

Decision Support Tools Referenced in this Report

Currently, there is no comprehensive list of DSTs used for wildfire risk assessment or treatment

prioritization, though USDA Forest Service Research and Development recently conducted an

inventory of common tools. Interviewees referenced numerous DSTs relevant to wildfire risk

assessment and treatment prioritization. The DSTs referenced by interviewees are organized

below according to the kinds of information they provide to help elucidate the different roles that

DSTs play in the realm of pre-fire planning. This is not an exhaustive list of wildfire risk

assessment and treatment prioritization DSTs, rather it is a list of the major tools that were

discussed by interviewees. Some interviewees also referred to DSTs developed for specific

projects or collaborative groups, which are not included in this list.

• Data sets: developing DSTs require data sets that provide the underlying data that are

analyzed as an input for decision making. Any modeling product or DST is only as good as

the data on which it relies. Interviewees referenced three data sets used in wildfire risk

assessment and treatment prioritization DSTs.

o LANDFIRE: a database of more than 20 geospatial layers such as vegetation, fire

regimes, and fuels for the United States.

o FSVeg, or Field Sampled Vegetation: stores data about trees, fuels, down woody

material, surface cover, and understory vegetation.

o FIA, or Forest Inventory and Analysis Program: a national census of forest data in the

United States.

• Wildfire Risk Assessment: many interviewees referenced fire simulation models, each of

which have different parameters and use different fire behavior models to compute

predictions. However, all the models below use the same surface fire spread equation

developed by Rothermel (1972). There are many challenges in modeling wildfire, depending

on weather conditions, patterns of fire ignition across landscapes, and historical distributions

of fires (Ager et al. 2017). All of the DSTs below provide some assessment of wildfire risk,

which can be defined as the likelihood of a fire occurring, the intensity of behavior when a

fire does occur, and effects of fire on highly valued resources and assets (HVRA) (Calkin et

al. 2010, Scott et al. 2013). The DSTs referenced by interviewees are organized below

according to whether they generate fire simulations (i.e., first order products) or use fire

simulations to create products to inform decision making (i.e., second order products).

o Fire probability models and simulators (i.e., first order products)

▪ FlamMap: a fire analysis system that calculates pixel-based measures of fire

behavior; smaller scale but similar inputs to LFSim.

▪ FARSITE: a computational approach to estimating wildfire growth and

behavior, which has now been integrated into FlamMap; tends to overestimate

extent of fire compared with on-the-ground observations.

7

▪ FSim or LFSim: also known as the Large Fire Modeling Simulation System,

is a method for simulating fire modeling across larger scales and across

seasons than FlamMap or FARSITE but using similar parameters.

▪ Fire Spread Probability (FSPro): a probabilistic model that estimates where

fire may go; tends to overpredict final wildfire footprint but underestimate

farthest extent burned.

▪ BEHAVE: a fire modeling system that simulates surface and crown fire rate

of fire spread and intensity, probability of ignition, fire size, spotting distance,

and tree mortality.

o Frameworks or tools that use outputs from fire simulation modeling (i.e., second

order products)

▪ Wildfire Hazard Potential: a national scale map that combines flame-length

probability and burn probability to inform wildfire risk assessment and fuels

management prioritization across large landscapes.

▪ Wildfire Risk to Communities: an interactive website that provides a national

scale assessment of wildfire risk to communities; best used to compare risk

among communities not within communities. This was developed in response

to the 2018 Omnibus Bill calling for a national map of wildfire hazard.

▪ Wildland Fire Decision Support System (WFDSS): an online system that

supports fire managers, agency administrators, and fire analysts in making

decisions on fire incidents; includes fire modeling capability, as well as

decision documentation processes.

▪ Quantitative Wildfire Risk Assessment (QWRA): a fire modeling process that

assesses how HVRAs respond to modeled fire behavior and determines both

benefits and threats to selected HVRAs. The background for this concept is

described in General Technical Report (GTR) 235 (Calkin et al. 2010) and the

process for developing QWRAs is detailed in GTR 315 (Scott et al. 2013).

▪ Potential Operational Delineation (POD): PODs refer to boundaries that

provide different fire response options across a landscape. POD polygons

provide an overview of fire risk according to potential control features and

values at risk (Caggiano 2019, Greiner et al. 2020). They are an approach for

making QWRA relevant in an operational context but can also be developed

without a QWRA (Caggiano et al. 2020). There are also efforts to explore

how to use PODs to guide treatment prioritization.

• Treatment Planning and Prioritization: interviewees referred to models that provide

simulations of forest growth and landscape processes, especially with respect to how

management changes can impact fire behavior. These models can be used to inform

treatment planning and prioritization decisions, though as with fire behavior models, there

are many challenges to simulating forest changes (Ager et al. 2017).

8

o Forest Vegetation Simulator (FVS): a forest stand-level modeling tool that simulates

forest growth in response to natural processes and management actions; used to

quantify effects for National Environmental Policy Act (NEPA) analyses.

• FVS Fire and Fuels Extension (FVS-FFE) simulates how fuels and potential

fires will change over time in response to management.

▪ LSim: a forest landscape simulation model integrates FSim into a modified FVS

Parallel Processing Extension (Ager et al. 2017).

▪ Scenario Investment Planning Platform (SIPP) or spatial scenario planning model

(FORSYS), formerly Landscape Treatment Designer (LTD): Ager Scenario Planning

Framework (scale of large fire events, more for resource allocation and budgeting).

o ArcFuels: an ArcGIS toolbar that uses FVS and fire behavior models to provide

decision support for vegetation management, fuel treatment planning, fire behavior

modeling, and wildfire risk assessments.

o Interagency Fuel Treatment Decision Support System (IFTDSS): an online system

that supports fuels treatment planning and analysis, including fire behavior modeling

under different conditions and processes for testing fuels treatment impacts on fire

behavior to determine priority areas for reducing fire behavior.

• Other:

o Risk Management Assistance (RMA): RMA is not a DST, but it was brought up by

interviewees as it is a form of decision support on fires. RMA teams travel to fires

and provide decision support in the form of analytics and discussion facilitation

(Schultz et al. 2020).

Decision Support Tool Definitions

As mentioned previously, the term DST means different things to different people. There is no

established definition of a DST in fire science or management. Therefore, we asked interviewees

how they define DST to get a general sense of common elements across disciplines and roles.

Interviewees spoke of a clear distinction between data, such as a geospatial data layer, and a

DST, which provides a higher-level function to facilitate risk-informed decision making. One

boundary spanner said, “An important distinction for me is that a decision support tool provides

some kind of capability to either evaluate scenarios or prioritize something. So, it has some kind

of higher-level function that evaluates or analyzes the data in a unique way to help someone

make a decision” (DST11). Another boundary spanner put it this way, “What I think separates

decision support tools versus any old piece of information is they’re set up to answer a specific

question, or help make a specific decision, and so they purposefully have multiple

considerations” (DST14).

Another distinction interviewees made was between an informational resource, such as a

website, and a DST as a tool or software program that can be asked specific questions. For

example, one scientist/developer said, “A decision support tool in my mind is something like

9

WFDSS or IFTDSS that are actual computer programs, software programs that provide people

actionable tools to ask questions and run analyses” (DST16). One manager also identified

programs like WFDSS, and said, “A decision support tool, I think of WFDSS, I guess. Pretty

much because that's what we use, that’s our decision of record” (DST17).

Some interviewees also emphasized that DSTs do not make decisions, rather they support

decisions. One manager stated, “It doesn’t make a decision, but it helps us take a quick look at

what we might expect and consider that when we need to move forward with a decision”

(DST21). Another manager noted, “I really look at them as a piece of information, not a

decision, not something to make a decision based totally on. That is just one of the pieces of

information that I throw into our decisions. It’s the best available science we have, so I’m pretty

adamant we use them, where a lot of other folks, they're like, ‘Oh, it’s just a model, I can

probably guess as well as the model does.’ But if I put that in a document, I feel like I have good

scientific backing behind this decision we’re making” (DST25).

Many interviewees described DSTs in broad terms. One manager described a DST as, “Anything

that helps inform a decision better than what you would have otherwise had available to you is a

decision support tool” (DST24). Another manager said, “So it’s anything from a fire behavior

model to the risk assessment model, the Zimmerman graphs that are in WFDSS right now … I

think those are all decision support tools that just help you think through the process and think

through what’s important to this decision” (DST25). Another manager stated, “A decision

support tool should be something that helps fire managers and agency administrators make

informed decisions using a deliberative process regarding wildfire management” (DST26).

The broad conceptualization of DSTs by interviewees corresponds to the definition used

throughout this report in which DST refers to a broad array of frameworks, models, analysis

approaches, and products and tools. One scientist/developer noted that the term DST is

misleading, as it is often used in this broad sense as anything that, “… helps inform managers

how to make better decisions” (DST4). The same scientist/developer suggested that another term

might be more appropriate. “I think decision support tools help people think about problems.

They shouldn’t be called decision support tools; they probably should be called something else.

Thought, thought analytics, you know? Or, I call it conversational analytics. It’s a way of

starting a conversation that takes advantage of all the tools” (DST4).

Decision Support Tool Development and Transfer

Six interviewees characterized themselves as scientists or DST developers and provided

information about the process of developing and transferring DSTs. Scientists/developers noted

that there is a recurring disconnect between some scientific work and the needs of managers in

the field. This disconnect manifests itself in multiple ways. For example, as one boundary

spanner noted, “Oftentimes scientists work on things that they think are important or the money

dictated they work on, but really aren’t helping meet the needs of the field. But every once in a

while, they make something that they really like, but we don’t have a really great mechanism

10

where scientists can fledge their science and it can go into operations and management”

(DST1). This comment highlights that there can be a disconnect between what scientists think

should be developed in terms of DSTs and what the potential end users of DSTs in the field need

to facilitate decision-making. This disconnect can also occur when an operationally relevant DST

is developed, but there is no clear process for taking that new tool and getting it into management

effectively and the capacity to do that work is limited. As the same interviewee later noted,

“There’s only a few people that can develop it and code it to apply it in management” (DST1).

Scientist/developer interviewees referred to multiple DST transfer approaches, with the most

common including in-person engagement, field trips, workshops, webinars, and online modules.

In-person engagement was highlighted as a critical element of successful DST transfer. One

scientist/developer provided an example of an ongoing, in-person DST transfer process that

resulted in building trust and relationships with end users. “It was just me there at these meetings

all day, twice a month, maybe more, just being willing to answer questions and interject and

explain over and over and over again. Particularly some of the quantitative pieces. And laughing

about it and enjoying it” (DST27). Similarly, field trips were cited as being effective for

communicating how DSTs work on the ground. As one manager stated, “And we brought them

out on a field trip and showed them all the work we had done ahead of time to decide where we

were doing what and why, and we got no pushback from them after that. They were like, ‘Now I

get it.’” (DST25). Workshops also serve a similar function in terms of communicating the utility

of DSTs and building trust, both in the DST outputs and the scientists/developers communicating

about the DSTs. One boundary spanner explained, in reference to workshops for developing

PODs, “Doing these workshops sometimes you get people who are like, ‘Ah, I don’t think this is

going to be very helpful.’ But by the end of the workshop, they see the utility of just having

everybody in the room, working through this stuff together” (DST10).

Decision Support Tool Use

Respondents were asked to describe how they have used or participated in efforts to use the

DSTs with which they were familiar. Interviewees provided examples of failures to use DSTs as

envisioned that nonetheless resulted in useful learning, as well as examples of successful DST

use. Additionally, some respondents referred to collaborative processes to develop context-

specific DSTs that are not referenced in the list of DSTs provided earlier in this report. Finally,

respondents referred to future goals and hopes for the effective use of DSTs.

Interviewees provided several examples in which DSTs were used to the best of a teams’

abilities but still resulted in undesirable outcomes. This highlights that even with the best

available DSTs, it is not always possible to prevent undesirable outcomes, but at the same time,

that does not mean that the tools or information were not valid. As one scientist/developer

explained, “I think that’s an important thing to be able to communicate that, like even when we

have the best information to try to make the best decisions, we’re not always gonna have a

perfect outcome because there are things that we just simply can’t control. But the hope is that

11

that doesn’t lead somebody to just say, ‘Okay, well then that approach to fire management is not

valid anymore because we had one bad outcome’” (DST16).

Another interviewee referred to the challenges of using DSTs in cases where stakeholders

already have pre-existing notions. In instances like this, it can be challenging to use DST outputs

if they do not validate these pre-existing notions. As one manager explained, “So you were going

to have people that disagreed from the initial response from the get-go. So, as it played out over

time, they continued to disagree depending on whose strategy got chosen under any given

condition. And, so they viewed the analytics much more of either validating their opinions, not

helping inform choices or validating the, quote, opposition’s opinions, and not helpful because

they didn’t just give an answer” (DST7). In order to address the issue of DST outputs not

matching the perceptions of fire managers, the same manager emphasized the importance of

having conversations about the use and role of DSTs early in a process and providing space for

local leaders to lead the conversation and build relationships.

In addition to lessons learned from unsuccessful attempts to use DSTs, interviewees provided

examples of successful DST use ranging from scene-of-action on a fire to long-term risk

assessment and planning. Interviewees highlighted the value of using DST outputs from WFDSS

and FSPro on a fire being managed for resource benefit to communicate with the local

community and build support and understanding of the planning that goes into letting a fire burn

for resource benefit. One scientist/developer explained, “And, so with all that information at our

fingertips and being able to show them we’re being really thoughtful about this … Generally, the

community was very happy … I think that for me it’s showed how powerful it can be, just on a

personal level, to be able to relay all of this fire modeling information to people and show them

that we’re managing fire with … the information that we can” (DST16).

Interviewees also talked about how longer-term risk assessments and DST outputs for planning

can also facilitate community understanding and support. As one manager explained, “Having

those maps and those documentations of why we were doing what we were doing and how we

chose to do what we do, I think, made a big difference” (DST25).

PODs were often mentioned by interviewees in describing success stories for pre-fire planning.

For example, one scientist/developer noted, “For specific actions on the ground, we often tout

the example from the Tonto National Forest, the Pinal Fire, and that’s out there in some

published sense as a true example of that fire response (see O’Connor and Calkin 2019). And the

importance of the discussions upfront and the pre-planning to facilitate a response that was

successful and was very beneficial in reducing risk for surrounding areas” (DST27).

Meanwhile, one manager highlighted that another value of pre-fire planning and successful

implementation of PODs mitigates fire hazards in the long-term, “Those are hard successes to

capture. That we haven’t had to mitigate hazards or expose people to putting out other

unplanned ignitions in all that country. And I know the day will come where … we’ll start getting

fires in those areas again, but we haven’t had to yet. And those are big wins in my book. We’ve

12

taken a piece of country that we’re responsible for and changed it, changed the hazard there. I

think it’s just hard to capture, hard to put numbers on that but it’s there. It’s real” (DST6).

DSTs are also used to support grant applications and provide justification for funding. As one

manager explained, “I used it two years ago to submit a proposal for Joint Chiefs project which

we were awarded … I used that as a justification and rationale for why we’re looking at these

various areas for large scale broadcast burn treatment and other types of manual, mechanical

work” (DST18). DSTs can also communicate collaborative agreement that can further boost

funding proposals. As one boundary spanner noted, “A 15-page document that has 20 signatures

of the diverse stakeholders, who’ve all said we buy into this thing, when you attach that to a

grant application or another type of funding proposal, you’re in a much better position to get

funded, because from a funder’s perspective it takes a lot of the guesswork out of whether this is

just one organization or one government opinion of what needs to be done, or 20 organizations

and governments’ opinion of what needs to be done. It gives them an assurance that this is a

well-thought-out proposal that’s part of a bigger strategy” (DST19). DSTs can also improve the

narrative and rationale for how funding will be used. As one scientist/developer explained,

“Managers are motivated and the people that have a better storyline about what they’re going to

do with their budget in terms of improving landscape conditions and meeting targets sometimes,

have a better chance of getting supplemental funding” (DST3).

Interviewees also referred to efforts to develop project-specific DSTs to assist collaborative

groups in prioritization. As one boundary spanner noted, using formal, complex risk assessment

tools is often beyond the means of smaller organizations. “And one of the challenges with

smaller nonprofits or organizations that are building some kind of risk assessment to make the

business case for the kind of investment they need to actually scale things up to have a

meaningful impact, is that they oftentimes don’t have the money on hand to really utilize

something that’s a large landscape scale planning tool” (DST11). One common approach for

smaller groups is a tailored process that brings together local partners and data. For example, one

boundary spanner referred to a process to develop priority areas for treatment that used local data

depicted to a collaborative group through large table-sized maps to facilitate a discussion about

prioritization. “We relied pretty heavily on local and expert knowledge to help us, to guide us

through the prioritization piece of it. And we identified four different areas in what we called the

landscape resilience strategy … It was a completely different, more open discussion, more

collaborative-led prioritization process. And the science served as the foundation or the hub that

conversations could rotate around and it was scientifically much less complex and but socially

more engaged than some of the other risk assessment work” (DST15). Interviewees noted that

some of the benefits of grassroots processes is that they facilitate relationship building and trust

among partners in a landscape but often require sustained effort and time to accomplish.

13

Decision Support Tool Barriers

Persistent tensions, disconnects, and cultural issues impact the use of DSTs

Responses from interviewees across the spectrum of science, management, and boundary

spanning organizations illustrated several persistent tensions and divergent perceptions of the

role, development, and use of DSTs for wildfire risk assessment and treatment prioritization

planning. These tensions manifested in two primary ways.

First, responses highlighted a tension between a desire for top-down directives on what DSTs to

use and how to use them and a desire for a more organic, grassroots, bottom-up development and

integration of DSTs. One manager articulated a desire for the grassroots approach to developing

and integrating new DSTs. “Everybody wants to be the tip of the spear, be the most advanced or

whatever, but you got to organically build that within your fire program for it to be a useful

tool” (DST24). Meanwhile, another manager acknowledged that a lack of interest in a new DST

at the local level warrants more top-down intervention. “I think there needs to be really tight,

good communication from the top down, and each individual district on the forest has their own

culture, and so a lot of times … All it takes is one or two people to say, ‘Hey, I’m not interested

in this,’ to blow the whole thing up. I think it really needs to be a top-down thing where, from the

Forest Service perspective, it forces you to say, ‘Hey, this is what we’re doing, and I want you to

make sure that all of your staff is coming along with this and moving it and staying on tap.’ And

then having those unit line officers, the district rangers giving that same message so everyone’s

on the same page from the communication standpoint” (DST23).

The second tension articulated by interviewees was a tension between a desire for scientists to

co-develop tools in concert with intended end users and a desire to not take precious time away

from intended end users in experimenting with a new DST that may not be adopted. One

manager summarized it this way, “Instead of the big workshops, computers, remote sensing

exercise, if you really want to make this valid, maybe take a few more steps, and bring in the

implementing part and relationship part, and go get into the weeds a little more, if you wanted to

reflect reality more. And then that’s a big rabbit hole, because when I complained about before,

it was like, ‘Well, am I going to take your finite time and ask you to go do something like I’ve just

described? And what is that going to do for you?’ Because … that might not add up to anything”

(DST12). This illustrates an ongoing disconnect between science and management that was

similarly articulated by another manager. “There’s the disconnect between the people that are

running and building these tools and the folks that are ultimately using the tools, and the

understanding of the intricacies of the landscape that they work on and the data that they would

need to run is maybe not there” (DST22).

Interviewees also alluded to persistent cultural issues that impact the use of DSTs. For example,

interviewees referred to the challenge of changing existing mindsets and cultural issues that

impact perceptions of the role of fire. As one manager noted, “We haven’t had that cultural shift.

It’s still, despite the messaging, very much viewed as, fire is a bad thing and the less of it we can

have the better” (DST24). These cultural barriers impact perceptions of the role of science, as

14

well. As one boundary spanner put it, “I’d say there’s a quiet undercurrent in fire to not

embrace science” (DST1). Another scientist/developer referred to it this way, “Almost like, if

you show me a map that says that there's an X percent probability of the fire reaching this place,

I’m going to take it as a challenge, and I’m going to make sure that doesn’t happen … There’s a

real challenge to changing people’s mindset to have a little bit more patience to not always want

to just get in there with the heavy machinery and try to go direct” (DST16).

Scientists/developers also brought up the challenges of trying to shift paradigms and propagate a

risk-informed decision-making approach throughout the agency. For example, one scientist/

developer stated, “It’s a bigger challenge to create decision support concepts and get those

propagated and understood than it is to create a model that goes and does something on some

national forest” (DST4).

Lastly, scientists/developers referred to cultural barriers that impact public perceptions of the

role of fire and the challenge of communicating fire management approaches like allowing fire to

burn. “On the community side, it’s a really hard sell. I feel like those of us in science are always

trying to make the sell and get the public messaging out that wildfire is not always bad on the

landscape … But when people are pounded over and over again in the media and other

messaging that it’s devastation, destroying landscapes, the adjectives are always so terrible

about the effects of fire. And, so it’s really hard to come in and then tell people, “Hey, look at

fire, it’s really actually not all that bad, and we’re going to be patient. We’re going to let it burn

up there …” I would say cultural barriers exists all around smart and thoughtful fire

management” (DST16).

Capacity issues inhibit both the development and use of DSTs

Interviewees across respondent groups referred to numerous capacity issues that impact the

development and use of DSTs. Capacity issues were described in general and specific terms and

occur in science, boundary spanning, and management organizations. As one scientist/developer

noted, “I think we need capacity on both sides. Those folks that can communicate and work with

groups, as well as those folks that can create the analytics. And both lack … There, of course,

are good people out there that can do it, they’re just tasked with other things” (DST27).

One of the most persistent capacity issues that interviewees brought up is simply a lack of time.

As one manager noted, “I think there’s a time component where we have a full program of work

for the people that are managing the forests and doing the work in the woods, and there’s limited

capacity to add anything new onto what they’re required to do” (DST22). Another manager

explained, “As a ranger, we’re so busy that I don’t really have time to take a deep dive into

stuff” (DST13). One boundary spanner said, on a similar note, “They’re so strapped for time that

some people might even have the skills to do some of these things if they actually had the time to

sit down and think and learn and work their way through a process like this” (DST11).

Interviewees also noted that there is a general lack of personnel capacity across land

management agencies, especially the Forest Service, that impacts the development and use of

15

DSTs. As one manager said, “And as you probably know, across the Forest Service, we just

don’t have the capacity to get a lot of work done. We’ve got a lot of vacant positions and that

causes a lot of other issues” (DST23). Turnover in personnel also leads to a persistent need to re-

educate and familiarize new staff with emerging DSTs and strategic fire management paradigms.

As one boundary spanner noted, “Now line officers’ turnover, on average, is about 20 percent of

them every year. So, in five years you’ve got a whole new set of line officers who need to become

knowledgeable at frames et cetera in fire management business” (DST2).

More specifically, interviewees noted a lack of personnel in the agency who have the scientific

background and analytical expertise to learn and use new DSTs. One scientist/developer noted

that existing technical experts on local Forest Service units are often too busy to take the time to

integrate new analytical approaches into their regular program of work. “What we find is that the

GIS professionals within the agencies, they’re so swamped with just their regular GIS path,

managing vector data set, making maps for different staff areas. The idea of doing … analysis

and detailed fire behavior stuff, they’re like, ‘Ah, no, I can’t do that.’ I think there’s a real lack

of people with the level of understanding and analytic skills to be able to help implement this

stuff broad scale” (DST16). Another manager noted that technical experts on local units are

often not directed to use new DSTs. “There’s also a problem with utilizing the folks with

technical skills to facilitate these kind of analyses where we might not be prioritizing or thinking

about what capacity we have to do more technical analyses” (DST22).

Interviewees also noted that existing analysts and scientists within Forest Service research and

enterprise units are oversubscribed. As one scientist/developer explained, “We're a fairly small

community of people doing the analytic work … It’s a lack of capacity building within the public

management agencies on the analysis side. We feel like there needs to be a whole lot more

people like us with enough understanding of the science to be able to explain it well and

understand it and to be able to do these analyses” (DST16). Another scientist/developer noted

that external analysts struggle to generate local ownership in new DSTs. “The biggest gap we

have is there is a small number of people who can actually develop the models and facilitate a

full workshop and most of them are here on our team. So, the other issue with it being located on

our team is not only the lack of capacity to cover all of the areas, but also that issue of

ownership. If it were a process that the regions could own or the forest themselves could really

engage with a little more directly, that would be, I think, a real positive” (DST3).

This lack of capacity also manifests itself in a lack of technical transfer specialists or boundary

spanners, who need to be comfortable working in both the realms of science and management.

As one boundary spanner noted, “I don’t think there’s enough people who act in that tech

transfer role. The managers that I work with the most are the ones who are inclined towards

technological, scientific approaches. There’s a lot of managers out there who aren’t. And I think

you need someone who’s familiar in both worlds in order to make that happen” (DST14).

Meanwhile, one scientist/developer argued that the individuals developing new DSTs may not

always be the best people to teach others how to use those tools. “I might not be the best teacher.

It’s not always the same person developing the tools that should be the champion or the one

16

doing the outreach for the tool, which I think is really true in science and research in general,

researchers aren’t always the best communicators” (DST5).

This lack of analytic capacity within the agency also impacts external scientists/developers and

boundary organizations that work with the Forest Service. Another scientist/developer noted that

the lack of analytical capacity in the agency ultimately leads to a lack of capacity in research.

“Instead of advancing research, we do a lot of the development and application piece because

we know it’s important to do that. We think it’s important to do that to affect real change, and so

we do it. But we should be on the research side, and we could continue to advance these tools if

we had the time to even sit down and do so” (DST27).

Funding issues also present persistent capacity issues and can manifest in different ways. For

example, one manager described a learning experience during the development of a new risk

assessment framework, for which the funding concluded when the final data was delivered but

did not include any funding to support the local unit on integrating that new data into the

program of work on the forest. “I didn’t really think that through when we first started down the

process, and so we’d used up all the money by the time we got to the end of it” (DST25).

Meanwhile, another manager brought up another issue, which is the perspective that the limited

funding that is available should ultimately be focused on getting acres treated. “How to get

funding into projects and get acres treated is my job. I always remind myself not to get wrapped

around the axel with all the other stuff that gets thrown at you, about the fancy new widget for

how to get something done” (DST12).

An ineffective process to integrate a new DST into a program of work can limit its outcomes

Interviewees referred to procedural challenges that can inhibit effective integration and

application of DSTs. One of the most referenced process barriers was a lack of participation or

missing participants, which can lead to a lack of buy-in and trust, especially for those who may

not have participated. For example, managers and boundary spanners alike noted the challenge

of getting collaborators to the table in the first place. One manager noted, “That’s one of the

bigger challenges is just to get people to sit around the same table” (DST18). Another manager

similarly explained, “We’re not quite there yet in terms of having complete buy-in from everyone

on what the priorities are but we’re working towards that, and that’s one of the challenges that

we’re starting to acknowledge, it’s hard to get all of the voices to the table to make sure you

didn’t miss anybody” (DST23). Furthermore, when integrating new DSTs into a program of

work is considered optional, a lack of buy-in can halt progress. As one manager noted, “It’s hard

to get something done that’s optional, yeah. People gotta really buy into it” (DST6). Simply put,

this can result in a lack of creative thinking. As one manager noted, “To some degree there’s

limited desire to re-think the processes that they have been using in the past” (DST22).

Many managers also noted that within the Forest Service, there are diverse perspectives among

personnel from fire to fuels to resource areas. One of the perspectives often missing that several

managers noted was resource specialist perspectives, which can include archeology, biology,

range management, and others. One manager noted that, in many cases, resource specialists have

17

been frequently asked to try new DSTs that have not led to meaningful outcomes, which has

resulted in some resistance to new DSTs. “I think a lot of resource specialists feel that, if it’s not

going to really lead to anything meaningful, they’re not interested in investing a lot of time into

it. And I can identify a little bit with where they’re coming from because sometimes they will go

through some sort of assessment like that and then we fall back internally in our ID teams to

what feels comfortable and what we know and which typically doesn’t always incorporate those

tools” (DST23).

Interviewees also noted that even when there is participation to develop and integrate a new

DST, that a lack of interest in the process and poor communication can create additional barriers.

For example, one scientist/developer described a process that lacked a champion in certain key

leadership positions. “I heard in the meetings … ‘Well, we only did this because we had to,

because 5140 tells us that we have to.’ So, there’s clearly no champion there, and I think that

work was never gonna really be used for anything important” (DST9). These types of issues can

arise, in part, because of poor communication from the appropriate channels about the need for a

DST. As one manager explained, “There wasn’t a lot of communication about the need for this

type of tool, or how this type of information would impact the ultimate users on the districts

coming up with their five-year plan” (DST22).

Several managers also noted that the process to develop and learn new DSTs takes time. An

efficient process can cause challenges by requiring an added time investment. For example, one

manager noted, “There was an ebb and flow in terms of how much engagement I had, and the

rest of the people not doing the analysis day-to-day had. That didn’t make for an efficient

process from my perspective … I had to go back and fix some things that I didn’t notice until late

in the game” (DST22). Furthermore, it takes time to learn new tools, and it takes even longer to

shift paradigms. As one manager noted, “The criticism that we hear is it takes a lot of time … I

went into the meeting thinking this, and I had a really good conversation, I understood why I

thought that better at the end of it, but that didn’t change anything, so I don’t know if it was a

good use of my time” (DST17). As one boundary spanner noted, there are tradeoffs between the

process to develop and integrate a DST and the DST itself. “And it’s always a fine line between

what is the process that you want to develop versus what is a tool you want to develop”

(DST11).

Technical challenges in the use of DSTs can impact perceptions of their accuracy and usability

Interviewees referred to technical challenges that create barriers to the use of DSTs and impact

perceptions of their accuracy and usability. The most frequently cited technical challenge was

limitations with the underlying data or models that serve as input to DSTs. As the famous saying

goes, “All models are wrong, but some are useful.” As one scientist/developer noted, “The best

way to uncover bad data is to start using it” (DST4). Nonetheless, when models are perceived to

be “wrong” it can lead to a series of challenges. As one manager put it, “For some people on the

ground-truthing side of things, it becomes … if it’s not 100 percent, the model has failed. If the

model doesn’t give me the answer of: the fire, will definitely, on day three, at approximately 4

PM, go up that drainage. And if it doesn’t, the model has failed” (DST7).

18

Model or DST limitations were something that scientists, managers, and boundary spanners alike

agreed create challenges. One of the limitations are the data that serve as input to DSTs. One

boundary spanner explained, “The outlook that we’re going to get for these DSTs is heavily

influenced by the data that is available. And if you don’t understand that limitation, you might

just run with the results without factoring in those considerations that are a little more difficult

to map” (DST14). Scientists/developers were also quick to acknowledge this challenge. “We

might have a false sense of knowledge from some of our assessments and those could lead to bad

decisions. So, I think we need to maintain our awareness of the limitations of our assessment and

our knowledge about, especially about natural disturbances” (DST4).

DSTs are limited in terms of how much and what type of information they capture and analyze.

As one manager noted, especially with firefighting, there are often many other considerations

that impact decision making. “As we look at a landscape, you know pre-fire, there’s so many

inputs and factors such as where we are at nationally with firefighting resources, where exactly

are the fuel moistures at, political considerations like fire restrictions … trying to do that stuff in

advance of the of an incident without knowing any of those details that I just mentioned is a

challenge” (DST20). One potential outcome of DST limitations is a resistance to new DSTs and

a reliance on what is already familiar. As one manager put it, “We tend to use models that we

have control over, that we understand better, and we set up ourselves. Because we know the data

coming in, we know the limitations of the data coming out” (DST22).

Data management issues can also impact the use of DSTs. If data are not stored somewhere

easily accessed, it can lead to problems during the development of DSTs or updates to existing

DSTs. One manager explained, “The issue is that as we try to update what makes this corporate

knowledge, there’s often a gap on where things were filed in the T drive, for example. Having

that information at a corporate level and consistent and updated at least annually, maybe more

than that, would be ideal. But I think that’s the struggle” (DST20). This can happen with data

collected during fires, as well. One manager explained, “When we used to make decisions on

wildfires, we would do all this analysis, and all this really good work. And then the fire would go

out, and that data, all that work, these reports, they just got left on someone’s hard drive, or they

got stored in a box in a district office and never to be accessed again” (DST8). Data

management is also a challenge across organizational entities doing longer-term, cross-boundary

planning. As one scientist/developer explained, “State GIS and federal GIS people have tons and

tons of data, but there really lacks a coherent way to bring the data together: stream layers, road

layers, veg layers, this layer, that layer. They’d show us 20 different layers and no one had

thought, well, how does all of this contribute to a decision in a sort of thematic way?” (DST4).

There are also inherent differences in data across geographic and temporal scales that can

contribute to challenges in developing and using DSTs. One manager explained there can be a

tension within Forest Service regions about what data sets provide the most appropriate outputs

for certain scales. “I have heard in the past where some of these regional modeling efforts have

used the best available data from the regional perspective, where individual forests might have a

better perspective or a better data set to use for their specific forest, or better modeling

19

parameters, or assumptions. And when you do it at too large of a scale, you lose that ability to

make it applicable for any particular place on the landscape when you run it across large areas”

(DST20). Certain geographic areas have numerous DSTs that overlap geographic scales, which

can create confusion in selection and interpretation of outputs for decision making. At the same

time, one scientist/developer emphasized that there are tradeoffs in accuracy and outputs across

geographic scales. “It’s an issue that when you downscale and there's some things that maybe

miss or maybe particular for one portion of the landscape. But when you’re making regional-

level decisions of how you might allocate dollars or resources, the Forest Service likes to

maintain that continuity so that the region can do that” (DST27).

Interviewees also noted a desire for one DST that can make decisions across all geographic

scales. One boundary spanner said, “A risk assessment at the national level is not going to look

anything like the risk assessment at the regional level, and that’s not going to look anything like

the risk assessment at the local level. They’re making different decisions, and accordingly, they

have different considerations. And asking one tool to effectively make decisions across multiple

scales … We’re asking more than we can possibly get” (DST14).

Another technical barrier that interviewees mentioned are technology malfunctions and

limitations. One manager noted that if a DST does not work in the field, people are likely to

resort to the tools with which they are familiar. “When you’re running a fire, if you have to

monkey with your iPad, and now this isn’t working and whatever’s going on, you’re just going to

go back to your steno pad. They don’t have time. People that are implementing this stuff don’t

have time to mess with it” (DST17). Integrating new DSTs into existing platforms can also take

time to work out successfully. One manager noted, “When we put PODs in WFDSS, they show

up with the same ugly, fat boundaries and you only can give them one value … It’s really clumsy

right now and hard to use in WFDSS” (DST21).

Lastly, interviewees noted that due to the limited longevity of some new DSTs it can be time

consuming to keep up with changes and can create frustration and confusion. One manager

referred to recent changes in day-to-day computer platforms in this way, as well. “We used to

have the JKNL drive, then we went to Pinyon, now we’re going to Teams ... The tech keeps

changing, but it’s hard for people to keep up quite frankly” (DST17). Meanwhile, one boundary

spanner noted that a possible solution to limited tool longevity is to develop smaller-scale DSTs

tailored to a unique context. “I’m very much well aware of the findings that people have seen

about decision support system use and longevity … Potentially, if you’re not spending a ton of

money on things, then it’s easier to make something that’s very tailored to a specific use in an

organization without making it overly burdensome to develop something that might be replaced

in two or three years for something that’s newer and better and fancier” (DST11).

20

Complex information in DSTs can be difficult to understand and can lead to confusion and

information overload

Boundary spanners and scientists/developers working to develop and apply DSTs acknowledged

that the concepts and information can be quite complex. As one manager explained, there is

simply information overload. “We just have a hard time, almost like information overload. The

fire managers, we feel like we have our hands full just trying to manage people and fleet and

budgets, et cetera. And, so these new tools, or new to some of us, they’re difficult to understand,

and if you don’t use them, they’re easy to forget” (DST20).

Another source of complexity is a lack of a common language to describe risk. One boundary

spanner explained, “Sometimes it just gets confusing for people if there’s so many different

options out there of how you prioritize, what a risk assessment is, and what it means. Our group

has tried really hard to standardize that and tried to make sure that everybody’s using the same

language” (DST10).

Another scientist/developer noted that it can be hard to select a DST without a foundation in

certain scientific and technical concepts. “There’s still understanding the fundamentals. The

basis of a lot of the things we do is fire behavior modeling, and there is definitely, I think, some

barriers to people understanding where to start or where to learn just the basics to fire behavior

modeling. Yeah, you can get into the system and click the buttons, but are you producing

meaningful outputs?” (DST5).

Managers also noted that the large number of DSTs available can be overwhelming. “It

definitely can get overwhelming to even stay up with everything that a ton of different folks are

doing, is really difficult to do. I don’t know if there’s an answer for it. I think every tool could

have a use. I'm not convinced they all do all of the time, but with the ever-growing list of tools

that are available out there, that’s more time that people need to learn a little bit about every

single one of these things” (DST24). Scientists/developers also noted that the sheer amount of

information available and the numerous different approaches for conducting a risk assessment

and prioritization process can be confusing. “It’s really easy, even for somebody like me who

now works in this stuff day in, day out, to get confused. And, so it can be really easily confusing

for the layperson and even for the fire management officer on a forest or a land use planner for a

community or a forest supervisor who doesn’t work in wildfire to understand this” (DST16). In

some cases, there is also a lack of awareness that certain DSTs are available in the first place.

Interviewees noted that even when there is an awareness of existing DSTs, there may still be a

lack of understanding of how to use them. As one boundary spanner explained, “Most people are

somewhat familiar with the different fire models that are in existence. But it still seems to be

challenging for some incident management teams to get their heads around how exactly they’re

going to use that information” (DST8). One scientist/developer noted that it takes time and a

concerted effort to explain the logic behind certain DSTs and develop an understanding of how

to use the outputs. “There’s a lack of understanding of the information fundamentally, and then

a lack of its intended use, I would say both … It’s fairly complicated, and it took a long time to

21

really slowly explain and work with everybody on explaining that” (DST27). Indeed, there is a

learning curve associated with any new concept. As one manager noted, “The short version is it

took them two years to get the local folk online and on board” (DST21).

Finally, interviewees noted that there can also be a lack of understanding or misunderstanding of

model outputs, which can be problematic for decision makers. As one boundary spanner noted,

“The pro is, you can do some predictive modeling, and get an answer, and have something as a

discussion point. The con is, maybe there’s more people in there that are running models that

don’t necessarily know exactly what they’re doing. And maybe they’re making some mistakes,

and that can be misleading to a decision-maker” (DST8). One boundary spanner explained that

this can turn people off to using DSTs in the future. “The danger though is people start using a

lot of different analytics and if some of them don’t work then some people will get put off by any

analytics … because they’ve had a bad experience with maybe one of them” (DST2). Or, as

another boundary spanner argued, if DST outputs are problematic, decision makers may move

forward without them. “It either validates what they’ve been thinking and then they’ll use it to

support … or it is contrary to their beliefs, and they move forward without it” (DST15).

Institutional issues and governance barriers can inhibit the use of DSTs

Institutional issues within the organizations responsible for developing, transferring, and using

DSTs can inhibit their use in several ways; although, the barriers within organizations tend to

vary widely based on the context. As one boundary spanner noted, “There’s probably different

kind of barriers to use within different types of organizations. And I spoke a lot about a non-

profit using risk assessment tools. I engage with agencies using risk assessment tools within

collaborative processes. And then I think I gave the example of someone trying to use IFTDSS

within their own organization. And I think there’s unique barriers in all those different contexts

depending on exactly how they view their use of those tools” (DST11).

Within land management agencies like the Forest Service, interviewees explained that one of the

major institutional barriers is a lack of direction on which DSTs to use and how to use them.

Across the administrative levels of the Forest Service, interviewees expressed a lack of clear

direction and alignment on DST use. As one scientist/developer noted, “The leadership intent,

they don’t do intent until they’re educated about a potential or an opportunity to do intent with a

decision support tool. But they won’t get educated unless a scientist spends a lot of time

somehow educating them on these, which is a difficult process” (DST4).

A lack of leadership intent regarding DST use can result in confusion across different

administrative units of the Forest Service. As one scientist/developer noted, “There just hasn’t

been a lot of help out there to direct forests on, okay, so now how do we take that next step of, we

have all this great information, how do we integrate it into our program of work?” (DST9).

Although there is a direction in Forest Service Manual 5140 to use risk-informed decision

making for pre-fire planning, some interviewees noted there can be a reluctance to use the

outputs of those analyses once they are available across all levels of the Forest Service. Or in

other cases, as the same scientist/developer noted, DST outputs just sit on a shelf. “It just seems

22

that a lot of them, they get done, and they get put on the shelf, and then they’re not really

integrated into a statement of work or a program of work of any particular unit” (DST9).

Another issue that interviewees cited specific to the Forest Service are restrictions about the use

of technology. One boundary spanner described this as CIO governance. “The way I would

describe it is just general computer technology. We call it CIO governance that enables us to be

able to do the tech transfer of research to mobile devices or to computer applications. There just

seems to be a delay on the government side to effectively support emerging research to get some

of their ideas or some of their tools to the masses … One of the biggest impediments is

technology and the advancement of things is moving so fast. It’s as if the government can’t keep

up with those changes because we’re not able to adapt as quickly as technology changes. So,

you’ve got researchers that are coming up with really cool stuff, and they’re usually able to

leverage the new IT technologies” (DST8). There is also limited IT capacity that results in

analysts devoting time to IT issues. As one boundary spanner noted, “If I had half a dozen folks

in IT to help, then I could free up my great analysts and tech transfer folks to do more tech

transfer … To be fair to IT, they’ve been shorthanded, and it’s very difficult for them to hire

folks” (DST2).

There are also barriers within scientific organizations. Specifically, interviewees noted that the

science reward system does not incentivize scientists to work on development and application of

DSTs for end users. As one boundary spanner noted, “If a scientist makes something really

useful that the field needs, there’s not a lot of incentives on their end to keep it going because

that’s not really how they’re structured in research” (DST1). Another scientist/developer noted,

“The struggle is, someone who’s supposed to be in research is balancing that and the cost of

doing that with carrying out a research function because you’re not rewarded” (DST4).

Lastly, one boundary spanner noted, infrastructure with which new DSTs can be incubated and

tested is largely nonexistent. “We struggle with, how do we develop stuff? How do we answer the

fields’ questions? Once we’re onto something, how do we grow it, and then implement it, and

teach it, and maintain it? We just don’t really have a mechanism to do that” (DST1).

Strategies for integrating DST outputs into decision making processes are often lacking

Many interviewees highlighted that one of the final challenges in the process of developing and

using DSTs for risk assessment and treatment prioritization pre-fire planning is a lack of

implementation plan for how to integrate DST outputs into existing decision-making processes.

As one scientist/developer noted, there is a tendency for efforts to develop new DSTs to end with

the data itself, and less attention is paid for how to integrate the new information into a planning

process or program of work. “A big issue is, then, the forest and regions, they get the data, and

they say, ‘We have our QWRA, now what?’… ‘We have our PODs, now what?’ And people don’t

know how to interpret it, so they need a little bit of assistance or a little bit of additional training

on how to interpret it” (DST9). One manager had a similar comment that highlighted that there

is often a lack of implementation plan for using new information. “We have a host of people

using this data that aren’t as savvy with the data and all the uses … there’s so many different

23

ways to use it, and I just am skimming the surface with what I put together for our folks to use”

(DST25).

One boundary spanner noted that even when end users can run new DSTs, they may not know

how to effectively use the outputs. “You might be able to run something like a linear

optimization tool and get a table output, but then for someone to take that and attach it spatially

to that data, visualize it, put it into a GIS and those kinds of things. Those are often next

barriers” (DST11). Another scientist/developer had a similar observation. “The tools aren’t

challenging, I mean they take a while to build, but that’s not the hard part. It’s the interpretation

of the outputs that you’re using to make your decisions and really having solid foundation and

training to understand what you’re looking at is really important. And that’s something we

struggle with a lot, being the ones developing the tool because our goal is to develop the tool.

It’s not necessarily to teach the science. We have to, of course, teach and train to a certain level,

but I feel like getting other people to really take the torch and understand and teach the

foundation and the understanding of how you use these outputs and maps and charts to make

more informed decisions is really important, and it’s really challenging for those developing the

tools” (DST5).

Interviewees also expressed concerns that implementation of DST outputs, especially during

fires, can be prevented by decision-makers at a higher level for various reasons. For example,

one boundary spanner relayed a concern that had arisen at a PODs workshop. “What I’ve heard

from the field … is that they are concerned that they’re going to have these plans in place and

then upper management won’t let them implement it. And even though they’re drawing these

PODS, they’ve got these great plans in place, they feel like if they get a fire in certain areas, and

they’re like, "Oh, we should let it burn,” that somebody much higher than them is going to come

in and say, ‘No’” (DST10). The same boundary spanner noted that there are also concerns about

a lack of resources to implement a DST like PODs during a fire. “They’re concerned that if they

have a POD where they feel like they can manage a fire successfully, but managing that fire

requires them to have a certain amount of resources available to do just that, if they decide to

manage that natural ignition, they go very low on the totem pole in terms of resource allocation

need. So, if a higher priority fire needs the resources that they have, it gets pulled off from their

fire, and then all of a sudden, the plan that they had in place can’t be maintained because of

resource shortages … The ability for the forest to maintain resources on these managed fires …

seems critically important in my mind” (DST10).

Decision Support Tool Benefits and Facilitators

DSTs spur new conversations and facilitate communication

One of the most frequently mentioned benefits of using DSTs in pre-fire planning is that they can

help to spur new conversations among fire staff, line officers, resource specialists, and other

decision makers and stakeholders. Interviewees noted that DSTs help to generate discussions

among staff that were not common in the past, especially when the development of a DST

involves collaborative participation from end users. As one scientist/developer noted in reference

24

to PODs, “Through all of this process, one of the biggest and most beneficial outcomes is

conversations between the fire folks and line officers and their resource specialist staff.

Conversations that have rarely happened in the past. The fire shop is even in a separate

building, typically, and they don’t engage and speak with the other Forest Service staff on a

regular basis, so it has helped bridge that, and those are important conversations that can lead

to downstream beneficial decisions” (DST27). When managers from different resource areas or

functions of the agency gather to discuss pre-fire planning, a DST and its outputs can serve as a

common space from which to share varying perspectives.

Although DSTs may have flaws, they can still provide a forum for joint decision making. As one

boundary spanner noted, “I think the value is in using these tools as a communication tool in

order to jointly make decisions. Or, at the very least, understand the perspectives of other people

in the audience” (DST14). Similarly, another boundary spanner noted that DSTs can facilitate

landscape-scale thinking, “I think that the decision support tools work really well as an excuse to

get people to the table and start having conversations at a landscape scale” (DST15). DSTs can

also facilitate communication across levels of the Forest Service. For example, one boundary

spanner provided an example of a case where the use of PODs on a fire had allowed a District

Ranger to communicate decision making rationale to the forest supervisor and regional forester.

Another manager mentioned that they are important for documenting decision making, “Why

folks made the choices that they did early on” (DST7).

DSTs can also facilitate conversations between land management agencies and the public. As

one boundary spanner noted, “We’re talking with [sic] our partners about this stuff long before

the fires ever occur” (DST1). DST outputs can provide communication tools that allow fire

managers to better explain their decision-making rationale to the public. As one boundary

spanner noted in reference to PODs, “It can be a great communication tool to explain why we’re

actively suppressing a fire on one portion of the landscape, while we’re letting a fire burn in

another portion at the same time. Realizing that all fire in all places is not the same” (DST10).

DSTs can also help bring community members into the conversation about pre-fire planning and

suppression during fire events by demonstrating the amount of planning that goes into addressing

fire events. As one scientist/developer noted, “I think the more we can bring public and

community, especially engaged community organizations, into those conversations to show

people that we are really trying to be as thoughtful about how we approach wildfire as possible

and to do as much pre-planning, pre-thinking as possible so that when events happen, we’ve

already thought through how we’re going to deal with them” (DST16).

DST outputs, such as visuals and maps, are essential communication objects that allow DSTs to

facilitate discussions and joint decision making. For example, one manager referred to the

process of developing POD boundaries to capture the collective knowledge of a group. “We had

a bunch of pre-made maps out. And the good part of it was we had the fire management officers,

line officers drawing maps together, putting lines on the maps and thinking about things, from

all the various history that was in their heads in addition to all the data that went into the

PODs” (DST13). Visuals developed from DSTs can also facilitate creative thinking. As one

25

manager noted, “That graphic illustration gives you some pause and makes you ask some

different questions. It doesn’t give you any answers. It just helps understand a frame for the next

choices that you’re faced with” (DST7). Maps and visuals are also helpful in explaining the

potential impact of a fire to the public and can help managers more effectively make arguments

for fuels treatment and restoration work. As one manager said, “Being able to explain the real

need for doing this is really what we’re trying to do, and that risk assessment, to be able to show

them a map that, overall, this one area is very high risk and the potential impact to the values

could be very severe. That really helps, just visuals, really help tell the story to the public, and

that’s what we’re going to use it for right now” (DST 25).

The process of developing and using DSTs impacts their perceived effectiveness

One of the key facilitators in the development and use of DSTs is a collaborative process in

which DSTs are co-developed with multiple perspectives in an iterative process that facilitates

building relationships, trust, and respect. This is often referred to as the co-production of

knowledge, which is a collaborative process to co-develop actionable scientific information for

decision-making. Organizations that span the spheres of science, decision making, and policy,

often referred to as boundary spanning or bridging organizations, tend to have unique functions

that allow them to facilitate the co-development of scientific knowledge and DSTs. Interviewees

frequently referenced these concepts as facilitators to the development and application of DSTs.

As one boundary spanner noted, the process of developing DSTs is critical to their perceived

effectiveness. “I think that all these tools have flaws. I think that it’s really hard to get over a lot

of the flaws. The value that I see in these decision support tools and these planning tools is in the

process. You all gather around what the tool shows you and how that aligns with your current

understanding of the landscape versus your colleagues’ understanding of the landscape”

(DST14). One manager put it this way, “The biggest thing was partnership buy-in and

collaboration … These tools are helping us literally come around the table and agree to doing

the same thing, regardless of who you work for” (DST18).

Interviewees referred to numerous factors in the process of co-developing DSTs that help to

enhance their role in decision making. Interviewees noted that an iterative and interactive process

for developing, testing, and using DSTs can increase their operational relevance. For example,

one scientist/developer noted, “And, so it’s kind of this interactive process of bringing

information to the field and having them confirm or reject … This iterative process of working

with the fire managers so that they’re owning it, it is their information. We’re just helping

support the development of it and learning where our model’s working well and where our

model’s not working well became really powerful” (DST3). One manager also highlighted

positive outcomes from an iterative process, “It was a conversation, yes. Let’s move forward on

it, and then we kind of revisited things as it progressed. But it wasn’t one get together about, ‘we

like this, let’s do this.’ It was an idea … Let’s see where we can work with it and then presented

it to the district ranger and got the support” (DST6).

26

Another important factor in the co-development of DSTs is the value of having multiple

perspectives represented, both in terms of providing necessary information, as well as in

generating support. As one boundary spanner explained, “Some of the factors are that the

managers themselves, the FMOs, the engine captains, up to the rangers and the forest

supervisors, the resource specialists, they were all involved in the process … The fact that lots of

people were involved, the fact that the Forest Service had meetings with their cooperators after

the fact in order to give them updates on what was happening, I think that helped generate some

support and buy-in and trust” (DST14). Furthermore, participation in the co-development of

DSTs from leadership and support from leadership for using DSTs can drive progress. As one

manager noted, “The fact that leadership was involved and pushing to get it done, and to come

up with a new way of looking at the problem was very useful” (DST22). Overall, an iterative,

participatory process helps to generate critical support and buy-in from end users. As one

manager explained, “It was good to have resource specialists involved in the discussions and

working groups to assign response function. And I think it created a fair amount of buy-in

because having the right people in the room to be a part of the process is more or less a verbal

agreement about the end products, especially with agency or line officers, so if that situation

arises where there’s a disagreement about how to manage a wildfire, there’s at least a start

point to say, ‘Hey, you remember we talked about this?’” (DST26).

Boundary spanning or bridging organizations were also referenced by interviewees as key

facilitators. Examples of boundary spanning organizations include the Southwest Ecological

Restoration Institutes in Arizona, Colorado, and New Mexico, The Nature Conservancy, and the

Forest Service Enterprise Team. Boundary spanning organizations and boundary spanners, the

individuals who help to communicate and facilitate across science, management, and policy

spheres, provide added capacity in terms of science and technology expertise, as well as

organization and facilitation. One manager noted, “Having CFRI [Colorado Forest Restoration

Institute] as someone who can do the legwork of a lot of the spatial analysis and with all of the

capacity issues that we have, is really important … I really feel lucky that we have CFRI

available to use, and I know a lot, some other forests don’t have that, but having that is really a

critical piece, I think, to our capacity” (DST23).

Boundary organizations can also help provide access to scientists and model developers who can

answer questions and provide guidance as DSTs are integrated into workflows. One manager

noted, “Being able to talk specifically with the person that created the model and had a

knowledge of how it was supposed to work, and then also the person that was doing the

modeling, to be able to provide guidance in terms of limitations of the data to some degree was

also useful” (DST22). Or as one scientist/developer described it, “It just takes some facilitator

out there working with them to help them explain and understand it,” (DST27). Boundary

organizations can also help to make DSTs more accessible to their intended users. One boundary

spanner explained, “I’m making a lot of these risk assessment tools more accessible so people

can use at, say, low cost … because that seems to be a major barrier, is you can go out and find

a lot of this information, but most people don’t have the skills to link it together. So most of the

work that I’ve done in this space is just making things more accessible” (DST11). Boundary

27

organizations can also help build credibility by providing access to local, place-based science. As

one boundary spanner explained, “We can point not to nearby mountain ranges but literally out

the window and say we collected from right there…do that in a way that’s completely grounded

in experience and science … It makes it really hard for any of those anti-management groups or

especially outsiders who are anti-management to come in and try to derail things” (DST15).

Another key element of the process of co-developing and applying DSTs is the creation of new

relationships, continuation of existing relationships, and facilitation of trust and respect. One

boundary spanner explained the importance of getting to know the people on a team to gain trust

and credibility. “I think what’s critical is getting to know the people that you’re working with on

a team and really gaining their trust and demonstrating that you have competency and

something to offer them as far as the products that I would bring to the table … A successful tech

transfer, it has to be through somebody that’s trusted by the decision-maker and the people that

are in the field doing the work” (DST8). One scientist/developer noted that it takes time and

iteration to build trust and relationships and have candid conversations about what may or may

not work. “So once there’s shared mutual respect across the room, then we could just have

conversations and work through all of these over time and get there” (DST27).

Interviewees also noted that consistency in participants and limited turnover help facilitate

building relationships, trust, and respect. As one scientist/developer explained, “There’s one or

two people on that forest that I would refer to as those champions. They came to us; they wanted

the risk assessment; they saw the application for it; and they haven’t retired or took a new job or

anything; they’re still at the forest” (DST9). Other interviewees highlighted the importance of

local champions who can articulate the need for a DST, drive progress, and create buy-in and

support that leads to use of new tools. One boundary spanner noted, “Another important factor is

the cheerleader. I think in order for these efforts to be successful on each landscape, there has to

be one or two people who really act as a champion, who want to use it, who know how to use it,

who understand how it was constructed … That’s the magic sauce. You need the people to really

want to use the tool. And I think they’re more likely to use the tool if they were invited to help in

its development and implementation from the start” (DST14).

DSTs can enhance the legitimacy of decisions

One primary facilitator to using DSTs is an inherent part of their name and existence, that is,

they support a desire for better information and facilitate the use of that information in decision

making. This is especially important when it comes to decision making about where to provide

funding for hazardous fuels reduction and forest restoration work. Managers and decision makers

are motivated to have a better narrative about why, where, and when they plan to spend money.

DSTs can provide information that justifies funding. As one scientist/developer noted, “They

[Forest Service leadership] want better analytics to understand what outcomes can be produced

from investments” (DST4).

28

Another benefit is that DSTs can serve as a centralized repository for information. As one

manager explained, “There's some utility in having a central spot for all this information that's

preloaded and pre-thought out, with a caveat that it's highly subject to change, right?” (DST12).

In a similar vein, DSTs can help to quantify existing knowledge and provide access to

information in a more effective format. As one manager noted, “One of the big benefits was

coming up with the data set across the landscape that allowed us to quantify all this information

to a degree that no one's had access to before” (DST22). Developing DSTs with local input can

help to capture and institutionalize local knowledge and experience, which can otherwise be

difficult to provide to new staff when there is turnover or retirement. As one manager explained,

“Some of the resource folks, maybe they're not from the area or haven't been in their job for a

decade. The line officers as you know, tend to come and go fairly frequently. We don't have any

line officers that have been here for 10 years, for example. And, so that institutional knowledge, I

feel it lies in a few hands on the forest. Some folks have that. But if those folks retire … that

information is lost. But if you capture it in a process like PODs or something, then it's more

institutional. It's more corporate knowledge, it will be retained” (DST20).

DSTs are also seen as facilitators when there are apparent outcomes on the landscape with

respect to planning. For example, when DSTs are used in a planning process, they can

demonstrate how previous work has helped lower risk on the landscape. For example, one

boundary spanner noted, “That was an interesting tool to be able to have that … area in the tool

because it really allows people to look, take a step back, and say, wow, this work does show up,

at least on the map, as being significantly reduced. And there’s this nice hole in the middle of the

map that’s relatively low risk compared to everything else” (DST15).

DSTs can also facilitate increased decision quality or confidence in decision making. For

example, during a fire, DST outputs can facilitate trade-off analyses, create understanding of

what really makes a difference in a decision-making process, and generally support risk-

informed decision making. When fire outcomes are positive and DSTs were used to support

decision making on that fire, there is often a positive feedback loop whereby there is increased

confidence in the DST itself. For example, one boundary spanner noted, “These fires might have

been managed the same even if PODs and the risk assessment were not in place. I think they

provided the justification. I think in doing so they helped provide confidence to the decision

makers, that they were making the right call” (DST14).

Interviewees noted that DSTs can also help to create public confidence in the decision-making

process. One boundary spanner explained, “Doing this painstakingly tedious planning and

priority identification ahead of time gives the agency a lot more flexibility when it comes to

setting up a plan of work for five years that include areas where they’re going to do a NEPA

analysis” (DST19). One manager explained how the science behind a DST provides an added

level of credibility to decisions. “Having more of that scientific or analysis background and

being able to show that to some of the concerned people about your project, it really helps clear

the air and make it clear to them that you’re not just willy nilly picking things to meet a timber

29

volume or because it’s a fire program that really likes to put fire on the ground. It really helps

put that solid reasoning to it” (DST25).

Lastly, interviewees noted that DSTs can help to encourage creative or strategic thinking. One

manager put it this way, “It feels like a new generation, a new culture in resource management

that’s recognizing that we’ve got some really cool technology and a lot of smart people that can

help us make better decisions” (DST18). And this is especially important today as fires continue

to increase in frequency, severity, and complexity. As one scientist/ developer noted, “The vision

that I’ve always had for it is: we are bringing a fire manager perspective forward and into the

planning environment. Because it’s such a critical issue today” (DST27).

Recommendations

Interviewees were asked to provide recommendations for improving the development,

integration, and use of DSTs in wildfire risk assessment and treatment prioritization planning.

Their recommendations are organized below according to their major topical areas.

Analytical capacity and funding to develop, integrate, and use DSTs needs to be increased

One of the most common recommendations across interviewees from science, management, and

boundary spanning organizations is a need for more people with analytical expertise to assist in

the development, integration, and use of DSTs. This is consistent with other research findings, as

well (Greiner et al. 2020, Rapp et al. 2020, Schultz et al. 2020, Thompson et al. 2019). One

manager explained, “Nationwide, having more analysts is important because those are the

people that can, at least, if they can’t run them all, they can understand what went into it and

explain it and interpret it. Right now, there’s not very many folks that are able to interpret a lot

of these tools out there” (DST24). One boundary spanner noted, “We really need to have more

analysts that can create some of the analytical tools that we use to develop these PODs and the

Quantitative Risk Assessment … And we need somebody that can help update these and keep

them relevant and maintain them year to year” (DST10). Another manager noted the importance

of having access to an analyst when making decisions. “I want that analyst in the room listening

to me, having these conversations with the stakeholders, and be thinking about what could help

them, not tell me the answer, but help me have this conversation” (DST7).

The ways in which additional analytical capacity could be realized varied among interviewees,

but in general, there was consensus that there should be, at a minimum, personnel within national

forests or regional offices tasked with learning DSTs, as well as translating and transferring them

to intended end users. As one boundary spanner stated, “We need more analysts. It’s that simple

… I wish we could have that capacity at every one of those forests. And then we would have

national vision and direction to help them” (DST1). Or as one manager put it, “If we want

people to be using these tools, all the forests need to invest in people that can use the tools”

(DST25). One boundary spanner also noted the importance of local analytical capacity in terms

of developing trust. “We need to have more technology transfer people that are positioned

around the region … these people, ideally, would have relationships with the fire community that

30

they’re sitting with, so it’s not just some random person that comes in” (DST8). Another

manager noted that this could also include building more capacity locally. “It would be more

people, or prioritizing people that we have, at the local level, building skills to do this kind of

work” (DST22).

For some managers, the lack of appropriate analytical capacity is due to inconsistency in

positions across forests, as well as a lack of funding for positions. As one manager noted, this

inconsistency warrants investment in positions dedicated to analytical work across national

forests. “It depends on the forest, some forests have people that are good at this stuff, some

forests don’t … If we want people to be using these tools, all the forests need to invest in people

that can use the tools” (DST25). Similarly, another manager noted, “There’s not positions whose

only job, It’d probably be like two positions or so, whose only job is to try to operationalize all

these analysis tools. I mean, we’ve got them at the research station, ERI, CFRI, but it’s getting

the field to trust that side of things that is going to continue to be the difficult part” (DST24).

Interviewees also talked about the need for additional assistance from boundary spanners and

boundary spanning organizations to explain the need for DSTs, interpret and communicate

outputs, and facilitate conversations. As one scientist/developer noted, “If you get a bunch of

analysts, they need to be able to communicate in some fashion, and those skills don’t always

come together. And, so there’s a broader need for those folks that are good communicators and

good workers with public or collaborative groups … We need a capacity on both sides. Those

folks that can communicate and work with groups, as well as those folks that can create the

analytics. And both lack … There, of course, are good people out there that can do it, they’re just

tasked with other things” (DST27). One manager described this as a bridging function, much

like the role that boundary organizations already play. “We need people to learn the computer

side, the data side of things and we also need them to have operational experience so that they

can translate it to people that only have operational experience … Something like a bridge

between that data and that operational, those two sides” (DST24).

One of the major capacity barriers noted by interviewees was that there is simply a lack of time

to devote to learning and integrating DSTs into a regular program of work. One recommendation

to address this barrier is simply acknowledging the time needed to integrate a new DST. As one

manager explained, “My recommendation would be to understand that it could take a full year,

maybe even a little bit more, to complete the process so you can actually start incorporating into

your planning efforts, and the acknowledgement that you’ve got to build in that capacity for

when you hit the field season, and because the people who are the implementers are really

important to making sure that this process is completed” (DS23). Acknowledging the time

needed for integrating a new DST is critical, because as one scientist/developer noted, it also

takes time to develop trust and buy-in with a new concept. “If we get it in a vacuum, simple, fast,

done. But if you really want to bring people along and have a shared stewardship or shared

ownership of the outcome and the direction you’re going, that’s what takes time. And I think

that’s critically important and should be pursued, but that’s where time goes and where

investments are needed” (DST27).

31

Respondents also noted that it is critical to ensure funding to support analytical work and the

integration of new DSTs. “I think a lot of the intent is being met supporting the ground with

these analysis and scientific support for field operations. But money could be leveraged much

better if there was kind of a cohesive agreement that this was going to be a priority” (DST18).

As one boundary spanner noted, a lack of capacity is due largely to a lack of funding. “The

personnel capacity, I think, is really limiting. Just more funding” (DST11).

Communication about DSTs must be done in a common language and sustained through time to

build relationships and trust

According to interviewees, different elements of communication can be barriers or facilitators to

the development and use of DSTs, but interviewees had recommendations as to what elements of

communication can be improved to better facilitate the use of DSTs. Respondents across groups

emphasized that a common language is needed to discuss risk and DSTs. The importance of a

common language has also been emphasized in GTR349 (Thompson et al. 2016). Developing a

common language also relates to appropriate education and training on structured, risk-informed

decision making, which is addressed later. As one manager explained, “Now also the other piece

that we are leaning into is with agency administrators, decision makers, line officers, to require

some training and education so we get to some common language and understanding about their

role in decision making and what quality decision making is, what risk-informed decision making

is” (DST7). One boundary spanner explained that establishing a common framework and

language is critical in engaging with partners across a landscape. “Have, at least, an established

framework, so that everybody’s cross boundary communication can be the same … Making sure

that everybody’s using the same terminology and similar methodology, the same framework at

least, so that you can talk to your neighbors about what they’re doing” (DST10).

In addition to establishing a common language, interviewees also noted the importance of setting

expectations up front for collaborative, pre-fire planning efforts, especially using DSTs. One

boundary spanner explained, “From the outset, set the expectations, like, ok, we’re going to this

big landscape-scale planning, and then we’re going to use it to find funding; we’re going to use

it to build community support; and we’re going to use this to get some stuff done. And just keep

coming back to the plan and keep coming back to the purpose” (DST15). Respondents also

emphasized the need to ensure that key people who will be using a DST are involved in its

development from the beginning. “Those key groups of people that are developing that and/or

the people that it’s being developed for, I think, they have a good understanding of those

assumptions, limitations. And to me, the best way to communicate is right there on the front end”

(DST20). Another manager echoed this sentiment. “I think it’s coming up with a more robust

conversation before you even enter into running a model. Really talking about what do we need

the model to do, what information do we need to know, what do we need the model to output, and

whether that model is the right one to use for any particular situation. Having those

conversations together with whoever designed the model and the people deciding to use the

model, and then the end user who's actually going to use the output. Because it may be a line

officer that decides to use the model, but it’s the people on the ground at the district level or at

the forest level that are actually going to use those outputs” (DST20).

32

In addition to engaging end users from the outset of DST development and integration,

respondents also highlighted the importance of sustaining that engagement throughout the

process. End user engagement can help to improve scientist/developer understanding of what

elements of a DST are most useful or viable in each context. As one scientist/developer

explained, this is often an iterative process with give and take from both ends. “It was an

interesting process of trying to be the facilitator and the guider of the process with some

background in how what they were doing would affect the outcome. And, so trying to nudge it in

the right direction when I felt like it was appropriate, but then also making sure that the local

specialists participating had ownership in the process. And, so there were times where maybe the

way that they characterized the fire effects on a particular HVRA, I didn’t a hundred percent

agree with, and I would push back a little bit and say, ‘Well, don’t you maybe want to consider

this or that?’ And they would push back and say, ‘Nope. On our forest, that’s how we feel about

it. This is how we want to characterize the fire effects.’ … That was definitely something that was

a learning experience for me to try to figure out how to walk that line, because in the end,

they’ve got to trust and then have ownership and belief in what they’ve produced” (DST16).

Ensuring that there is a common language, as well as communication and engagement,

throughout an effort to develop and integrate a DST helps to build relationships, trust, and buy-

in. Effective communication can help forge relationships between scientists and managers,

which, as one manager noted, can also ensure progress. “Where the rubber meets the road, is

where you have and you forge these individual relationships, either which way the road’s going.

Whether it’s the scientist approaching the manager or the manager approaching the scientist”

(DST12). And as one boundary spanner noted, trust is a critical element of success. “A

successful tech transfer, it has to be through somebody that’s trusted by the decision-maker and

the people that are in the field doing the work” (DST8).

Another communication challenge identified by respondents was the issue of information

overload with new DSTs. As one manager explained, it is important not to overwhelm managers,

“I think PODs is a good thing. If we were to develop something outside of there, I think we

would just want to be careful that we’re trying to integrate it all and not overwhelm fire

managers because it seems like that’s easy to do” (DST20). But as one scientist put it, there also

needs to be a balance between understanding the role that a DST can play in informing decisions

and appreciating how DST outputs can lead to changes in conceptual understanding over time.

“Decisions come in many, many small baby steps. No one makes radical decisions because you

can't in our climate or business climate that is highly collaborative, highly interactive” (DST4).

Interviewees also talked about the importance of in-person engagement to facilitate

communication, though as one manager noted, that was complicated by COVID-19 in 2020. “We

can’t sit down with his folks. It has to be done with the electronic versions of the map, where

normally you’d have all your heads together, and go, ‘What about this one? Well, that one

doesn’t work, but we razed the crap out of this one, so this one will work now.’ You know, that

kind of thing” (DST21). Interviewees also talked about sustaining communication to ensure that

continued attention is paid to new approaches. As one manager explained, sustaining

33

communication is a way to maintain momentum for change after initial planning efforts are

completed. “Being available and continuing to engage with the folks that are going to be the

ones making decisions on these fires, that’s going to be the most important thing” (DST24).

Lastly, interviewees highlighted the importance of sharing success stories and case studies that

provide examples of successful DST integration in new contexts. For example, one boundary

spanner said, “I wish there were better examples of case studies that showed how other places

have gone through the process that I just described us going through, and not to say that it was

better or more appropriate, but to give the folks that I was coaxing to the table more confidence”

(DST19). Success stories can also move the needle in cases where there is skepticism of fuels

treatments. As another boundary spanner explained, “A risk assessment in itself is not going to

convince someone who's dogmatically opposed to forest thinning or chainsaws or prescribed fire

or anything like that. That’s not going to happen, but if you can put a risk assessment in front of

a skeptical city counselor and walk them through it, having that sort of a success case … helped

move some of those conversations forward” (DST15).

Implementation planning can ensure that DST outputs are used

Once a set of DST outputs are developed, interviewees emphasized the need for a plan to ensure

that those outputs are integrated into decision making and that data and outputs are updated as

needed. One manager emphasized the need for an implementation plan and schedule, “The

biggest recommendation I have is once you start and you pick the tool you want to use, whether

it’s PODs or whatever it is, to get your stuff, get your folks on a schedule and stick to it …

because we’ve moved the process along and then stalled out, and people got disengaged. And, so

my biggest recommendation would be just once you started, bringing it all the way through to

the end, keep yourself on a schedule. Don’t drop it” (DST23). Scientists/developers also noted

that identifying ways to use and implement the results of a DST effort is critical. As one

scientist/ developer said, “Listening to the forest or region, those stakeholders ... ‘Okay, so what

does your program of work look like? How do you think this information can be useful? How are

you identifying and prioritizing your fuel treatments now?’ Working with them together in, sort

of, a workshop setting to identify ways in which the QWRA results could be integrated into their

program of work” (DST9).

Respondents also noted that it is important to update data and DST outputs on some sort of pre-

determined schedule in order to keep the information up-to-date and relevant. One manager

explained, “It shouldn’t be a static thing. Conditions on the ground change. There could be

something out there that wasn’t captured in your risk assessment when you made it. They’re all

not static tools. They need to be updated to have relevance. If you had a big fire season and fire

burns across a number of your PODs, you need to look at updating some things because maybe

those outlines weren't in a good spot to start with, but they probably are now” (DST24). Another

manager suggested that starting with annual updates to a DST like PODs would be appropriate

and could be scaled back as needed. “I think a good starting point would be annually, and we

may find that maybe it’s biannually or every three years or something. But you know, last year,

34

as an example, we had 55,000 acres of what we used to call wildland fire use. So, that 55,000

acres represents a big change on many of our landscapes” (DST20).

Determine the question that a DST is meant to inform and develop and/or apply it accordingly

Across respondent groups from science, management, and boundary spanning organizations,

interviewees emphasized the importance of first determining what is the question that a DST is

meant to inform and then developing its technical capabilities so that it is tailored to produce

outputs to inform the key question. As one boundary spanner noted, “Early on in a process,

define what a management question actually is that needs decision support … For people who

are developing decision support tools, or applying decision support tools, that’s a necessary skill

beyond just the technical capacity is really helping coach people through a process of helping

them define what are the questions they’re trying to answer, what information do they need and

then how are they going to use that information.” (DST11).

Other boundary spanners articulated a similar process of reverse engineering the development of

a DST by first determining what questions need decision support. As another boundary spanner

explained, “In my work, I start at the other end. I figure, what’s the question that we need to

answer, and what’s the best way to answer that question? I think a lot of these tools are working

backwards. They develop the tool and then ask how can we use this? I think the tools that I work

on don’t take that approach, or at least try not to” (DST14). Similarly, another boundary

spanner explained the importance of understanding a setting before determining what DST to

apply. “We go in and listen and learn and pay attention. And whether it’s like, do a formal needs

assessment process so that we can identify which tools to bring to bear, or just through

conversations over coffee and in meeting rooms and things like that, to understand the

complexity, the social-ecological complexity of a place, and then develop the tools that can help

solve that” (DST15).

However, because there are numerous DSTs available that could be brought to bear on a specific

question, it can be challenging to select from the available tools, especially if developing a more

tailored approach is not feasible. One scientist/developer emphasized that it is, therefore,

incumbent on the scientists/developers to explain what kinds of questions a given DST can

answer. “I certainly have tried, whenever I present on the analyses that I'm working on, to

explain the kinds of things that I just talked about that, hey, look, what I’m giving you provides

one type of information. But doesn’t provide all the answers, and this is what you can do with the

information I’m giving you and these are the types of questions that this information can answer.

But if you want answers to these questions, which are different, you maybe need to turn to

different tools” (DST16). One manager also explained the importance of understanding the

question before selecting from the whole suite of DSTs that are available. “We have this whole

suite of tools available, and a lot of the times, you don’t need a lot of this information. That’s

where you become a choke point in the pipeline … If you can get that question phrased and

written down and you all agree on it, it’s actually a lot easier to pick through all of these tools to

figure out what the right one might be for the situation” (DST24).

35

Another boundary spanner highlighted that it is also important not to create DSTs that try to

accomplish every objective or answer every question. DSTs that try to do everything are often

problematic. “There’s a push to have these tools do everything. And they become multi-tools, the

Leatherman, it can technically do everything, but it’s not really good at anything, right? Once

you over-generalize, it’s hard to be good at all those different things. So, I would prefer multiple

models and multiple decision support tools, which each address one or two problems, instead of

one gigantic tool that is supposedly designed to address all things” (DST14).

In short, there are tradeoffs that need to be considered between applying established, formal

DSTs and developing smaller-scale, context-specific DSTs that are less formalized but use a

common language and fire science framework. As one boundary spanner explained, “I’m not

sure if one model is better or not … But it’s certainly much lower costs to say take this scripting

approach to decision support where you’re really just trying to automate a process for someone

without necessarily making it highly functional complicated software for someone” (DST11).

One scientist/developer explained that in order for a DST to be used to support decisions, a few

key factors need to coalesce, including 1) a standard, guideline, or requirement to conduct a risk

assessment or prioritization process; 2) a biological or physical model that serves as a underlying

framework; 3) software or some other platform on which to run the model; and 4) people who

are trained to run and interpret the model. “All those things have to line up before you can build

a decision support tool. It’s going to get used” (DST4).

DSTs should be validated and tested with end users

Another recommendation from interviewees was that DSTs need to be ground-truthed in order to

validate and monitor the outputs that they provide. The importance of model validation for

decision support has also been identified in other research (Rapp et al. 2020). One boundary

spanner noted, “After you’ve drawn stuff on a map, you need to get out in the woods” (DST10).

The importance of ground-truthing was also identified by managers. As one manager explained,

“In all of our fuels projects, we always went out and ground-truthed everything, too, and I think

any model that you use, you need to ground-truth anyways, because you never know how good

the data is, what’s underlying it” (DST25).

Respondents also emphasized that validating DSTs and their outputs is an essential way to

improve them over time. One manager said, “Verify it on the ground with the folks that are

closest to it, because all the models in the world, applying all of the pieces that we might know in

a geospatial lab sense out on the ground always is improved by ground-truthing” (DST7). One

boundary spanner also emphasized DST verification to improve outputs. “Having some feedback

loop so that those people who end up using information can then tell you how to improve it and

make it better for them. It’s really missing in a lot of cases” (DST11). However, as another

boundary spanner noted, in order to effectively validate DSTs, there also needs to be a thorough

system of documentation of their use and outcomes. “What we need to do is we need to validate

them better than they have been. And, so in order to validate them, we need to use them

extensively and then document not only what they said might happen, but what actually

36

happened. And we need to do a good job of doing that so that we can make some assessments on

do they work. And if they don’t, why? Can we fix that? If so, let’s fix it and try it again … To use

them more effectively, we need to have a rigorous program of documentation” (DST2).

In addition to DST validation, respondents also noted that the caveats of different DSTs need to

be clearly explained to end users as part of testing and integration. One scientist/developer

emphasized, “I think we have a real obligation, those of us that work in this, to try to explain this

kind of stuff as clearly as possible” (DST16). Another manager noted that it is important to also

communicate how exactly DST outputs are intended to be used. “I think the best thing we could

do is be really clear on the front end about what this is and what it is not … I have to remind

folks a lot that this isn’t a decision-making tool. This doesn’t replace our executives, our line

officers” (DST20).

Interviewees also recommended that DSTs be tested with end users in order to ensure that they

are as user-friendly as possible. One manager said, “The thing that I would say is just make sure

that things are tested with folks that are out there doing it … make sure that it’s operationally

sound. If it doesn’t make sense on the ground, then people are going to ignore it or be resistant

to it or whatever. Honestly, we got a bunch of iPads for folks, which is all fine and dandy, but if

they’re glitchy or if the user interface isn’t good, people are … going to use their steno pad and

their topo map” (DST17).

Another manager emphasized that it is important that the people who will be using DSTs are also

the people that help to test them and provide feedback. “The people that are testing it are people

that are pretty good with these tools. They really need to get some people that are the ones using

them on the ground that would use them, testing these and making them user-friendly” (DST25).

Furthermore, many end users want to be able to run DSTs themselves so that they have some

control over the process and feel ownership in the information being provided. As one boundary

spanner explained, “A lot of people want to be able to actually run this tool. They don’t want to

just know that it’s on some scientist's computer, and it's coded in this language that they can't

read, like R or Python. They actually want to be able to click on the buttons, change the

numbers, rerun the analysis, and learn and understand how the tool works” (DST11).

These recommendations emphasize the importance of continuing to improve tools based on user

feedback. For example, one scientist/developer explained how a user advisory group and a user

forum were developed to help improve the use of a new DST. “The user advisory group’s a new

concept, so we’re still figuring out the best way to utilize their time, their skills, and their input

with it, but I think it’s much better than us just sitting in a cave and building a tool that we think

is most important” (DST5). The same scientist/developer also explained how a user forum can

help connect end users with DST developers to solicit feedback and improve the DST, “… a user

forum where anyone can post questions or suggestions, and a lot of times, we use what they have

posted to enhance, update, or change something” (DST5).

37

DST end users need both education and training in the concepts and tools

Another key component of successful DST development, integration, and use is both education

and training. Education refers to developing a comprehensive understanding of risk and risk-

informed decision making. Training refers to the necessary steps to develop and apply DST

outputs in programs of work and decision-making processes. As one boundary spanner said,

“Our line officers and fire managers need to be educated in risk, not trained but educated.

Training tells you how to go through step A to step B, and C or D, or whatever. Education gives

you the ability to work on a problem and apply concepts to the problem. Right now, most of our

folks are not well educated in risk or educated at all in risk … The perfect example is it’s very

difficult to get the agency administrator to look at an individual fire and the risk associated with

it, as well as the risks associated down the road that will occur based on the actions they take on

that fire” (DST2). Another scientist/developer put it this way, “I think we need education at

multiple levels and training at multiple levels. But I think that education in decision making is a

critical component” (DST3).

Education is also a way to change existing paradigms and cultures that exist in fire management

that were identified as barriers by some respondents. One manager explained, “I don’t know

what that education campaign would look like, but I think just as you go to more places that are

doing different things with fire, eventually you change the culture of the people that are

eventually going to be the fire staff and the directors and all of that sort of thing” (DST24).

Another managed shared a similar sentiment, “The other piece that we are leaning into is with

agency administrators, decision makers, line officers, to require some training and education, so

we get to some common language and understanding about their role in decision making and

what quality decision making is, what risk-informed decision-making is” (DST17).

Changing paradigms and cultures takes time, so some interviewees recommended that continuing

education programs and revised or new qualifications could facilitate both education and

training. One boundary spanner said, “I think in the end it would get everybody up to the same

level, that if we knew every five years there was this mandatory continuing education piece, so to

get everybody up to speed on what’s the latest and greatest science and tools and information

that we’re all sharing during wildfires” (DST8). Another manager agreed that qualifications

could be modified to facilitate an understanding of the underlying concepts that shape DSTs.

“We have those IFPM [Interagency Fire Program Management] qualifications and I think if

they went back and revised them to include more of that fire ecology, fire behavior, fire analysis

type of things, they could start there to where you have to have some certain basic classes.

Whether you are an engine captain, or you are a fuels specialist, I think all of our fire people

need more of that” (DST25).

With respect to training, one approach called “train-the-trainer” was mentioned by several

respondents as a useful strategy to deliver new concepts and DSTs to the numerous and diverse

Forest Service units around the country. Trian-the-trainer is a model whereby a local

representative is training to use a DST and then returns to a local unit to assist in training his or

her colleagues. One scientist/developer explained it this way, “Train-the-trainer is a model that

38

we’re really trying to work on, because … we’re so busy trying to develop new content, we don’t

necessarily have the time to get out to every person who wants training, but we are slowly …

trying to find people that are self-motivated and want to learn it and willing to go teach beyond

just their small groups so we’re slowly building that momentum” (DST5).

Respondents also noted that user workshops are critical in training people to use new DSTs. One

of the DSTs referred to by many respondents was PODs, which are often developed in a

workshop setting. In addition to using workshops to develop DSTs, respondents also noted that

there need to be workshops to train people on how to use the DST outputs. One manager noted,

“We’re working on … more of a user type of workshop, how do you use this data in ArcGIS, how

can you put it in a NEPA analysis, how could you use it in a burn plan, and work with the users

of this more” (DST25). One scientist/developer described the need for workshops that focus on

the application of DSTs. “After the results have been presented, sit down with … the key users of

it, sit down with those folks and provide a little bit of additional interpretation or additional

training on how you interpret QWRA results” (DST9).

Respondents also provided additional recommendations about how to structure an effective DST

training. For example, one scientist/developer noted that many people prefer in-person training.

“We have user surveys ... One of the questions was, how would you most like to get training?

And the most common answer is: in-person training” (DST5). Additionally, one scientist/

developer explained that working through exercises for how to use DST outputs has also been

helpful. “I think exercises where they’re actually working through an exercise that shows what

was predicted, what actually occurred … If you have two or three exercises that show how you

work through that, that seems to, in my experience anyway, resonate with folks better and give

them better appreciation for what it is saying” (DST2). Lastly, one scientist/developer noted that

having a user manual can also provide long-term support for local technical experts.

Policies, guidance, and authorities must match leadership intent for integrating and using DSTs

Interviewees also recommended several ways in which policies, guidance, and authorities could

be changed or improved to facilitate a better understand of leadership intent for using DSTs in

wildfire risk assessment and treatment prioritization. One boundary spanner said, “I think we

need national guidance saying, just like we did with LANDFIRE, all right, we need these risk

assessments, let’s start, let’s lead the efforts, start turning the crank nationally, and we’re going

to create these risk assessments for, at least the western states, every few, four years” (DST1).

Although there are policies that articulate the need for doing risk assessments, as one scientist/

developer noted, they are not explicit. “The policies, I think, are there. They’re just not as

explicit as, you should do this and use these tools, which at any point in time is useful in setting

an agenda and a direction for an agency, organization” (DST27).

However, developing guidance for DSTs is challenging, in part, because the science behind

DSTs is constantly changing. As one scientist/developer noted, “Science is constantly changing.

Take PODs, for example. It’s starting to take off, and there’s a lot of interest and new

development, how this is being developed and applied. Is there official guidance from the regions

39

or the Washington Office that are saying to use PODs? No. They’ve had to read between the

lines about what the Washington Office wants, more strategic planning, and identify places to

use fire and risk management. And, so there’s a tool that’s crafted towards that. Now, some

forests are waiting for somebody to tell them, thou shalt do this. Other forests are willing to just

kind of dive in” (DST14).

One of the ways that more clarity could be gained about the need for doing risk assessments is

through a clear articulation of leadership intent. One manager argued, “There has to be some sort

of directive or emphasis, literally stated from top leadership, that this is something that they

want to see completed and give people a reasonable amount of time, you know, five years”

(DST18). A clear articulation of leadership intent is especially important in an organization like

the Forest Service that has dispersed units from the Washington Office to local district offices, so

there needs to be clear and consistent communication across the different levels and units of the

Forest Service to avoid confusion about what is intended.

However, as another manager argued, it is also important that leadership intent consider

perspectives from the local level. “I think it’s more of having leadership know and communicate

their desires to have their people involved in it, knowing the benefits of having local experts

involved from the very beginning and throughout the process, knowing the value of that on the

back end would facilitate the use of these tools and making the outputs more usable and making

people more likely to use them in the end” (DST22). One scientist/developer put it this way,

“Somebody at the national level should set an agenda that is in-line with what the public wants

and aligns their staff to that need” (DST27). As one boundary spanner explained, this requires

communication from local levels up the chain of leadership to help provide that local perspective

across the units of the Forest Service. “It’s really important that these higher-up decision makers

know that there’s a lot of nuance to planning and prioritization at a local level. And there’s also

this big, I would say, trust and understanding that comes from applying these tools within a

localized context. And, so I think making sure that people know that it’s unlikely that some single

national analysis that’s done is going to be somehow downscaled and well-received and easily

applied at a local scale is an important message to come out with or to communicate” (DST11).

To that end, it is important to strike a balance between articulating leadership intent from the top,

while also making sure to communicate local and public perspectives up the chain so that they

can inform top-down directives.

Respondents also recommended that leadership intent and communication about the need for

using DSTs should include an emphasis on why it is important. There needs to be a compelling

“why” in order to garner support across levels of the agency. One of the ways in which that

compelling “why” can be communicated is by supporting and empowering thought leaders and

change agents within the agency. As one manager explained in reference to attending PODs

workshops, “A lot of the people that went to these workshops, they’re not your 25-year seasoned

fire managers who came. And they grew up in a very different world. What you need is your

younger people to come back and inspire the old people. But the younger people coming back

are not in any position of any authority or influence. And, so, you’ve got a bunch of young

40

people coming back. And I say young, but I mean change agents coming back. And then, they

basically just get shoved down because there isn’t any parallel policy pressure placed on it. And,

so, it loses traction” (DST18). Another scientist/developer noted that there is also a need from

leadership to support a reasonable level of risk-taking. “From an outside perspective, it feels like

they just aren’t willing to step up and accept decisions, the risks that come with that uncertainty

in the future and the outcomes. And then support each other when things go awry” (DST27).

In addition to articulating intent to use DSTs, interviewees also noted that it is important to

develop incentives and accountability for doing in order to facilitate the process. As one

scientist/developer explained, “We don’t have incentives and accountability. And I’m talking

about accountability, not when somebody screws up, they’re not getting punished. I’m talking

about just basic, are we measuring and reporting what’s happening? And the incentives are very

driven at a local level. There’s no clear direction on how we’re incentivizing good fire

management. We don’t even really know how to define good fire management. So how do you

incentivize it when you haven’t defined it? I think that’s a critical component” (DST3).

In order to develop incentives and accountability for using DSTs in decision-making, there also

has to be some degree of evaluation and documentation of decisions made using DSTs. As one

boundary spanner argued, “I think line officers should be evaluated. Part of the performance

rating should be on how they manage risk on the landscape. We can quantify risk on the

landscape, right? And, so we should quantify risk on every unit, on the landscape every year.

And a line officer should be evaluated against that … It’s probably the most important thing that

they are charged with managing” (DST2). As another scientist/developer recommended, “And,

so this process of risk management requires that we get away from outcome-based metrics and

get to a decision to really be able to evaluate the quality of our decisions and that requires us to

plan and document” (DST3).

In short, many respondents argued for a more structured approach to decision-making, which has

also been articulated by other researchers (Calkin et al. 2010, Thompson et al. 2019). As one

manager explained, “So I have a dream. We need an integrated approach and commitment to the

structured, risk-informed decision making. And this is where you can start throwing around all

the rest of the buzzwords, evidence-based, data-driven, all the rest of it … And one of the most

next steps is some assessment of the analytic capability in the agency and how we want to

leverage it … We’ve got to bring these pieces together and at the core of it, as a body of

specialty, around analytic capacity. How do we assess it and then determine what we might

need? And if we use a structured decision-making approach, I actually might come up with a

fairly reasonable set of next steps” (DST7).

41

Conclusion

The purpose of this project was to analyze the use and adoption of wildfire risk assessment and

fuels treatment planning and prioritization decision support tools (DSTs) by federal land

managers, especially in the USDA Forest Service. We sought to demystify the topic of spatial

analysis for pre-fire planning, specifically with respect to assessing wildfire risk and determining

areas for fuels treatment prioritization to facilitate effective development and use of DSTs for

pre-fire planning. Although there were many barriers identified to the effective development,

integration, and use of DSTs in pre-fire planning, interview respondents had numerous

constructive recommendations for improving this process, which we categorized into the

following themes: capacity, communication, implementation, question identification, testing,

education and training, and policy, guidance, and authorities. We hope these recommendations

can help shape the perspectives of science, management, and decision-making audiences for how

to improve the use of DSTs for wildfire risk assessment and treatment prioritization in order to

effectively meet the goals of national policies and frameworks.

42

References

Ager, A.A., R.M. Houtman, R. Seli, M.A. Day, J. Bailey. 2017. Integrating Large Wildfire

Simulation and Forest Growth Modeling for Restoration Planning. In. Keyser, Chad E.;

Keyser, Tara L., eds. 2017. Proceedings of the 2017 Forest Vegetation Simulator (FVS)

e-Conference. e-Gen. Tech. Rep. SRS-224. Asheville, NC: U.S. Department of

Agriculture Forest Service, Southern Research Station. 129-137pp.

Caggiano, M.D. 2019. Collaboratively Engaging Stakeholders to Develop Potential Operational

Delineations. Colorado Forest Restoration Institute, CFRI-1908. 10p.

Caggiano, M.D. 2020. Potential Operational Delineations and Northern New Mexico’s 2019 Fire

Season. Colorado Forest Restoration Institute, CFRI-2002. 8p.

Calkin, D.E.; Ager, A.A.; Gilbertson-Day, J., eds. 2010. Wildfire risk and hazard: procedures for

the first approximation. Gen. Tech. Rep. RMRS-GTR-235. Fort Collins, CO: U.S.

Department of Agriculture, Forest Service, Rocky Mountain Research Station. 62 p.

Colavito, M., T. Wasserman, and D. Vosick. 2018. All-lands planning, coordination, and wildfire

risk reduction project report. Ecological Restoration Institute, Northern Arizona

University, 54p. Available online:

https://cdm17192.contentdm.oclc.org/digital/collection/p17192coll1/id/913/rec/2/

Esch, B. and A.E.M. Waltz. 2019. Assessing metrics of landscape restoration success in

Collaborative Forest Landscape Restoration Projects. ERI White Paper – Issues in Forest

Restoration. Ecological Restoration Institute, Northern Arizona University, 12p.

Available online:

https://cdm17192.contentdm.oclc.org/digital/collection/p17192coll1/id/1015/rec/36

Greiner, M., C. Kooistra. C. Schultz. 2020. Pre-Season Planning for Wildland Fire Response: An

Assessment of the US Forest Service’s Potential Operational Delineations. Public Lands

Policy Group, Colorado State University, 16p.

O’Connor, C.D. and D.E. Calkin. 2019. Engaging the Fire Before it Starts: A Case Study from

the 2017 Pinal Fire (Arizona). Wildfire 28 (1): 14-18.

Rapp, C., E. Rabung, R. Wilson, and E. Toman. 2020. Wildfire Decision Support Tools: An

Exploratory Study of Use in the United States. International Journal of Wildland Fire,

Online early: https://doi.org/10.1071/WF19131.

Schultz, C., C. Kooistra, L. Miller, and M. Ferguson. 2020. Findings from a Third-Party

Assessment of the U.S. Forest Service’s Risk Management Assistance Teams. Public Lands

Policy Group, Colorado State University, 40p.

Scott, J., M. Thompson, and D. Calkin. 2013. A wildfire risk assessment framework for land and

resource management. Gen. Tech. Rep. RMRS-GTR-315. Fort Collins, CO: U.S.

Department of Agriculture, Forest Service. Rocky Mountain Research Station. 83p.

Stratton, R. 2020. The Path to Strategic Wildland Fire Management Planning. Wildfire 29 (1):

24–31.

43

Thompson, M.P., Y. Wei, D.E. Calkin, C.D. O’Connor, C.J. Dunn, N.M. Anderson, and J.S.

Hogland. 2019. Risk Management and Analytics in Wildfire Response. Current Forestry

Reports 5 (4): 226–39. https://doi.org/10.1007/s40725-019-00101-7.

Thompson, M.P., T. Zimmerman, D. Mindar, and M. Taber. 2016. Risk Terminology Primer:

Basic Principles and a Glossary for the Wildland Fire Management Community. General

Technical Report 349. USDA Forest Service, Rocky Mountain Research Station. 19p.

USDA Forest Service. 2020. Forest Service Manual 5100 – Wildland Fire Management, Chapter

5140 – Hazardous Fuels Management and Prescribed Fire. USDA Forest Service National

Headquarters (WO), Washington D.C. 17p.

USDA Forest Service. 2018. Toward Shared Stewardship Across Landscapes: An Outcome-

Based Investment Strategy. USDA Forest Service National Headquarters (WO),

Washington D.C. 28p. Available online: https://www.fs.usda.gov/sites/default/files/toward-

shared-stewardship.pdf

Wildland Fire Leadership Council. April 2014. The National Strategy: The Final Phase in the

Development of National Cohesive Wildland Fire Management Strategy. 101p. Available

online:

https://www.forestsandrangelands.gov/documents/strategy/strategy/CSPhaseIIINationalStra

tegyApr2014.pdf

44

Study Contact: Melanie Colavito (melanie.colavito@nau.edu)

Barriers and Facilitators in the Use of Wildfire Risk Assessment and Treatment Prioritization

Decision Support Technology

Open-Ended Interview Questions*

*Additional, relevant questions may be posed to research participants depending on the nature of their

answers to the questions below in order to get clarity or gather additional information relevant to the

study objectives. The order of the questions may vary for each respondent, and respondents will not

necessarily be asked all of the questions below. Some respondents will be asked questions from multiple

sections depending on the nature of their job role with respect to decision support tools and models.

The questions in the introduction will help determine which questions are asked of a given respondent.

Primary Investigator: Melanie Colavito, Ecological Restoration Institute / melanie.colavito@nau.edu

Verbal Consent

1. Are you over 18 years of age, do you consent to participate in this research, and do you agree to be

audio recorded?

Introduction

2. Please tell me about your job, including what your current role entails, how long you have been in

your current role, and any other relevant background.

3. How would you characterize your job role (e.g., scientist, land manager, policymaker, etc.)?

4. Please describe your involvement in wildfire risk assessment and treatment planning.

5. Please describe your involvement in the use of decision support tools and models for wildfire risk

assessment and treatment planning.

6. How would you describe “wildfire risk assessment”?

7. How would you describe “treatment planning and prioritization”?

8. How would you describe “decision support tool or model”?

Development of Decision Support Tools

9. Are you involved in the development of decision support tools or models for wildfire risk assessment

or treatment planning or prioritization? If yes, please explain.

Potential Follow-Up Questions

a. Which tools or models have you helped develop?

b. Who else is involved in the development of these tools or models?

c. What are the objectives of these tools or models?

d. At what geographic scale do these tools or models provide information?

e. What are the ideal uses of these tools or models?

f. Who are the ideal end users of these tools or models?

g. What are the needs of the end users/ audience of these tools or models?

h. Do you see a need for end user participation in the development of these tools or models?

Appendix. Open-Ended Interview Questions

45

i. Can you provide any specific examples of the tools or models being used that you think

were especially successful or unsuccessful?

j. Overall, what works well in the development of these tools or models?

k. Overall, what are barriers to the development of these tools or models?

l. Have you received feedback from end users about the tools or models with which you have

worked? If yes, please explain.

i. Overall, what works well in the use of these tools or models?

ii. Overall, what are barriers to the use of these tools or models?

m. Are there any improvements for the use of these tools or models that you would

recommend?

n. Are there any improvements for the development of these tools or models that you would

recommend?

o. Do you see any gaps in capacity that would facilitate the development and use of these tools

or models?

p. Do you see any gaps in development or use of these tools or models that could be filled by

new or revised guidance, policy, or authorities? If yes, please explain.

q. What do you see as the future of decision support tools for wildfire risk assessment and

treatment prioritization? Are new tools needed?

Use of Decision Support Tools

10. What do you use to inform your decision making about wildfire risk or treatment planning and

prioritization?

a. What are the biggest barriers you face in decision making about wildfire risk assessment or

treatment planning and prioritization?

1. What do you need to facilitate your decision-making process?

11. Are you involved in the use of decision support tools or models for wildfire risk assessment or

treatment planning and prioritization? If yes, please explain. If no, why not?

a. Are there tools or models with which you are familiar but have not used?

Potential Follow-Up Questions

a. Which decision support tools or models have you used to enhance your wildfire risk

assessment or treatment planning and prioritization work?

b. What are your objectives in using these tools or models?

c. How have you used these tools or models?

d. Who else was involved in using these tools or models?

e. What is the ideal use of these tools or models?

f. Have you interacted with the developers of these tools or models? If yes, please explain.

ii. What works well in the development of these tools or models?

iii. What are barriers in the development of these tools or models?

g. Do you see a need for end user participation in the development of these tools or models? If

yes, what is needed? If no, why not?

h. Can you provide any specific examples of the tools or models being used that you think

were especially successful or unsuccessful?

i. Overall, what works well in the use of these tools or models?

46

j. Overall, what are barriers to the use of these tools or models?

k. Are there any improvements for the development of these tools or models that you would

recommend?

l. Are there any improvements for the use of these tools or models that you would

recommend?

m. Do you see any gaps in capacity that would facilitate the development and use of these tools

or models?

n. Do you see any gaps in development or use of these tools or models that could be filled by

new or revised guidance, policy, or authorities? If yes, please explain.

o. What do you see as the future of decision support tools for wildfire risk assessment and

treatment prioritization? Are new tools needed?

Wrap Up

12. Do you have any other thoughts about decision-support tools or models for wildfire risk

management and treatment prioritization that you would like to share?

13. Is there anyone else that you would recommend I interview for this study?

14. Can I contact you again in the future regarding this study?

15. What kinds of outputs would be most helpful to you from a study like this?

16. Would you like to see the outputs of this study?

17. Do you have any questions for me?

47