Communicating Weather Forecast

Uncertainty Information

WAS* ISWAS* ISweather & society * integrated studies

www.sip.ucar.edu/wasis/

CULTURE CHANGE

Andrea Bleistein and Julie DemuthSummer WAS*IS, July 18, 2007

The forecast high temperature for Boulder tomorrow is 95ºF.

a) 95ºFb) 94-96ºFc) 93-97ºFd) 90-

100ºFe) 85-

105ºF

What do you think the actual high temp will be?

All the choices listed below are the same as a probability of

precipitation of 20%.

a) Chance of precipitation tomorrow is 20%.

b) There is a 1 in 5 chance of precipitation tomorrow.

c) The odds are 1 to 4 that it will rain tomorrow.

d) There is a slight chance of rain tomorrow.

Do you like this information given as…

Definitions

• Definition of uncertainty– NRC– Wikipedia– NOAA group definition

Outline• Motivation• NWS’s related efforts• NCAR’s related efforts• Discussion session• Other resources

Motivation• Atmosphere is nonlinear, chaotic, and

complex

• First public forecasts in the modern weather forecasting era were called “probabilities”

• But forecasting generally evolved into more deterministic products

Forecast uncertainty is inevitable!

*

* Single future state of a system or single-value forecast

*

* Single future state of a system or single-value forecast

Motivation (cont)• Yet, most users understand that

forecasts are imperfect• By communicating uncertainty

information, we can– avoid misrepresenting the capabilities of

weather prediction science– better convey what meteorologists know– help users make more informed decisions

and avoid problems

An Example…

Community Interest and Action• American Meteorological Society (2002)

statement – endorsing probability forecasts and

recommending an increase in use

• National Research Council (2006) report commissioned by the National Weather Service

• National Research Council (2003) report on communicating uncertainties in weather and climate forecasts

NWS’s Related Efforts

NWS Forecast Uncertainty Steering Team

• How to corporately address need/opportunity to improve generation and communication of forecast uncertainty products and services

• How to address and respond to 2006 NRC Report, “Completing the Forecast. Characterizing and Communicating

Uncertainty for Better Decisions Using Weather and Climate Forecasts”

• In Jan 07, informal group began looking at forecast uncertainty from corporate perspective

• Purpose not to hinder or duplicate ongoing efforts, but rather to corporately plan for success by ensuring necessary components of an “end-to-end” forecast uncertainty information system are accounted for

NWS Forecast Uncertainty Steering Team

• Vision– NWS provides forecast uncertainty

products, services, and information meeting customer, partner, and forecaster needs

• Mission– Advise and coordinate NWS activities

related to development, implementation, and evolution of forecast uncertainty products, services and information

Internally

• Documenting ongoing NOAA forecast uncertainty activities

• Collecting NWS forecaster needs and requirements (gaps) for forecast uncertainty tools

• Looking at any “low-hanging fruit” changes/additions to products/services

• Working with NOAA Programs and Goals to define forecast uncertainty needs for FY10-14 planning, programming, and budget cycle.

Externally

• Working with Chair of 2006 NRC Study and other members of the NRC Panel to identify opportunities and feasibility for engaging the broader Weather-Water-Climate Enterprise in a Forecast Uncertainty partnership(s)

• Led to the AMS Commission on Weather and Climate Enterprise hosting: ACUF (Ad Hoc Committee on Uncertainty in Forecasts)

– Possible conjunction with the AMS Annual Partnership Topic

ACUF• Announcement on June 25, 2007 through AMS

Commission on the Weather and Climate Enterprise (CWCE) residing under the Board on Enterprise Communication (BEC)– Charged to engage the weather and climate enterprise in

identifying a vision of forecast uncertainty characterization and communication as motivated by the 2006 NRC report

– Identify future paths for more thoroughly addressing forecast uncertainty products, services, and information needs of the Nation.

– Propose roles and responsibilities of enterprise partners in developing, generating, and providing uncertainty information to the user community.

– Reps from government, the private sector, academia, and the user community.

NOAA-wide Uncertainty Interests• NOAA Stakeholder Forum – May 2007

– Hazard Mitigation and Resilience Strategies recommendation: “Develop advanced decision making and risk analysis tools and

procedures including available and to be available information on the inevitable uncertainties in data analysis and forecasts.”

– Ecosystems: “Risk/Uncertainty – Ecosystem links are inherently uncertain, need

to know the risks and costs of uncertainty.”• NOAA FY07-11 Research Plan

– Addresses development of more useful products that convey uncertainty in all environmental information

• Weather and Water Goal:– Generation and communication of forecast uncertainty

information is a major theme/goal for FY10-14 planning, programming, and budgeting cycle

NCAR’s Related Efforts

Objective• To effectively communicate uncertainty-

explicit forecasts, social science research is needed to better understand people’s perceptions, understanding, use, and preferences

• To support provision of this information through survey research– To assess the U.S. public’s attitudes toward

forecast and forecast uncertainty information

Uncertainty Research Questions

• How much confidence do people have in different types of weather forecasts?

• Do people infer uncertainty into deterministic forecasts and, if so, how much?

• How do people interpret probability of precipitation forecasts?

• Do people prefer to receive deterministic or uncertainty-explicit forecast information?

• In what formats do people prefer to receive forecast uncertainty information?

Do people infer uncertainty into

deterministic forecasts and, if so, how much?

Suppose the forecast high temperature for tomorrow for your

area is 75F.What do you think the actual high temperature will be?

0%

10%

20%

30%

40%

50%

75 74-76 73-77 70-80 65-85 OtherN=1465

Do people prefer to receive deterministic or

uncertainty-explicit forecast information?

Suppose you are watching the local evening news …

• The Channel A weather forecaster says the high temperature will be 76F tomorrow

• The Channel B weather forecaster says the high temperature will be between 74F and 78F tomorrow.

0% 10% 20% 30% 40% 50%

Channel A

Channel B

Both

Neither

I don't know

N=1465

In what formats do people prefer to receive forecast uncertainty information?

All the choices below are the same as a probability of precipitation of

20%.Do you like the information given this way?

• Chance of precipitation is 20%• There is a 1 in 5 chance of precipitation• The odds are 1 to 4 that it will rain• There is a slight chance of rain tomorrow

Percent Frequency Odds Text

Asked this question 3 ways -- using 20%, 50%, and 80% probabilities of

precipitation with corresponding descriptions

Asked this question 3 ways -- using 20%, 50%, and 80% probabilities of

precipitation with corresponding descriptions

Overall distribution (% yes)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Percent Frequency Odds Text

20%

N=489,

50%

489,

80%

487

Future Work – Building on our Results

• What are the next research questions to pursue?

• How do we translate this understanding into improving how weather forecast information, including uncertainty information, is communicated?

Discussion Questions• How do we strike a balance between “educating”

the users and understanding users’ needs and perceptions?

– E.g., hurricane cone of uncertainty

• How do we provide good uncertainty-explicit information given the proliferation of weather information, and how do we ensure the information is consistent from public and private sector sources?

• How much of the role of social science does/should NOAA take on regarding the provision of uncertainty-explicit forecast information?

Resources• AMS, 2002: Enhancing weather information with probability

forecasts. • Broad et al., 2007: Misinterpretations of the “cone of uncertainty”

in Florida during the 2004 hurricane season.• Gigerenzer et al., 2005: How does the public understand

probabilistic weather forecasts?• Murphy, 1998: The early history of probability forecasts: Some

extensions and clarifications. • Murphy et al., 1980: Misinterpretations of precipitation probability

forecasts.• NRC, 2003: Communicating Uncertainties in Weather and Climate

Information: A Workshop Summary. • NRC, 2006: Completing the Forecast: Characterizing and

Communicating Uncertainty for Better Decisions Using Weather and Climate Forecasts.

• Roulston et al., 2006: A laboratory study of the benefits of including uncertainty information in weather forecasts.

• Many, many more!