Extracting Meaningful Extracting Meaningful Data: Distinguishing Data: Distinguishing Signal from Noise in Signal from Noise in

Climate ChangeClimate Change

Q. Steven HuQ. Steven Hu

School of Natural ResourcesSchool of Natural Resources

University of Nebraska-LincolnUniversity of Nebraska-Lincoln

In general, noise is the part of information that we In general, noise is the part of information that we want to tease out and signal is the part that we want want to tease out and signal is the part that we want

to keep. to keep. Noise is “stubborn” and always present and interferes Noise is “stubborn” and always present and interferes with signal, forcing us to identify it and find ways to with signal, forcing us to identify it and find ways to

separate it in the data.separate it in the data.We don’t want to throw away everything (data) we We don’t want to throw away everything (data) we

have collected because of presence of noise, but we have collected because of presence of noise, but we cannot keep everything either. We want to keep the cannot keep everything either. We want to keep the signal (the baby) after clearing the noise (the bath signal (the baby) after clearing the noise (the bath

water)!water)!

Except for some “absolute noise,” noise and Except for some “absolute noise,” noise and signal are relative and they are determined by signal are relative and they are determined by the interest of studies.the interest of studies.

Example 1Example 1: The atmosphere contains variations at : The atmosphere contains variations at rather wide ranges of frequencies and spatial scales. rather wide ranges of frequencies and spatial scales. All those variations are signals to the climate. Yet, if All those variations are signals to the climate. Yet, if we are interested in studying a particular frequency we are interested in studying a particular frequency variation, e.g., interannual variation – changes of variation, e.g., interannual variation – changes of rainfall from one summer to the next, or variation at a rainfall from one summer to the next, or variation at a specific spatial scale, e.g., the synoptic scale – in the specific spatial scale, e.g., the synoptic scale – in the order of 1000km, all the other signals become order of 1000km, all the other signals become “noises.” We must identify and remove them before “noises.” We must identify and remove them before we can examine variations at the interested we can examine variations at the interested frequency and scale and understand their behavior frequency and scale and understand their behavior and change.and change.

Example 2Example 2: In phonological research, an : In phonological research, an overwhelming number of studies have overwhelming number of studies have examined long-term data on phonological examined long-term data on phonological patterns and life-events of animals and plants, patterns and life-events of animals and plants, and found earlier migration to breeding sites, and found earlier migration to breeding sites, birds laying eggs on earlier dates, and plants birds laying eggs on earlier dates, and plants flowering earlier. These rather diverse yet flowering earlier. These rather diverse yet consistent changes of phenology are, as consistent changes of phenology are, as believed, responses to a warming climate. believed, responses to a warming climate.

This phenological signal of change is an independent source of This phenological signal of change is an independent source of information of climate change. To a great extent, this signal is information of climate change. To a great extent, this signal is

free of errors and noises resulting from gathering and free of errors and noises resulting from gathering and manipulating instrumentation records, thus providing a manipulating instrumentation records, thus providing a

independent check of this following result from instrumentation independent check of this following result from instrumentation data.data.

However,However, the phonological change can the phonological change can only tell us the direction of climate only tell us the direction of climate change and cannot tell the rate or change and cannot tell the rate or magnitude of the change. magnitude of the change.

Can we know how may degrees the Can we know how may degrees the temperature may have increased from temperature may have increased from how many days earlier a bird or a how many days earlier a bird or a butterfly has migrated north to certain butterfly has migrated north to certain latitude?latitude?

Not yet, because the correlations, which have Not yet, because the correlations, which have been exclusively used in connecting changes been exclusively used in connecting changes in evolution of life-events of animals and in evolution of life-events of animals and plants with environmental conditions, “do not plants with environmental conditions, “do not allow us to discern whether the earlier allow us to discern whether the earlier reproduction is a direct response to warmer reproduction is a direct response to warmer temperatures, or to other factors that may temperatures, or to other factors that may also vary with climate, such as reproductive also vary with climate, such as reproductive resources and inter- and intra-specific resources and inter- and intra-specific competition,” as elaborated in Post et al. competition,” as elaborated in Post et al. (2001, in (2001, in Proc. R. Soc. LondonProc. R. Soc. London, B)., B).

When we try to tease theWhen we try to tease the information and information and single out those responses, the one organism single out those responses, the one organism

that biologically connects a species that biologically connects a species reproduction behavior with warmer reproduction behavior with warmer

temperature and dominates the other temperature and dominates the other organisms would be considered the signal, organisms would be considered the signal,

and the rest would be noises. and the rest would be noises. This signal-noise relationship can change in This signal-noise relationship can change in different analyses of varying aspects of the different analyses of varying aspects of the

problem. problem.

There are, of course, other ways to treat There are, of course, other ways to treat several major organisms simultaneously, and several major organisms simultaneously, and even include their nonlinear interactions (e.g., even include their nonlinear interactions (e.g.,

data mining).data mining).

To summarize the previous To summarize the previous slides: slides:

There are “absolute” noises There are “absolute” noises and erroneous information in and erroneous information in

data, but more often the data, but more often the noises are relative to the noises are relative to the

signal we want to examine.signal we want to examine.

Now, let’s examine what are the “absolute Now, let’s examine what are the “absolute noises” in the meteorological and climatic noises” in the meteorological and climatic

datadata► Instrumentation drift induced noise to data (ground Instrumentation drift induced noise to data (ground

sensor and satellite drift) sensor and satellite drift) ► Instrumentation upgrading induced changes in data Instrumentation upgrading induced changes in data

(sensor differences)(sensor differences)► Station’s geographical location change induced Station’s geographical location change induced

shift to the data (terrain and surface differences)shift to the data (terrain and surface differences)► Local surroundings change induced noise (a tree Local surroundings change induced noise (a tree

grows into full canopy and cools the surroundings of grows into full canopy and cools the surroundings of the station, inducing a cooling noise. Similarly the the station, inducing a cooling noise. Similarly the urban expansion may warm a previously rural area, urban expansion may warm a previously rural area, inducing a warming noise to the rural station inducing a warming noise to the rural station temperature data)temperature data)

► Different ways observers read the instrument (low Different ways observers read the instrument (low vs. high angle) vs. high angle)

► Observation time differences add noise in the data Observation time differences add noise in the data (for precipitation in some frequencies)(for precipitation in some frequencies)

How well have we extracted climate signal?How well have we extracted climate signal?(How have we identified and treated the (How have we identified and treated the

noises?)noises?)Many of the noises have been identified and their Many of the noises have been identified and their effects on signals minimized.effects on signals minimized.

► Instrumentation drift induced noise to data (Instrumentation drift induced noise to data (satellite position drift have been satellite position drift have been calculated and included in retrieval schemes for surface temperature and calculated and included in retrieval schemes for surface temperature and precipitationprecipitation) )

► Instrumentation upgrading induced changes in data (may have Instrumentation upgrading induced changes in data (may have considered in developing temperature data series)considered in developing temperature data series)

► Observation time differences add noise in the data (Observation time differences add noise in the data (Observation time Observation time effects were also estimated and included in finalizing the station observed effects were also estimated and included in finalizing the station observed precipitationprecipitation))

Others remain to be specified and their effects Others remain to be specified and their effects removed in developing climate dataremoved in developing climate data (lacking station (lacking station history has made these following noises very difficult history has made these following noises very difficult to clarify).to clarify).

► Station’s geographical location change induced shift to the dataStation’s geographical location change induced shift to the data► Local surroundings change induced noiseLocal surroundings change induced noise► Different ways observers read the instrument (can never be certain)Different ways observers read the instrument (can never be certain)

Those noises and biases in the climate Those noises and biases in the climate data have left uncertainties in the data have left uncertainties in the

results derived from the data.results derived from the data.

They have made it particularly difficult They have made it particularly difficult to examine local climate change – to examine local climate change –

because for large regions the noise and because for large regions the noise and biases may cancel each other and biases may cancel each other and reduce their effects on the results.reduce their effects on the results.

A very brief discussion onA very brief discussion on

Noise in climate models and their Noise in climate models and their outputsoutputs

Let me use an example to show the noise Let me use an example to show the noise resulting from numerical treatment of the resulting from numerical treatment of the governing equations of the atmospheric and governing equations of the atmospheric and oceanic motions.oceanic motions.

The generalized linear system of governing The generalized linear system of governing equations, describing a number of types of equations, describing a number of types of wave motions in the atmosphere and ocean, wave motions in the atmosphere and ocean, can be written as:can be written as:

dU

dti U , U U t ( ).

In finite difference, this equation is written (at In finite difference, this equation is written (at time = ntime = n××ΔΔt)t)

So, the true solution to the equation should So, the true solution to the equation should have an invariant amplitude, U(0) = the initial have an invariant amplitude, U(0) = the initial

amplitude.amplitude.

U t U e i t( ) ( ) 0

U n t U e in t( ) ( ) 0

In numerical models, various “finite differencing In numerical models, various “finite differencing schemes” are used to calculate the values of U at schemes” are used to calculate the values of U at time t and locations.time t and locations.These values are estimates of true U’s and contain These values are estimates of true U’s and contain noises intrinsic to those schemes. noises intrinsic to those schemes.

To evaluate the effect of those “numerical noises” on To evaluate the effect of those “numerical noises” on the solutions we use the von Neumann method. By the solutions we use the von Neumann method. By defining a variable defining a variable λλ (“distortion” of the solution from (“distortion” of the solution from the true one) the true one)

we can getwe can getU U en n in( ) ( ) 0

U Un n( ) ( ) 1 e i

Let’s focus on the amplitude of the amplitude of the Let’s focus on the amplitude of the amplitude of the modeled solution, |modeled solution, |λλ||nn×U(0). It is different from the ×U(0). It is different from the analytic (true) solution, and the coefficient analytic (true) solution, and the coefficient ||λλ||nn measures how different the amplitude, U(nmeasures how different the amplitude, U(nΔΔt), at time t), at time step n is from the true solution. We have these step n is from the true solution. We have these possibilities:possibilities:

1. |1. |λλ|>1 |>1 the numerical noise grows every time step the numerical noise grows every time step and quickly overwhelms the signal (the solution of the and quickly overwhelms the signal (the solution of the equation);equation);2. |2. |λλ|=1 |=1 neutral solution, noise is minimal (good); neutral solution, noise is minimal (good); andand3. |3. |λλ|<1 |<1 damping solution, noise “erodes” the damping solution, noise “erodes” the

signal and it will be gone during the model signal and it will be gone during the model integration. integration.

The value of The value of ||λλ| for some popular time | for some popular time differencing schemes is shown below.differencing schemes is shown below.

To summarizeTo summarize: Although researchers have : Although researchers have strived to minimize the numerical noises strived to minimize the numerical noises

resulting from various numerical schemes resulting from various numerical schemes used in models, those noises remain and used in models, those noises remain and

make numerical models and their predictions make numerical models and their predictions of climate suffer uncertainties. of climate suffer uncertainties.

Concluding remarks:Concluding remarks:► Except for the “absolute noise,” noise and signal are Except for the “absolute noise,” noise and signal are

relative and are determined by the nature of a problem.relative and are determined by the nature of a problem.► The sources of noise can be determined after the The sources of noise can be determined after the

research question is well defined.research question is well defined.► Various methods can be used to filter out or attenuate Various methods can be used to filter out or attenuate

the noises and minimize their effect on signal, although the noises and minimize their effect on signal, although such effect may always exist to varying magnitudes. such effect may always exist to varying magnitudes.

► Conventional climate data have many types of noises.Conventional climate data have many types of noises.► Phenology data of life-events of animals and plants Phenology data of life-events of animals and plants

provide an independent source of information to detect provide an independent source of information to detect climate and environmental change. A challenge for us to climate and environmental change. A challenge for us to use the data effectively is that the signal are biologically use the data effectively is that the signal are biologically and chemically intertwined with “noises.”and chemically intertwined with “noises.”

(“(“Sandhills Cranes in FlightSandhills Cranes in Flight” – photo by Michael ” – photo by Michael Forsberg)Forsberg)