NASSP Masters 5003F - Computational Astronomy - 2009

Lecture 3

• First, a bit more python.

• Then some noise statistics.

NASSP Masters 5003F - Computational Astronomy - 2009

Python ins and outs• We’re going to mostly read our data from

FITS files, using a module called pyfits.– http://www.stsci.edu/resources/

software_hardware/pyfits

• We’ll crunch up the data using a module called numpy.– http://numpy.scipy.org/

• For graphical output we’ll use module ppgplot, (2 ps) which is a (fairly crude) wrapper to a package called PGPLOT.– http://efault.net/npat/hacks/ppgplot/– http://www.astro.caltech.edu/~tjp/pgplot/BREAKING NEWS! pylab is better for graphical output.

NASSP Masters 5003F - Computational Astronomy - 2009

Manuals and example code

• You won’t need to download manuals for pyfits, numpy, scipy or pylab. I’ll do so and make them available from my home page. It’ll be much quicker for you to copy (or simply access) them from there.

• Some of these manuals are huge – I recommend you neither print or read them in entirety, but rather:– Read the bits you need using acroread.– Look at the example code I’ll provide you.

NASSP Masters 5003F - Computational Astronomy - 2009

Random variables – probability density

x

p(x)Average μ:

Variance σ2:

Units: probabilityper unit x.

xpxdxx

NASSP Masters 5003F - Computational Astronomy - 2009

Random variables – probability density

x

p(x)Estimate of μ:

Estimate of σ2:

N

iixN 1

22 ˆ1

1ˆ

NASSP Masters 5003F - Computational Astronomy - 2009

Random variables – probability density

x

p(x)

NASSP Masters 5003F - Computational Astronomy - 2009

Random variables – probability density

x

p(x)

x0

NASSP Masters 5003F - Computational Astronomy - 2009

p(x)

Noise• 2 most important distributions:

– Gaussian• …but not necessarily ‘white’

2

2

2exp

2

1,

xxp

NASSP Masters 5003F - Computational Astronomy - 2009

Noise– Poisson

(= i)

…but note Central Limit theorem.

NASSP Masters 5003F - Computational Astronomy - 2009

Gauss and Poisson

NASSP Masters 5003F - Computational Astronomy - 2009

Combinations of random variables

Weighted average:

Uncertainty propagation: if y = f(a,b),

Uncertainty in this:

Best SNR when:

where:

NASSP Masters 5003F - Computational Astronomy - 2009

Filtering and correlation

Uncorrelated ie ‘white’ noise (with a Gaussian probability distribution.)

NASSP Masters 5003F - Computational Astronomy - 2009

Filtering and correlation

Fourier transformed – looks the same – same power at all f – hence ‘white’.

NASSP Masters 5003F - Computational Astronomy - 2009

Filtering and correlation

Autocorrelation function.

Power spectrum:

where F is the Fourier transform of signal f, eg:

The autocorrelation function is:

NASSP Masters 5003F - Computational Astronomy - 2009

Filtering and correlation

Filtering operation correlates the noise. It is still Gaussian but no longer white.

NASSP Masters 5003F - Computational Astronomy - 2009

Filtering and correlation

FT is dominated in this case by low frequencies.

NASSP Masters 5003F - Computational Astronomy - 2009

Filtering and correlation

Autocorrelation function is broadened.

NASSP Masters 5003F - Computational Astronomy - 2009

Signals and noise

Natural signal

+ background

Instrumentalfiltering/correlation

General:

+ uncorrelatednoise

+ uncorrelatednoise

Natural signal

+ background

Instrumentalfiltering/correlation

XMM-Newton:

+ uncorrelatednoise

+ uncorrelatednoise

Natural signal

+ background

Instrumentalfiltering/correlation

Interferometry:

+ uncorrelatednoise

+ uncorrelatednoise

NASSP Masters 5003F - Computational Astronomy - 2009

Signal detection

Much harder.An obvious source.

NASSP Masters 5003F - Computational Astronomy - 2009

Signal detection• Parent function; data; model.

• Probability that the data results from a certain model:

• Reduced chi2: divide by deg free.

• Recall that Psignal = 1-Pno signal.

• Many types of signal, but only 1 no-signal.

• Hence test model with no source.

• Called the ‘null hypothesis’.

N

i i

ii my

12

22

NASSP Masters 5003F - Computational Astronomy - 2009

Signal detection

• Null hypothesis requires:– Perfect knowledge of background B.– A good estimate of σ.

• Sparse sources among gaussian: ok.

• But…– what about poisson bins with zero counts?

• Answer: Maximum Likelihood.

– what about crowded fields?• Answer: Bayes..?

– can we trust the distribution?• Answer: Monte Carlo.