Christian Wolf

Photometric Redshifts PHAT Meeting

Pasadena3-5 Dec 2008

Photo-z Ingredients & Application

Farb-bibliothek

Schätzer/Klassifikator

result

model data

estimator

spectral energy

distribution

PDF: p(z)

empirical dataor

external template

2-fittingartificial neural netlearning algorithms

Frequentist precision statistics:= “Using what IS there: N(z)!”

Bayesian frontier exploration:= “What do we (not) know: p(z)=?”

Model + Estimator Combinations

Code 2 NN

Model

Template Empirical

2

+ PDF Ambiguity warning

• NN– No PDF, no warning

• Template model+ Can be extrapolated in z,mag– Calibration issues– Priors’ issues

• Empirical model+ Good priors+ No calibration issues– Can not be extrapolated

Class Decision and z-EstimationClass Decision and z-Estimation

PStar

PGalaxy

PQuasar

type

redshift

redshift

ML

MEV

<z> ± z

∫∫∫ >> QuasarStarGalaxy ppp

Can include morphology etc.in statistics or neural net

Examples from COMBO-17

Classification ~98% complete at R<23Stars (~3,000)

White Dwarfs (~30)

Ultra-cool WD (1)

Galaxies (~30,000)

QSOs (~300)

Redshift Error Regimes

• Three regimes in photo-z quality

• Saturation– Model-data calibration offsets;

intrinsic dimensionality of class

• Transition– Locally linear colour(z) grid

• Breakdown– Globally nonlinear

colour(z) grid€

z ∝σ colour ∝N

S∝

1

100.4m⇒ logσ z ∝m

stars galaxies quasars

21 22 23 24 25 21 22 23 24 25 21 22 23 24 25

100%

0%

CA+B A+B

A+BCC

21 23 25 21 23 25 21 23 25 21 23 25

1

0.1

0.01

quasars @ z<2.2 quasars @ z>2.2quiescent galaxies starburst galaxies

Δz

R-mag

mag

SbSb

EE

z=0.52z=0.56

colour

colo

ur

Galaxies: Saturation & Transition

R=20 R=22 R=23.7

Galaxies at z~0.45

QSOs: Saturation at R<24

QSOs at z~2.8

R=23.8R=22.9R=21.5

rms 0.0087%-20%

outlier

Photo-Z Trouble

• Catastrophic failures & misclassifications

• Large z errors

• Local z bias

• Unrealistic z errors

• Model ambiguities in colour space (spotted?)

• PDF too unconstrained

• PDF wrong (calib, prior)

• Mismatch between data and model

Common Fixes

• Catastrophic failures & misclassifications

• Large z errors

• Local z bias

• Unrealistic z errors

• Model ambiguities in colour space

• PDF too unconstrained

• PDF wrong (calib, prior)

• Mismatch between data and model

Repair modelsor data

Add more data

Add model priors

Realistic data errors

Add More Data: Wider Range

• Wider coverage – Covers spectral features

across wider z range

• Add NIR data– For z > 1 galaxies

– Only weak & high-variance features in rest-frame UV

– Red z>1 galaxies with noisy optical data

• Add UV data (e.g. GALEX)– For QSOs (Ball et al. 2007)

– Lyman break at z < 2-3Abdalla et al. 2007

Add More Data: Narrow Filters

• Improve localization and contrast of features– QSO line detection avoids

catastrophic failures at z < 3

– Galaxies+QSOs: improve z

– Galaxy, star, QSO, WD,… ?

λ / nm

QE (

%)

Wolf 2001

Wolf, Gray & Meisenheimer 2005

z/(1+z) ~ 0.006

Add More Data: Narrow Filters

• Improve localization and contrast of features– QSO line detection avoids

catastrophic failures at z < 3

– Galaxies+QSOs: improve z

– Galaxy, star, QSO, WD,… ?

λ / nm

QE (

%)

z/(1+z) ~ 0.015

Wolf et al. 2004

Wolf 2001

Add Priors…

• Impact: ptot = pprior pcolour

– Reduce rate of bimodal PDFs

– Reduce larger z (up to √2)

• Explicit for template models– Luminosity function, range– Mag / z extrapolation ~”ok”

• Implicit for empirical models– Restricted in mag & z– Mag extrapolation wrong– Z extrapolation impossible– Representative sample?

p

p

p

S/N medium

S/N low

S/N high

prior

Repair (or Make) Templates

Budavari et al. 2000,2001

Repair Models: Uncommon Objects

• COMBO-17 field Abell 901/2– Super-cluster at zspec~ 0.16

– 800 members with R < 21

• Photo-z 2002: using SEDs by Kinney et al. (1996)

– 25% of S/N~100 members outliers with zphot~ 0.06

Δz ~ 0.1 SHOCK!

– Red spirals!

• Photo-z 2003: include dust-reddened old SEDs – ~1% outliers

Age

Dust

Wolf, Gray & Meisenheimer 2005

Kinney et al. templates 25% z outliers

Abell 901 z~0.16

Photometry & SEDs

• Photometry:– PSF-matched

– Calibration (obs. frame)

– Artefacts: instrumental, data reduction

– Error distribution, non-Gaussian systematics in Gaussian error floor

• Source variability– Stars: RR Lyr, long-term

– Galaxies: supernovae

• Photometric Blends– Transient blends by moving

objects

– Close neighbours

– Line-of-sight projections, strong lenses

– Binaries, WD+M etc.

• SED composition– AGN component

– Composite stellar populations

PSF-Matched Photometry

• Basic method:– Assume Gaussian PSF– Convolve to worst PSF– Photometry in aperture A

• Problems:– Local PSF variations– Non-Gaussian PSF

(Capak et al. 2007)

• Special case: Gaussian aperture & PSF

– Stronger weight to brighter object centre

– Aeff = PSF A(“space-based aperture”)

– If A and PSF Gaussian, then Aeff Gaussian as well

– Minimize computations:Fix Aeff & adjust A to PSF

2A = 2

eff - 2PSF

(Röser & Meisenheimer 1991)

• Non-gaussian aperture & PSF– Shapelet-based method

(Kuijken 2008)

What We Understand by Now

• Origin of local z bias– Observed-frame + rest-frame calibration– Non-flat priors: p(zph|zsp) vs. p(zsp|zph), z>0

• Origin of catastrophic outliers– Unrecognised z ambiguity in colour space– Wrong data / errors: blends, instrumental issues

• Minimum z variance levels– Intrinsic SED variations– Spectral resolution

Local Z Bias: Calibration

• N = number of filters, i.e. independent data points

• Calibration offsets in N=3 D– 1-D normalisation– 1-D z-bias– 1-D restframe SED bias– 1 out of N offset dimensions

causes a photo-z bias Δz

• More filters smaller Δz (proj. component ~1/√N)

• Narrow filters small Δz (larger col/z on feature)

• Spectroscopy with N~102..3: z without flux calibration

SbSb

EE

z=0.52

z=0.56

colourco

lour

Few-filter photo-z’s limited by calibration accuracy

Many-filter photo-z’s limited by number and resolution of filters

Catastrophic OutliersResult from undetected ambiguities: Also: wrong data/errors- Example: see shrinking training sample- 20% sample in 1:20 ambiguities causes overall 1% unflagged outliers

Intrinsic Variety: Z Error Support

• Example: – QSO near g-r~1 or z~3.7– Main signal: Ly forest in g,

but SEDs >0-D family

• Training sample in box– Redshift distribution:

mean 3.66, rms 0.115– RMS/(1+z) = 0.024

• Testing sample in box– RMS/(1+z) error 0.023

What To Work On: Data

• Define most effective & efficient data sets:

– From simulations (…which don’t rule out outliers)

• Describe data correctly:

– Consistent apertures across bands

– True photometric scatter by object

• Minimise unrecognised error sources in data:

– Error floor from photometric blends & transients

What To Work On: Models

• Templates etc.:– Best templates, rare objects with “different SEDs”

– Best priors, best extrapolation in (mag, z)

• Training samples:– Discretization effects, confidence limits on random-ness

– Propagation into n(z) errors and outlier risks

– Size matters: “What do I need?”

• Combine all approaches? – Empirical + extrapolated template model for all kinds of use