1 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Overview of SDSS: I. Data Product Yun-Young Choi...

Aug 23 2005Aug 23 2005 SDSS-KSG Summer Workshop@AnmyeondoSDSS-KSG Summer Workshop@Anmyeondo

1

Overview of SDSS: I. Data Product

Yun-Young Choi

Korean Scientist Group

KIAS


2

ContentsData Distribution

Sky Coverage Data Products

Imaging Data Products Spectroscopic Data Products Other data products

Database Access Catalog Archive Server:

SkyServer Data Archive Server

User Interfaces SDSS Query Tools

Access to Files Imaging Files Spectroscopy Files

Hardware 2.5m Telescope and Instruments Imaging System

Filters Great Circle Drift Scanning

Spectroscopic System Photometric Telescope Data Acquisition

Software: (See also Algorithms on the DR4 web!)

Data Processing Factory Astrometric Pipeline The Postage Stamp Pipeline The Frames Pipeline

Measurements of Flux Measurements of Shape and Morphology

Photometric Calibration Target Selection

Galaxies Luminous Red Galaxies Quasars Other Science Targets

Plate Definition Spectroscopic Pipelines

Extraction and Calibration of Spectra Measuring Spectra

Ref. EDR paper by Stoughton et al. 2002 AJ 123, 485


3

Learning about the SDSS and the Data

PLEASE make use of SDSS documentation on the web!!

Survey summary: York et al. (2000 AJ 120, 1579) - technical summary paper of SDSS SDSS Project Book (1996-1997) - to get a general idea for the scope of SDSS

Data Processing and Formats of the outputs: EDR paper by Stoughton et al. (2002 AJ 123, 485) – the best description

available of the photometric and spectroscopic pipelines and target selection algorithms.

Technical Papers: Many of the pipelines and much of the SDSS hardware

Data Release SDSS DR4 web site (http://www.sdss.org/dr4/ ):o Data Products, Sky Coverage, Data Access, Algorithms

http://www.astro.princeton.edu/~strauss/welcome/welcome.html


4

Accessing the data

CAS and DAS Data Archive Server (DAS)

Flat files generated by the pipelines: flat-field images, the catalogs of objects in each field, the spectra, and so on.

Collaborator's Resource of the DAS at FNAL Catalog Archive Server (CAS)

SQL database with fast search capabilities See Schema Browser of Skyserver

Auxiliary Data Value-added catalogs (both public and private versions)

NYU Value-Added Galaxy Catalog SDSS quasar catalog Princeton reductions of the imaging and spectroscopic data CMU-PITT spectroscopic galaxy and cluster catalogs etc.


5

SDSS Data Flow1. Imaging Data Acquisition at APO2. DLTs mailed to Fermilab3. Pipelines run in successive order:

Serial Stamp Collecting (SSC) Pipeline (bookkeeping) Postage Stamp Pipeline (PSP) (flatfields, skybackground, PSF, etc) frames (aka Photo) astrom (astrometric calibration: assign precise coordinates to each

objects) nfcalib (final calibration: refinement in the positional and photometric

calibration4. Stuffing operational database, resolving

5. Spectroscopic target pipelines See DR4 web page, Target selection

6. Plate drilling at UW7. Spectroscopic Data Acquisition at APO8. DLTs mailed to Fermilab9. spectro2d and spectro1d runs10. Stuffing database11. All data eventually loaded into CAS

http://www.sdss.org/dr4/dataflow/index.html


6

SDSS Terms

● Run: a scan made by the imaging camera

● Runs make up a strip, 2 interleaved N and S strips make up each 2.5 deg wide survey stripe

● Rerun: number assigned to each (re)processing of the same run

● Camcol: 1 of the 6 camera columns comprising each run

● Field: 2048 pixel x 1361 pixel areas dividing each run

● Target: reduction used for spectroscopic target selection

● Best: best available reduction● Primary: ``main'' observation of an

object. Objects in the SDSS can be duplicated (runs, stripes, fields, etc.)

scan direction


7

Survey coordinate system (, )


8

DR4 Sky Coverage

Imaging Area 6670 sq. deg. Spectroscopic Area 5320 sq. deg.

Use “Foot” (footprint) server to convert RA/Dec to run/rerun/camcol/field See sky coverage page for the lists of target/best stripes, runs, fields, …


9


10

SDSS Imaging

Corrected frames

● fpC-$run-$filter$camcol-$field.fit (fpC-002126-g6-0428.fit )

● bias subtracted, flat-fielded, and purged of bright stars. 0.396”/pix 2048 x 1489 pixels (note adjacent fpC images overlap; e.g.,

along field direction primary area is only 1361pixels long) Background includes SOFTBIAS (=1000) + SKY (in the

header keyword SKY)

Retrieve from Data Archive Server (DAS)


11

Image-Related Products1. Atlas images

● fpAtlas*.fit: “postage-stamp” images of individual objects in a single field

2. PSF files ● psField*.fit: Photometric Calibration By Field - PSF

information etc.● SDSS PSF ~ a sum of 2 Gaussians and a power-law wing

3. Image masks● fpM*.fit: binary masks indicating saturation, interpolation,

detected objects, etc.4. Bad region masks

● mask*.csv: ascii files indicating regions affected by saturation/bright stars, satellite trails, bad seeing, and survey holes

Retrieve from Data Archive Server (DAS) Codes to read: See DR4 web page, Getting and using images


12

● tsField (Target Summary for a field)

● Field table in CAS (ref. schema browser) ● tsField-$run-$camcol-$rerun-$field.fit binary fits tables

● Field quality (good, acceptable, bad, …)● Photometric calibration zeropoints and extinction

coefficients● PSF and seeing information ● Sky (atmospheric extinction corrected)● Astrometric transformation● Plus other stuff …

Field Information


13

● tsObj (Calibrated Photometric Catalog)

●PhotoObj and similar tables/views in CAS● tsObj-$run-$camcol-$rerun-$field.fit binary fits tables

●Field quality, seeing, sky are also in fits header ●Fits tables contain

Run, rerun, camcol, field, idPosition: RA/Dec, etc.Object classifications and photometric processing flagsTarget selection flagsVarious calibrated SDSS magnitudes Surface brightness, concentration index, radial profilesAdaptive momentsLots of other quantities … see data model

Calibrated Object Lists


14

SDSS Photometry● Count rate f/f0 = counts/exptime * 100.4*(aa + kk * airmass)

exptime=53.907456, aa=zeropoint, kk=extinction coefficient tsField file or Field table

● Conventional magnitude = -2.5 * log10(f/f0) ● asinh magnitude = -(2.5/ln(10)) * [asinh((f/f0)/2b)+ln(b)]

Lupton, Gunn, & Szalay 1999, AJ, 118 b is a softening parameter, set to be about 1 sigma of sky noise in each filter

(see DR4 web site) Difference between asinh and conventional mag < 1% for objects brighter

than asinh mag m(f/f0 = 10b) = 22.12, 22.60, 22.29, 21.85, 20.32 for ugriz

Corrections to convert SDSS magnitude system to AB magnitudes:

u(AB) = 22.5 - 2.5*log10(fu) - 0.042 g(AB) = 22.5 - 2.5*log10(fg) + 0.036 r(AB) = 22.5 - 2.5*log10(fr) + 0.015 i(AB) = 22.5 - 2.5*log10(fi) + 0.013 z(AB) = 22.5 - 2.5*log10(fz) - 0.002


15

SDSS Magnitudes

1. PSF: Fit of a PSF model to the object2. Fiber: Magnitude inside the 3” aperture to estimated the flux

seen by spectroscopic fiber aperture (images convolved to 2” seeing first)

3. Petrosian: Magnitude including the same fraction of flux, independent of galaxy’s angular size. Very noise for faint galaxy

4. Model: The better of the fits to a PSF-convolved deVaucouleurs (deV) or exponential (exp) galaxy profile (aperture set by r band)

5. cmodel: The best linear combination of the deVaucouleurs and exponential fits in each band; the flux is Fcomposite= fracDeV FdeV + (1 - fracDeV) Fexp

See DR4 web page, Photometry in Algorithms


16

SDSS Magnitudes (cont’d)

Galactic Extinction Correction Schlegel, Finkbeiner & Davis (1998) Fluxes are not Galactic extinction corrected, and are in

nanomaggies. Given as reddening in the tsObj files or extinction in the CAS

To compute magnitudes in arbitrary circular aperture, can use the radial surface brightness profiles Given as the average surface brightness in a series of annuli Units: maggies/sq. arcsec, where 1 maggie of flux has an

AB magnitude of 0 See profMean in the tsObj files and the PhotoProfile table in

CAS


17

Example Magnitude Usage

Photometry of bright galaxies: e.g. for main sample galaxies, use Petrosian magnitudes (model independent, S/N remains good to r=20 or so)

Photometry of galaxies: cmodel magnitude (smaller noise for faint galaxies)

Colors of galaxies: model magnitudes (aperture from r band applied to all filters, unlike for cmodel magnitudes)

Photometry of distant quasars: PSF magnitude (unresolved objects)

Colors of stars: PSF magnitude


18

Target Selection Flags

● Flags are used extensively to indicate type, status, quality, etc., etc.

primTarget: 32-bit primary target selection flagMain galaxies, quasars, high-z quasars, LRGs ..

secTarget: secondary target selection flagUsed to indicate calibration stars, skies, guide stars, etc.

See DR4 web page, Target selection


19

primTarget flags

QSO_HIZ = 0x1 = bit # 0 = 1 QSO_CAP = 0x2 = bit # 1 = 2 QSO_SKIRT = 0x4 = bit # 2 = 4 QSO_FIRST_CAP = 0x8 = bit # 3 = 8 QSO_FIRST_SKIRT = 0x10 = bit # 4 = 16 QSO_MAG_OUTLIER = 0x2000000 = bit #25

=33554432QSO_REJECT = 0x20000000 = bit #29

=536870912 GALAXY_RED = 0x20 = bit # 5 = 32 GALAXY_RED_II = 0x4000000 = bit #26 =

67108864 GALAXY = 0x40 = bit # 6 = 64 GALAXY_BIG = 0x80 = bit # 7 = 128GALAXY_BRIGHT_CORE= 0x100 = bit # 8 = 256ROSAT_A = 0x200 = bit # 9 = 512 ROSAT_B = 0x400 = bit #10 = 1024 ROSAT_C = 0x800 = bit #11 = 2048 ROSAT_D = 0x1000 = bit #12 = 4096 ROSAT_E = 0x8000000 = bit #27 =134217728

STAR_BHB = 0x2000 = bit #13 = 8192 STAR_CARBON = 0x4000 = bit #14 = 16384 STAR_BROWN_DWARF = 0x8000 = bit #15 =32768 STAR_SUB_DWARF = 0x10000 = bit #16 = 65536 STAR_CATY_VAR = 0x20000 = bit #17 = 131072 STAR_RED_DWARF = 0x40000 = bit #18 = 262144 STAR_WHITE_DWARF = 0x80000 = bit #19 = 524288 STAR_PN = 0x10000000 = bit #28 =268435456 SERENDIP_BLUE = 0x100000 = bit #20 = 1048576 SERENDIP_FIRST = 0x200000 = bit #21 = 2097152 SERENDIP_RED = 0x400000 = bit #22 = 4194304 SERENDIP_DISTANT = 0x800000 = bit #23 = 8388608 SERENDIP_MANUAL = 0x100000 = bit #24 =

16777216

e.g., to select main sample galaxies: (primTarget & 64) > 0

0.24


20

Photometric Processing Flags

● Flags objc_flags, objc_flags2 in tsObj files, flags in CAS (also flags for each individual filter)

Important to check flags to eliminate problem objects or junk Examples given to derive clean samples of stars and galaxies

Primary objects to avoid duplicates Reject saturated objects Reject objects with deblending problems Reject objects with interpolation problems Other cuts …

See DR4 web page, Object Flags See sdssMaskbits.par in IDLUTIL


21

Tiling Tile – spectroscopic fiber-plug plate

A circular FoV with r=1.49°, 640 fibers (48 for observations of blank sky and spectrophotometric standards)

approximately 2000 plates by the end of its 5 year survey Minimum separation of fiber center, 55”: 100 kpc at about z = 0.2 Problem: the dynamics of binary galaxies, the study of subclustering, and velocity

dispersions in rich clusters, etc. Minimizing the # of tiles necessary to observe all the desired targets and

maximizing efficiency when placing these tiles and assigning targets to each tile. Allocating fibers to the set of decollided targets (about 90% of all targets) → using

unallocated fibers to resolve fiber collisions in the overlaps. a higher priority target is guaranteed never to be eliminated from the sample due to a

collision with a lower priority object Priorities: brown dwarfs and hot standards > QSOs > galaxies and LRGsPriorities: brown dwarfs and hot standards > QSOs > galaxies and LRGs

Tiling Spectroscopic Platesreference: Blanton et al. (2003) AJ 125,2276

55"

approximately 2000 plates by the end of its 5 year survey


22

Tile Placement Distributing tiles uniformly across the sky and then using a cost

minimization scheme to perturb the tiles to a more efficient solution. SDSS requirements: assign fibers to 99% of the decollided targets.


23

Spectra

1. 2D spPlate-$plate-$mjd.fit 640 spectra together in single fits file for each plate Flux-calibrated spectra, inverse variance, quality masks, and other

supporting information No measured parameters (e.g., redshifts)

2. 1D spSpec-$plate-$mjd-$fiber.fit 1 fits file for each fiber of each plate Calibrated spectrum, continuum-subtracted spectrum, noise, quality

mask Spectral classification, redshift, redshift error and confidence Binary tables with line measurements, line indices, emission-line and

cross-correlation redshift information, …

3. spDiag-$mjd-$plate.par Summary information for all 640 spectra on each plate Object id, target flags Spectral classification, redshift, redshift error and confidence

In CAS, use the SpecObj, SpecLine, and SpecLineIndex tables/views to get spectroscopic parameters


24

Spectroscopic Information

● Generate vacuum wavelengths using = 10(COEFF0 + COEFF1*i), where i denotes the (zero indexed) pixel number, and COEFF1 and COEFF2 are from file headers

● Check redshift confidence z_Conf, and also status z_Status and warning z_Warnin flags (eg. z_Conf > 0.7 based on eye checks of galaxy spectra)

● The spectrophotometry is good (e.g., overall residual offsets vs. gri fiber magnitudes/colors of < a few percent); see documentation on DR4 web site for details


25

● Mostly on Stripe 82, including u-selected galaxies, low-z galaxies, deep LRGs, faint quasars, spectra of everything, stellar programs, …

Retrieve from DAS or using PlateX table in CASsee sdss-archive/2511, and instructions are forthcoming

on DR4 web site

See the Southern Equatorial Survey plates page at http://www-sdss.fnal.gov/targetlink/southernEqSurvey/

See Ivan Baldry’s page and catalogs at http://mrhanky.pha.jhu.edu/~baldry/sdss-southern/

Southern Survey and Special Spectroscopic Programs

1 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Overview of SDSS: I. Data Product Yun-Young Choi...

Documents

Transcript of 1 Aug 23 2005SDSS-KSG Summer Workshop@Anmyeondo Overview of SDSS: I. Data Product Yun-Young Choi...