Beating Noise Observational Techniques
description
Transcript of Beating Noise Observational Techniques
![Page 1: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/1.jpg)
Beating NoiseObservational Techniques
ASTR 3010
Lecture 11
Textbook
![Page 2: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/2.jpg)
From previous classes• We learned that in the astronomical data pre-processing,
bias subtraction, dark subtraction, flat-fielding, bad pixel correction, cosmic ray hits removal
are important.
• There are several observation techniques that can take care of the above correction nearly automatically.
![Page 3: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/3.jpg)
Subtraction of two images taken with a small offset• image1 – image2
• bad pixels, bias, and dark are removed!• But, cosmic rays remain. • Need to apply flat-fielding
- =
![Page 4: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/4.jpg)
?Dither Observation?• Dither Observation
Series of images taken with telescope offsets.
box5
box4
![Page 5: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/5.jpg)
Dithering Observation
dead pixels
hot pixels
cosmic rays
Observation No 1
Detector Array
![Page 6: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/6.jpg)
Dithering Observation
dead pixels
hot pixels
cosmic rays
Observation No 2
![Page 7: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/7.jpg)
Dithering Observation
dead pixels
hot pixels
cosmic rays
Observation No 3
![Page 8: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/8.jpg)
Dithering Observation
dead pixels
hot pixels
cosmic rays
Observation No 4
![Page 9: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/9.jpg)
Dithering Observation
dead pixels
hot pixels
cosmic rays
Observation No 4
![Page 10: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/10.jpg)
Dithering Observation• Five observations
obs1 … obs5
flux1
flux2
flux3
flux4
flux5
True Flux
![Page 11: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/11.jpg)
Dithering Observation• Five observations
This will take care of bias, dark, cosmic ray hits, bad pixels!
obs1 … obs5
flux4 < flux5 < flux2 < flux1 < flux3
Ordered list
median value
![Page 12: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/12.jpg)
Dozens of Dithered images with small offsets• SuperResolution
![Page 13: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/13.jpg)
With the P&A telescope
Not Yet!
![Page 14: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/14.jpg)
P&A Dithering by Hand
![Page 15: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/15.jpg)
Real example : Keck NIRC2 AO observation of HIP88945
![Page 16: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/16.jpg)
Shift and Add
look at those hot pixels!!
![Page 17: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/17.jpg)
# demonstrating the Shift and Add concept.## import librariesfrom ds9 import *from numpy import *import pyfits
im1=pyfits.getdata('n0002.fits',ignore_missing_end=True)im2=pyfits.getdata('n0003.fits',ignore_missing_end=True)im3=pyfits.getdata('n0004.fits',ignore_missing_end=True)im4=pyfits.getdata('n0005.fits',ignore_missing_end=True)im5=pyfits.getdata('n0006.fits',ignore_missing_end=True)
# create a big enough array to cover any shifted images...BIG = zeros((1201,1201))# Then, we will find the positions of the star in each image.# Shift the image in the BIG array such that the star appears# at the center of the array (601,601)posx_im1=276posy_im1=279BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im1
posx_im1=397posy_im1=162BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im2
posx_im1=155posy_im1=403BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im3
posx_im1=397posy_im1=401BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im4
posx_im1=154posy_im1=161BIG[601-posx_im1:601-posx_im1+512,601-posy_im1:601-posy_im1+512] += im5
d=ds9()d.set_np2arr( transpose(BIG) )
![Page 18: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/18.jpg)
Shift and Median is better!!
Python HW2:
Create a Python script doing this!
![Page 19: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/19.jpg)
Median sky subtraction• When your object is
extended… or you need to make a pretty picture…
• in near-IR: useful for 10-20 min depends on the quality of the night.
![Page 20: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/20.jpg)
In Astronomical observations…• Three major noise sources…o Poisson noise (aka “shot noise”)o Sky noiseo Electronic noise
• Sky noise limited observation• Dark current limited observation• Readout noise limited observation(?)
![Page 21: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/21.jpg)
Typical stellar sources in the Solar Neighborhood
Noise versus signal at various wavelengths
Optical near IR mid IR
101
109
Sky noise
Detector noise
Sign
al S
tren
gth
Extremely bright objects
High red shift galaxies
105
![Page 22: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/22.jpg)
Coadds• Because of the increased sky background, frames will get saturated in short
exposures. variable sky is the limiting factor sky limited observations.
• Take as many unsaturated frames as necessary and combined them later.too many files for a single object (several hundreds)on-chip combine of multiple exposures = “coadd”
![Page 23: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/23.jpg)
Near-IR observation Tips• Need to sample the skyo through ditheringo or from a dedicated sky observation in the nearby field needs to be dithered
also. why?• Choose a right number of coadds. o Choose the maximum frame exposure time which will stay in the linear regime.o Then, from the required total exposure time, calculate the necessary number of
coadds.• Sky level is varying faster at longer wavelengths!o at K-band (2.2 μm) : up to 20 minuteso at L-band (3.6 μm) : max is about a minuteo at mid-IR (10 μm) : fraction of a second different strategy
• Dither• Repeat the dither sequence as many times as necessary.
![Page 24: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/24.jpg)
Observation Planning : near-IR• You will be observing at the K-band (2.2 micron), and you expect that your
source would have a count rate of about 1,000 counts per second where this signal will be spread over ~10 pixels due to the PSF structure. Some other relevant data areo Detector linear regime: count < 10,000 countso Sky brightness at K = 100 counts per second per pixelo Readout noise = 10 counts per readouto Dark current = 25 counts per second per pixelWhat would be the best observing strategy for this object if you want your final S/N~100?
![Page 25: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/25.jpg)
Observation Planning : near-IR• Source ~ 2,000 count per second over 4 pixels. Goal S/N >= 100o Detector linear regime: count < 8,000 countso Sky brightness at K = 1,000 counts per second per pixelo Readout noise = 10 counts per readouto Dark current = 100 counts per second per pixel
Per pixel per second signal = 500 source count + 1,000 sky count + 100 dark current = 1,600 counts/pxl/sec In 5 sec exposure, the source will get into a non-linear regime max exposure time should be less than 5sec or we should “coadd”
In a single 5 second exposure : S/N = 500 / sqrt(1600)~12 if we coadd 16 frames, then a single exposure will be 40 second long and S/N=12*sqrt(16)=48
For a box5 dither observation pattern, we will get five images total readout noise = 10 * sqrt(5) = 22.4 counts. This is negligible to shot noises from source+sky+detector.
If we take one image (40 sec long with 8 coadds) using a box5 pattern final S/N=48 * sqrt(5) = 107
The total duration of the observation is 5 x 40sec + total overhead (~1 min) = 4 min. At K-band, the sky will be invariable for up to ~20 min. So, this will be OK.
![Page 26: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/26.jpg)
Nod & Shuffle• "Nod and Shuffle" is a technique used to obtain very faint spectra• the natural glow of the nighttime sky would overwhelm the extremely faint
objects (e.g., far, far away galaxies)• N&S allows astronomers to effectively subtract away the bright spectral emission
lines and fainter continuum of our atmosphere's nighttime glow while retaining the faint spectra of dim, red galaxies. This also significantly reduces readout noise.
1. obtaining a spectrum of an object2. moving the electrical charge built up by the image of the spectrum on the CCD
to another location "buffer storage" on the CCD (Shuffling)3. shifting the position of the telescope slightly (Nodding) so that the spectrum of
the target object shifts to a different part of the CCD4. obtaining another spectrum in the nodded position5. shuffling the charges moved in step 1 back to the original position (while
moving the charges from step 4 to the buffer storage)6. nodding the telescope back to the position in step 17. repeating until enough light is collected.
![Page 27: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/27.jpg)
Sky cancellation: ‘nod and shuffle’Storage of ‘sky’ image next to object image via ‘charge shuffling’Zero extra noise introduced, rapid switching (60s)
A
B
A-B
Typically A=60s/15 cy: 1800s exposure10-3 subtraction
![Page 28: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/28.jpg)
Another example
![Page 29: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/29.jpg)
Weird image artifacts
Latent images : badly saturated pixels do not come back to normal right away!
![Page 30: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/30.jpg)
Weird image artifacts
ghost images : multiple reflections within an optical element (e.g., filter, beam splitter, entrance pupil) seen near a very bright star
![Page 31: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/31.jpg)
Handling various problems at IRNon-uniform QE Flat-field
High background level• Point sources• Extended sources
Dithering and coadds• Small dither offset• Off-chip dither
Non-linearity of detectors Stay within linear range
Memory effects (latent) Do not saturate
Bad pixels Mask or median out from dithers
Cosmic rays Shift and add
Dark current Subtracts out with sky
Hot rows, hot pixels, amplifier glow
Subtracts out well
![Page 32: Beating Noise Observational Techniques](https://reader036.fdocuments.us/reader036/viewer/2022062520/56816252550346895dd29a3b/html5/thumbnails/32.jpg)
In summary…
Important Concepts• Various noise sources at different
wavelengths and their relation to the observing techniques.
• Median combining of images
Important Terms• Dither observation• Shift and add• Shift and median• Coadd• Nod-and-shuffle
Chapter/sections covered in this lecture : 9