Quality Control: closing the loop Reinhard Hanuschik, DFO/QC
description
Transcript of Quality Control: closing the loop Reinhard Hanuschik, DFO/QC
![Page 1: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/1.jpg)
July 2006 Instrument Scientist Retreat, Santiago
Quality Control: closing the loopReinhard Hanuschik, DFO/QC
1. Principles
2. Processes
3. Critical components
4. Conclusions
![Page 2: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/2.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• purpose of calibration:– calibrate the science observations
= remove (atm+tel)*(ins+det)– know the status of tel+ins+det
maintenance, intervention– calibrate the instrument
• purpose of QC:– ensure that calibs can do that – to a known and predictable accuracy
1. Principles
![Page 3: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/3.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• QC group:– 7+1 scientists– data processing
and quality control– roughly 400 GB
raw data per month, 11,500 processing jobs
– 80 SM packages– 9 VLT, 2 VLTI; 2
survey telescopes
![Page 4: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/4.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• aspects of QC:– formal compliance of file format:
PSO, DMD/DBCM, DMD/SAO– compliance with user constraints:
PSO (OB grading)– check raw files against reference files:
PSO– check pipeline products, do optimized
association, optimized processing: QC group
– compare to similar data (trending):QC group
![Page 5: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/5.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• standard QC
Paranal SciOps
QC Garching
raw proc (on-line pipe)
raw proc (off-line pipe)
certification
trending
10 days
fits file transfer
![Page 6: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/6.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• standard QC
• shared QC: close the loop
• feedback to enable corrections
Paranal SciOps
QC Garching
raw proc (on-line pipe)
raw proc (off-line pipe)
certification
trending
10 days
fits file transfer
HC process
HC monitor
< 1 day
op
s l
og
ftp
![Page 7: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/7.jpg)
July 2006 Instrument Scientist Retreat, Santiago
2. Processes
Components:
• On-site QC
• Health Check monitor
• QC processing and certification
• QC trending
10 days!
![Page 8: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/8.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• On-site QC– day/night astronomers:
• completeness check • visual quality checks: exposure level, patterns • inspection of pipeline products: are there any?
do look reasonable?
![Page 9: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/9.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• Reference frames– www.eso.org/qc/ALL/ref_frames/ref_frames.html
![Page 10: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/10.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• Health Check monitor– www.eso.org/qc/ALL/daily_qc1.html– day astronomer + QC scientist:
• how do new calibs compare to previous ones
• do new calibs show up at all
• outliers, alerts? trigger actions
• currently being upgraded to better organize information, closeups etc.
![Page 11: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/11.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• upgraded: Giraffe, VIMOS
![Page 12: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/12.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• QC process, certification– QC scientists in Garching:
• process all calibration data, SM science data• associate best-possible calibs to new data
– best: closest in time, certified, dynamically updated– correct dependencies (following the cascade)
• process with best-possible pipeline setup– optimized, customized parameters– maybe patched/improved beta versions
• measure and assess product quality– customized QC reports, experience– trending analysis
• certify products– released for archiving and delivery– released for usage in processing other data
![Page 13: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/13.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• Trending– done and maintained by QC group
• set of current and history trending plots• customized for each product type or instrument component• access to ASCII data
– www.eso.org/qc
![Page 14: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/14.jpg)
July 2006 Instrument Scientist Retreat, Santiago
– useful for:• maintenance planning (“by how much do the
zeropoints degrade over a year by dust accumulation”)
• complete instrument history (“how often in the past has grating XZ shown this kind of shift”)
• InsScientists, PIs, archive users (“how does my current dispersion rms compare to earlier observations in P74”)
![Page 15: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/15.jpg)
July 2006 Instrument Scientist Retreat, Santiago
3. Critical components, challenges
• calibration plan– definition of calibs and frequency must be
complete– must include Health Check calibs– execution must be complete:
• daytime (calOBbuilder)• twilight/nighttime (…)
![Page 16: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/16.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• on-site QC, Health Check monitor– define within IOT:
• regular set of HC calibs (regular spacing, daily or so)• set of key QC1 params to be included (cannot be all,
must focus on most critical ones):– DET, FILT, GRISM/GRAT, LAMPs, system throughput etc.
– don’t expect all trending info on the HC monitor• simply too much …• find under www.eso.org/qc/:• full history, all plots, all tutorial info, all text files
![Page 17: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/17.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• pipelines – QC1 parameters need to be pipeline-
produced and included in ops log files, to show up on the Health Check monitor
• Health Check monitor, trending:– maintenance continuous effort– complexity is a challenge!
![Page 18: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/18.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• Data transfer– delay by 8-14 days– large impact on QC task:
• large data volume affected by any problem discovered in Garching
• delivery bursts processing delays may happen
dvdMonitor
![Page 19: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/19.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• complexity– supporting 9 VLT + 2 VLTI plus 2 VST/VISTA– publish and maintain ~600 web pages– we are largely on our own with maintenance/
development of most components
![Page 20: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/20.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• IOTs– extremely important for setting up and
maintaining the QC process– shared QC requires each side to know what
the other side is doing and why
– we visit you come and visit us!
![Page 21: Quality Control: closing the loop Reinhard Hanuschik, DFO/QC](https://reader035.fdocuments.us/reader035/viewer/2022070419/56815a81550346895dc7ebf7/html5/thumbnails/21.jpg)
July 2006 Instrument Scientist Retreat, Santiago
• shared QC: – in excellent shape– but needs continuous efforts– can still be improved …
4. Conclusions
• key issues: – data delivery pattern– daytime QC/HC monitor– operational stability