S2 Data Quality Investigation
description
Transcript of S2 Data Quality Investigation
![Page 1: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/1.jpg)
LIGO-G030492-00-E
S2 Data Quality Investigation
John ZweizigCaltech/LIGO
![Page 2: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/2.jpg)
LIGO-G030492-00-E
Data Quality Investigation
Goals: Tabulate known problem Publish useful data segments with full info Analysis groups to decide what is appropriate for their
analysis pipelines
![Page 3: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/3.jpg)
LIGO-G030492-00-E
Data Quality Segment List
Official segment lists are maintained by Keith Riles &Peter Shawhan and are available via: http://tenaya.physics.lsa.umich.edu/~keithr/S2DQ/S2seglists.html
List of all locked segments generated/verified by Peter Shawhan.
Segments are given a permanent ID - split if bad or non-existent data found in part of a segment.
Error bit-mask associated with each (sub)segment – indicates which errors occurred in the segment.
Graphical segment status.
![Page 4: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/4.jpg)
LIGO-G030492-00-E
Segment Summary (Sample)
![Page 5: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/5.jpg)
LIGO-G030492-00-E
Tests Underway
We are studying the following potential problems:
1. DAQ Errors / Missing data
2. Control parameter changes
3. ADC Saturation
4. Peak-peak Outliers (from minute trends)
5. Calibration validity
6. Excessive noise in AsQ bands
![Page 6: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/6.jpg)
LIGO-G030492-00-E
DAQ Errors / Missing Data
Used BitTest triggers to find data with >32 successive 0 samples in AS_Q» Two 1 second drop-outs in H2:LSC-AS_Q» One 33-second and one 6-second drop-out in L1.
![Page 7: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/7.jpg)
LIGO-G030492-00-E
Control Parameter Changes
Rana (LLO & LHO), Betsy (LHO) and Gaby (LLO) reviewed the control parameter values and E-log throughout S2.
Non-standard values of several parameters were found from time to time including:» Filters not set.» MICH_GAIN, PRC_GAIN
Most problems don’t invalidate data although noise level or calibration may change in some cases.
![Page 8: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/8.jpg)
LIGO-G030492-00-E
ADC Saturation
Digital IFO signals may saturate if1. RF signal exceeds amplifier range
2. Demodulated signals exceed ADC range
AS, REFL and POB ADC values reconstructed from raw data and searched for either1. Signal values exceeds (216-1), or
2. Signal value repeated > 4 times in successive samples
Majority of times flagged due to » Loss of lock
» Accidental triggers (5 repetitions occur at a level of ~1/hr)
Saturation flagged in L1, H1 segment lists. H2 has segment–dependent saturation rates. Under investigation.
![Page 9: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/9.jpg)
LIGO-G030492-00-E
Peak-to-Peak Outliers
Gaby scanned IFO channel (AS, REFL & POB) minute trends for outliers in max-min values
Produced graphs of 102 outliers (>140cts) in H1 and 89 outliers (>70cts) in L1. All graphs are available online at: http://www.phys.lsu.edu/faculty/gonzalez/S2trends/
Outliers further classified by whether the are» Clustered: one of several in consecutive minutes» Correlated: Oulier seen in other IFO channels in addition to AsQ» AsQ Only: no other evidence seen.
![Page 10: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/10.jpg)
LIGO-G030492-00-E
Outlier 2 IFO Channel Trends
![Page 11: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/11.jpg)
LIGO-G030492-00-E
Outlier 2 Power Channel Trends
![Page 12: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/12.jpg)
LIGO-G030492-00-E
Calibration Validity
Gaby checked all SenseMon trends for valid calibration constants.
Segments where calibration is not valid are generally due to:» Missing calibration lines (=0)» Calibration not stable near 35Hz ( < 0.6 in L1)
Flagged segments totaled ~0.4% of H1 data, ~2.4% of L1 data.
![Page 13: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/13.jpg)
LIGO-G030492-00-E
Excess Noise in AsQ
RMS noise level monitored in wide bands by Data Quality Monitor (DataQual).
PDFs of average and maximum values for entire run may be found in:
» http://blue.ligo-wa.caltech.edu/gds/monitor_reports/DataQual/60daysummary» http://london.ligo-la.caltech.edu/gds/monitor_reports/DataQual/60daysummary
Preliminary selections made on all H1 bands by comparing maximum 4s RMS to a threshold
Low frequency bands seen to be correlated with windy conditions during data taking, but predominantly uncorrelated with glitches or noise in other bands.
![Page 14: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/14.jpg)
LIGO-G030492-00-E
Excess noise in AsQ(cont’d)
Excess noise in higher frequency bands (100-200, 200-400, 400-1kHz, 1k-7kHz) correlated with (glitches seen by DataQual glitch tool) and between bands.
Sources include» Broad-band noise episodes» Violin spontaneous ring-ups» ADCU reboots (should be vetoed independently)
![Page 15: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/15.jpg)
LIGO-G030492-00-E
Sample Average Histogram
![Page 16: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/16.jpg)
LIGO-G030492-00-E
Average RMS in 200-400Hz Band
![Page 17: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/17.jpg)
LIGO-G030492-00-E
Maximum 4s RMS in 200-400Hz
![Page 18: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/18.jpg)
LIGO-G030492-00-E
Violin Ring-ups?
![Page 19: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/19.jpg)
LIGO-G030492-00-E
H1 Broadband Noise Syndrome
![Page 20: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/20.jpg)
LIGO-G030492-00-E
10-100Hz Noise Spikes
![Page 21: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/21.jpg)
LIGO-G030492-00-E
200-400Hz Noise Trends
![Page 22: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/22.jpg)
LIGO-G030492-00-E
6 Glitch Rate Trend
![Page 23: S2 Data Quality Investigation](https://reader036.fdocuments.us/reader036/viewer/2022062322/56815018550346895dbdff14/html5/thumbnails/23.jpg)
LIGO-G030492-00-E
Questions For Burst Analysis
Cutting calibration, controls, saturation segments should be safe. Are they necessary?
Cutting p-p outliers and enormous noise excess in 10-100Hz band is probably safe (thresholds >> expected GW strain). Can we afford not to cut them (would a discovery be believable if there was a coincident l.f. noise excess?)
Are cuts based on AS_Q RMS alone safe? Under what circumstances can segments be cut based on these data?