QUALITY CONTROL PROCEDURES -- DAILY RAINFALL...
24
Dr. G. KRISHNAKUMAR National Data Centre Office of Climate Research and Services QUALITY CONTROL PROCEDURES -- DAILY RAINFALL DATA
Transcript of QUALITY CONTROL PROCEDURES -- DAILY RAINFALL...
Dr. G. KRISHNAKUMAR National Data Centre
Office of Climate Research and Services
QUALITY CONTROL PROCEDURES -- DAILY RAINFALL DATA
4/25/2018 GKK 2
OUTLINE
DATA ARCHIVES
DATA STANDARD FORMS
GENERAL QC & QA PROCESS
STAGES & PURPOSES
UPDATION/MERGING OF RAINFALL DATA
PREPARATION OF WEEKLY AND MONTHLY
SUMMARY
4/25/2018 GKK 3
NDC FUNCTIONS UNDER HEAD CLIM. RES. AND SERVICES (CR&S),
PUNE MANDATES INCLUDE: COLLECTION AND PRESERVATION OF LONG
TERM CLIMATE DATA
COMPUTERISATION BEGAN FROM 1970s PUNCHING CARDS –> MAGNETIC TAPES – DISKS – CDs /DVD/HDs
STARTED WITH IBM 360, LATER TO EC 1040, VAX MAIN FRAME MACHINES AND NOW CLISYS CDMS
DATA SUPPLY STARTED WITH MEAGRE 100s TO NOW 5000 PER YEAR
ANNUAL REVENUE FROM SALE OF DATA : A FEW THOUSANDS IN EARLY PERIOD NOW SCALING CRORES INR
ABOUT N.D.C
4/25/2018 GKK 4
CHANGED INFO SCENARIO
4/25/2018 GKK 5
NDC DATA ARCHIVES
4/25/2018 GKK 6
DATA SOURCE • Of all network of observatories;
– 556 Surface Stations – 66 Pilot Balloon – 35 RS/RW – 206 Agro-meteorological – 701 Hydro-Meteorological – 10 Air-pollution BAPMON (Global Atmospheric
Watch) – 12000 Rain gauge Stations (Non-Departmental /
States) - Largest and maintained hetrogeneous – Rainfall Registration Authorities (State Governments)
• AWS & ARG/AGRO net works – 1500 & OVER 5000
respectively include rainfall parameter being very vital
4/25/2018 GKK 7
DAILY RAINFALL DATA FORMAT
LARGE NETWORK (IN INDIA)
HIGHEST DEMAND BY USERS
MANDATORY FIELDS AND DATA FIELDS
IMPORTANCE OF FORMAT
NECESSITY TOWARDS ARCHIVES
RAINFALL DATA FORMAT
EXPLANATION
DAILY RAINFALL DATA
4/25/2018 GKK 8
EL
EM
EN
T
CA
TC
HM
EN
T
NO
.
LA
TIT
UD
E
LO
NG
ITU
DE
ST
AT
ION
NO
.
BL
AN
K
YE
AR
MO
NT
H
RE
CO
RD
2
DAILY RAINFALL IN 0.1 MM FOR THE DATE
ST
AT
E N
O.
(IN
CO
DE
)
DIS
TR
ICT
NO
.
ST
AT
ION
NO
.
DIS
TR
ICT
NA
ME
ST
AT
ION
NA
ME
CL
AS
SIF
ICA
TIO
N
SE
CR
ET
CL
AS
SIF
ICA
TIO
N C
OD
E
CE
NT
UR
Y
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
MO
NT
HLY
T
OT
AL
RE
CO
RD
1
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
BL
AN
K
CO
LU
MN
1-3
4-5
6-7
8-9
10
11-1
2
13-1
4
15
16-1
9
20-2
3
24-2
7
28-3
1
32-3
5
36-3
9
40-4
3
44-4
7
48-5
1
52-5
5
56-5
9
60-6
3
64-6
7
68-7
1
72-7
5
76-7
9
80
81-8
2
83-8
4
85-8
6
87-1
02
10
3-1
18
11
9-1
23
12
4-1
25
12
6-1
27
MANDATORY DATA FIELDS MANDATORY
4/25/2018 GKK 9
COL.NO. ELEMENT AND EXPLANATION 1-3 Catchment No.
I. One digit in col. 1 to designate region number II. Two digits in col.2-3 to designate River Catchment Number III. Country has been divided into 6 regions IV. Each region has been divided into various river catchments (Max. = 26).
4-5 Latitude in full degrees (°N) 6-7 Longitude in full degrees(°E) 8-9 Station No. Two digit number in degree square. Numbers have been allotted to stations in
degree square to identify the exact station.
10 Blank 11-12 Year in two digits – Last two digits of the year 13-14 Month 01- Jan, ... ,12 – Dec 15 Record No. 1 or 2
Record 1 Dates
Record 2 Dates
16-19 1 17
Daily Rainfall (0.1 mm) for the date
20-23 2 18 24-27 3 19 28-31 4 20 32-35 5 21 36-39 6 22 40-43 7 23 44-47 8 24 48-51 9 25 52-55 10 26 56-59 11 27 60-63 12 28 64-67 13 29 68-71 14 30 72-75 15 31 76-79 16 Monthly Total 80 Blank 81-82 State No. (in code) 83-84 District No. 85-86 Station No. (available in case where Lat./Long. not available) 87-102 District Name 103-118 Station Name 119-123 Classification e.g. OBSY,HYDRO,REV,PWD,AERO,SRG,AGRO 124-125 Secret classification code. 126-127 Century
MA
ND
ATO
RY
DAT
A F
IELD
S M
AN
DAT
OR
Y
STATE NAMES AND THEIR CODES
4/25/2018 GKK 10
Code State Code State
01 Andhra Pradesh 18 Punjab
02 Arunachal Pradesh 19 Rajasthan
03 Assam 20 Tamilnadu
04 Bihar 21 Tripura
05 Gujarat 22 Union Territories
06 Haryana 23 Uttar Pradesh
07 Himachal Pradesh 24 West Bengal
08 Jammu & Kashmir 25 ISLS
09 Karnataka 28 SIKKIM
10 Kerala 29 JHARKHAND
11 Madhya Pradesh 30 UTTARANCHAL
12 Maharashtra 31 CHATTISGARH
13 Manipur 32 GOA
14 Meghalaya 33 DELHI
15 Mizoram 34 TELANGANA
16 Nagaland 99 North East India
17 Orissa
4/25/2018 GKK 11
THRESHOLD VALUES
Daily Rainfall (DRF) :
DRF ≤ 30 Cms (being considered heavy to be consistent and verifiable along with weather parameters and systems in a day-to-day basis).
Entered MRF and computed MRF (general check)
HRF ≤ MRF (general check)
MRF ≤ µ ± σ; µ - Mean ; σ - Std.Dev. (ext.check)
check with extremes from previous Climatological Tables
Check by comparing data from various sources of DRF (surface, DRF/DRMS) network stations
4/25/2018 GKK 12
QUALITY CONTROL CHECKS Stage 1: Primary level checks (manual) originating Clim. Station / hydrology division data entry level (manuscript to format) station and meta data details add if absent or wrong overall errors in format/values Stage 2: Secondary Level checks (program/manual) NDC & HS (if reqd.) Sequence (year, month) blank records invalid character/field value/multipunch, etc., mandatory fields (state codes, state/station names) duplicates (identical & mandatory fields) repitition of (non-zero rf values for consecutive days) consistencies (heaviest ≥ 3000 match monthly totals
4/25/2018 GKK 13
CONSISTENCIES
TEMPORAL DRF (d-1) ; DRF (d) and DRF (d+1)
SPATIAL
4/25/2018 GKK 14
DATA PROCESSING
QUALITY CONSISTENCIES PROCESS THROUGH PROGRAMMES
FOLLOWING WMO GUIDELINES TO CLIMATOLOGICAL
PRACTICES
PROGRAMS IN-HOUSE DEVELOPED FROM MAIN FRAME TO PC BASED
DUPLICATE, BLANK, SEQUENTIAL AND CONSISTENCY
CHECKS (PHYSICAL AND STATISTICAL)
CONSISTENCY WITH RESPECT TO TIME, SPACE STRINGENTLY BASED ON THRESHOLD VALUES ARE DONE
INTERNAL CONSISTENCIES ONLY ON SPECIAL CASES
4/25/2018 GKK 15
STAGES OF PROCESSING
Primary validated
Rf data (HS /originated)
Correct and list
error records
Multi-punch Invalid
character, state century year
blanks
Mandatory fields
Station names
Sorting (sequence)
Correct and list
error records
Correct and list
error records
Identical and other Duplicates
Repeat of same DRF, ≥ 30 , MRF, check for
blank records
DATA Archives
4/25/2018 GKK 16
INITIALISATION
ALL TYPES OF FIELD CHECKS
ALL TYPES VALUE CHECKS
4/25/2018 GKK 17
ERROR SAMPLES CENTURY, YEAR CHECK
4/25/2018 GKK 18
ERROR SAMPLES STATION NOT MATCH
4/25/2018 GKK 19
ERROR SAMPLES
IDENTICAL DUPLICATES/MANDATORY DUPLICATES CHECK
4/25/2018 GKK 20
ERROR EXAMPLE
4/25/2018 GKK 21
WEEKLY, MONTHLY AND ANNUAL RAINFALL
DAILY RAINFALL DATA TO WEEKLY RAINFALL DATA 52 OR 53 WEEKS ONE CARD WITH 13 WEEKS’ TOTAL COMPUTATION OF WEEKLY VALUES: ALL THE DAYS VALUES NECESSARY OTHER WISE BLANKS DAILY TO MONTHLY RAINFALL DATA 2 CARDS FOR ANNUAL EACH CARD 6 MONTHS COMPUTATION OF MONTHLY VALUE: ALL DAYS VALUES (WITHOUT BLANKS) MONTHLY VALUES WITH NO. OF RAINY DAYS/OBSNS SECOND CARD ANNUAL TOTAL ALSO, HEAVIEST 24 HRLY RAINFALL IN THAT YEAR RAINFALL AMOUNT WITH DATE AND MONTH
EVIDENCES / EVENTS
MAJOR FLOODS, MUMBAI 2005, UTTARAKHAND, 2013, CHENNAI 2015
EVIDENCES / EVENTS
NILAM CYCLONE (2012)
Drought (2015-16)
266 districts (11 states)
