11

Quality Assurance and Data Quality Assurance and Data Quality for Stormwater Quality for Stormwater

ProgramsPrograms

Richard SmithRichard SmithINCOGINCOG

June 27, 2006June 27, 2006

88thth Annual EPA Region 6 MS4 Annual EPA Region 6 MS4 Stormwater ConferenceStormwater Conference

22

Data Quality and Quality Assurance• ““QA” is an essential component of creating and QA” is an essential component of creating and

maintaining the maintaining the usefulness of datausefulness of data and information. and information.

• QA addressesQA addresses::

• ““MonitoringMonitoring” data (e.g. field and lab measurement data)” data (e.g. field and lab measurement data)

• ObservationObservation data (e.g. visual inspections) data (e.g. visual inspections)

• RecordsRecords of conversations, meetings, emails, letters, etc. of conversations, meetings, emails, letters, etc.

• Data Data summarization and analysessummarization and analyses (e.g. statistics, estimates) (e.g. statistics, estimates)

• Reports and data transfersReports and data transfers (e.g. spreadsheets, letters) (e.g. spreadsheets, letters)

• In-house processesIn-house processes for communication and record-keeping for communication and record-keeping

• Storage, calibration and useStorage, calibration and use of equipment and supplies of equipment and supplies

“QA = CYA”

33

How Will Your Data Be Used ?• Permitting AuthorityPermitting Authority – (Annual Report / DMRs / Phase II – (Annual Report / DMRs / Phase II

audits / P.A. pollution & compliance enforcement).audits / P.A. pollution & compliance enforcement).

• Other Other State and Federal AgenciesState and Federal Agencies – to fulfill their own data – to fulfill their own data collection efforts for projects / databases.collection efforts for projects / databases.

• City CouncilsCity Councils / / County CommissionsCounty Commissions – program updates, – program updates, budgets, justify additional resources or changes to budgets, justify additional resources or changes to ordinances, etc.ordinances, etc.

• Internal Staff RequestsInternal Staff Requests – information flow within city. – information flow within city.

• Citizens / Local PartiesCitizens / Local Parties – public education, data requests. – public education, data requests.

• Local Enforcement ActionsLocal Enforcement Actions – building your case. – building your case.

• Local MediaLocal Media – radio, newspaper, television. – radio, newspaper, television.

The Data Needs will dictate how you collect, store and report data.

44

Budget itemsBudget items

PhotographsPhotographs

Phase II Data Streaming

Data Managemen

t System

Lab & Field DataLab & Field Data

Visual

ObservationsVisual

Observations

Inspection

InterviewsInspection

Interviews

Data from Agencies

Data from Agencies

Management ReportsManagement Reports

Reports to City StaffReports to City Staff

Reports to CitizensReports to Citizens

DATA IN-FLOWS DATA OUT-FLOWS

Data to AgenciesData to Agencies

ODEQ Annual ReportODEQ Annual Report

Program AuditsProgram AuditsComplaint

Tracking Complaint

Tracking

Who, what, where, when, why, how

Who, what, where, when, why, how

BMP QuantitiesBMP Quantities

Training Records

Training Records

Map DataMap Data

Reports to CouncilsReports to Councils

Data for CourtData for Court

Information to MediaInformation to Media

Enforcement DataEnforcement Data

55

MS4 / FACILITYINSPECTIONS

CONSTRUCTIONINSPECTIONS

EDUCATION & TRAINING

DATA MANAGER& REPORTING

CITY MANAGERPUBLIC WORKS DIR.

CITY COUNCIL

STORMWATERPROGRAMMANAGER

CONSULTANTS

AGENCIES &SUBSTATES

LOCAL CITIZENS & BUSINESSES

VENDORS

Stormwater Program ManagerMany cities and counties

struggle to define a management structure.

66

• Map dataMap data. . . . . . . . . . . .. . . . . . . . . . . . GIS, GPS, AutoCAD, paper.GIS, GPS, AutoCAD, paper.

• QA methodsQA methods . . . . . . .. . . . . . . What’s required, most efficient.What’s required, most efficient.

• Electronic Electronic data setsdata sets. . . . . .. . . . . . Tier II, RCRA, TRI, etc.Tier II, RCRA, TRI, etc.

• Field Field test kitstest kits. . . . . . . . . .. . . . . . . . . . D.O. pH, Cond., etc.D.O. pH, Cond., etc.

• Other Other field gearfield gear . . . . . . . .. . . . . . . . Samplers, bottles, etc.Samplers, bottles, etc.

• Data ManagementData Management::

• Software for supervision, records, reportsSoftware for supervision, records, reports

• Forms for inspections, contact records. etc. Forms for inspections, contact records. etc.

• Data and monitoring Quality Assurance (QA)Data and monitoring Quality Assurance (QA)

• Sampling and Lab servicesSampling and Lab services

Most Phase IIs Need Help With...

77

Need for Data Management

Paper Filing System

Computer Software:

Spreadsheet or Database

Commercial Vendor Software

Customized / Consultant Developed

Data Management Options

Consider Your Ability to:

• Purchase

• Learn

• Use

• Maintain

88

• Many circumstances require Many circumstances require verification of verification of informationinformation::

• Measurement data for Measurement data for court casescourt cases or enforcement. or enforcement.

• ReportsReports to City Councils, Commissions, citizens, to City Councils, Commissions, citizens, contractors, consultants, agencies and in-house use.contractors, consultants, agencies and in-house use.

• Regulatory decision-makingRegulatory decision-making (e.g. violations of WQS). (e.g. violations of WQS).

• Political falloutPolitical fallout if your data is discredited. if your data is discredited.

• Formal documentation of QA procedures is wise.Formal documentation of QA procedures is wise.

• Don’t expect to have a fully developed QA program Don’t expect to have a fully developed QA program quickly – give yourself time to learn and experiment quickly – give yourself time to learn and experiment with what works best for your local needs.with what works best for your local needs.

Why is QA Important ?

99

To What Activities Should QA Apply ?• Technically, Technically, all activitiesall activities pertaining to any type of data pertaining to any type of data

collection, analysis, manipulation and reporting....collection, analysis, manipulation and reporting....

• Determine your Determine your highest priorities firsthighest priorities first. For example:. For example:

• Visual inspections / lite-monitoring of MS4Visual inspections / lite-monitoring of MS4

• Source tracking inspectionsSource tracking inspections

• Construction site inspectionsConstruction site inspections

• If If private consultantsprivate consultants and/or and/or labslabs are used, be sure that are used, be sure that they employ appropriate and adequate QA to their they employ appropriate and adequate QA to their activities.activities.

• Make this part of your Make this part of your contract specificationscontract specifications, especially if , especially if their work is needed for enforcement or court evidence.their work is needed for enforcement or court evidence.

1010

Even Simple Test Kits Need Attention

• Expired chemicalsExpired chemicals - field test kits have chemicals that expire - field test kits have chemicals that expire – keep records of refresh purchases.– keep records of refresh purchases.

• Some kits require “Some kits require “calibrationcalibration” – use fresh calibration ” – use fresh calibration standards and keep a log of calibration activities.standards and keep a log of calibration activities.

• Some kits can measure “Some kits can measure “check samplescheck samples” (of known analyte ” (of known analyte concentration) – perform checks frequently and keep concentration) – perform checks frequently and keep records of all tests.records of all tests.

• SOPsSOPs help ensure continuous proper use, and aid in training help ensure continuous proper use, and aid in training new employees.new employees.

• MaintenanceMaintenance includes verifying equipment function and includes verifying equipment function and replacing defective parts before re-use of the kit. Detailed replacing defective parts before re-use of the kit. Detailed maintenance records should be kept.maintenance records should be kept.

1111

Types of Field Instruments

Comparator Test Kits

Test Strips

Titrations

Colorimeters

Multi-Test Kits

These types of kits rely upon

chemical reactions to indicate

the presence and/or

concentration of one specific

parameter.

No endorsement of products is intended.

An SOP should be written for

each type of kit.

1212

Types of Field Instruments

Test Pens

Single Parameter Meters

Multi-Parameter Meters

Field Fluorometer

Turbidimeter

Flow Meter

These types of kits

measure directly the

concentration of one or

more parameters.

No endorsement of products is intended.

An SOP should be

written for each device

and procedure.

1313

QA Concepts

Precision is a measure

of how close multiple

readings are of the same

sample.

Accuracy is a measure

of how close a reading

is to the known value.Both are important concepts

in assessing quality of data.

Both are normally reported

with laboratory data.

1414

More QA Concepts

Representativeness

Completeness

Comparability

Chain of Custody

Sample Labels

Does the sample represent what you intend?

Do you have enough samples?

Are sampling methods useable to others?

Legal documentation of “ownership” of samples during all phases of transport.

Adequate information on each sample bottle to show ownership, collection information and intended analyses.

1515

Types of QA Documentation to Consider• QAPPQAPP – QA Project Plan - – QA Project Plan - how you conduct data how you conduct data

gathering, analysis and reporting for your Phase II gathering, analysis and reporting for your Phase II program.program.

• DQOsDQOs – Data Quality Objectives – – Data Quality Objectives – deciding in the deciding in the beginning what types of data you will be collecting beginning what types of data you will be collecting and, most important, and, most important, how it will be usedhow it will be used..

• QMPQMP – Quality Management Plan – – Quality Management Plan – down the road, down the road, you may consider this – describes your overall data you may consider this – describes your overall data management and communication system.management and communication system.

• SOPsSOPs – Standard Operating Procedures – – Standard Operating Procedures – detailed detailed instructions for how you perform specific data instructions for how you perform specific data collection activities, including use, storage and care collection activities, including use, storage and care of measurement equipment.of measurement equipment.

1616

• Each State’s General Permit should provide specific Each State’s General Permit should provide specific requirements.requirements.

• In Oklahoma, OKR04 Part V lists a number of In Oklahoma, OKR04 Part V lists a number of requirements for “Monitoring, Record Keeping and requirements for “Monitoring, Record Keeping and Reporting”Reporting”

• ““Samples and measurements...shall be representative...”Samples and measurements...shall be representative...”

• ““If lab analysis is conducted it must [use] 40 CFR Part If lab analysis is conducted it must [use] 40 CFR Part 136” [methods]136” [methods]

• ““Monitoring records must include...[names, dates, times, Monitoring records must include...[names, dates, times, place, methods, etc.]”place, methods, etc.]”

• ““Retain records of all monitoring information” (e.g. Retain records of all monitoring information” (e.g. copies of reports, instrument charts, etc.)copies of reports, instrument charts, etc.)

What Is Absolutely Required ?

1717

Are Formal QA Documents Required ?• Your State’s General Permit will state which, if any, QA Your State’s General Permit will state which, if any, QA

documents will be required (e.g. QAPP, DQOs, QMP documents will be required (e.g. QAPP, DQOs, QMP and/or SOPs).and/or SOPs).

• In Oklahoma, OKR04 does not specifically require any In Oklahoma, OKR04 does not specifically require any of these documents.of these documents.

• However, one must consider the “CYA” aspect of QA as However, one must consider the “CYA” aspect of QA as well as what the minimum permit requirements are.well as what the minimum permit requirements are.

• Particularly if your data are to be used in court, for Particularly if your data are to be used in court, for local enforcement or for making regulatory decisions... local enforcement or for making regulatory decisions...

• Or for that matter, just giving yourself credibility to Or for that matter, just giving yourself credibility to your manager, peers, mayor and citizens.your manager, peers, mayor and citizens.

1818

How to Start a Formal QA Program• Managers, Mayors and Councilmen must learn the Managers, Mayors and Councilmen must learn the

importance of good QA’d dataimportance of good QA’d data, and how it will help them , and how it will help them defend their programs.defend their programs.

• Don’t try to do it all at onceDon’t try to do it all at once – expect changes as you go. – expect changes as you go.

• Decide Decide what level of QA you needwhat level of QA you need to apply to each type of to apply to each type of data and activity.data and activity.

• Start with Start with SOPsSOPs and a and a basic QAPPbasic QAPP..

• Seek guidanceSeek guidance from experienced resources. from experienced resources.

• Be comfortableBe comfortable with what you create – stick to basics. with what you create – stick to basics.

• AmendAmend as your Phase II program grows – as your Phase II program grows – QA is dynamicQA is dynamic..

1919

A Good Beginning QA Program• Start applying QA to “hard data” firstStart applying QA to “hard data” first – e.g., chemical data – e.g., chemical data

from field measurements, field test kits and lab data as well from field measurements, field test kits and lab data as well as visual observation data from visual inspections.as visual observation data from visual inspections.

• DQOsDQOs - Decide upon the level of data quality you want for - Decide upon the level of data quality you want for the main types of data you will be collecting. the main types of data you will be collecting. UseUse of data of data..

• SOPsSOPs – identify all routinely repeatable actions and write – identify all routinely repeatable actions and write detailed step by step instructions and list supplies needed.detailed step by step instructions and list supplies needed.

• QAPPQAPP – prepare an umbrella QA plan for all activities – prepare an umbrella QA plan for all activities requiring formal QA attention. requiring formal QA attention.

• Lab and Service contractsLab and Service contracts – include in your contract (and – include in your contract (and QAPP) what technical requirements you want for lab QAPP) what technical requirements you want for lab services and from vendors and consultants that will be services and from vendors and consultants that will be performing technical services (e.g., inspections).performing technical services (e.g., inspections).

2020

• EPA guidance documents:EPA guidance documents:

• http://www.epa.gov/earth1r6/6wq/ecopro/watershd/monitrng/qappsprt/http://www.epa.gov/earth1r6/6wq/ecopro/watershd/monitrng/qappsprt/

• DQO GuidanceDQO Guidance = QA/G-4= QA/G-4

• QAPP GuidanceQAPP Guidance = QA/R-5= QA/R-5

• QMP GuidanceQMP Guidance = QA/R-2= QA/R-2

• SOP GuidanceSOP Guidance = QA/G-6= QA/G-6

• These are not approvable by EPA unless they pertain to These are not approvable by EPA unless they pertain to projects funded by EPA water quality grants.projects funded by EPA water quality grants.

• EPA periodically offers QA training courses.EPA periodically offers QA training courses.

• Contact your Permitting Authority and substates for QA Contact your Permitting Authority and substates for QA advice.advice.

EPA’s QA Resources

Thank you for your Thank you for your infinite patience.infinite patience.

Any Questions ?Any Questions ?