Federal Leadership Forum

Three-State Air Quality Studyand

Data Warehouse Project

Cooperative Institute for Research in the AtmosphereColorado State University

Shawn McClure

Groups at CSU Foothills Campus

• Operates under a 5-year renewable Cooperative Agreement (CA) with NOAA

• Complementary CAs with DoI/NPS and DoD/ARL

• 146 scientists, staff and students

• Including 6 Federal NESDIS scientists/4 Federal NPS Scientists

• Including 7 post docs, 21 graduate students, 16 undergraduates

• Including 14 academic faculty (part time)

------------------------------------------------------------------------

• $13M/year in research and outreach funding

• $8M/year from NOAA

CIRA at a glance

CIRA Sponsors

Groups at CSU Foothills Campus• Cooperative Institute for Research in the Atmosphere (CIRA/CSU)• Colorado State University Atmospheric Sciences Department (Richard Johnson)• CSU Atmospheric Chemistry Department (Jeffrey Collett)• National Park Service (NPS) Air Quality Research Group (Bret Schichtel, Bill Malm)• NOAA/NESDIS Regional and Mesoscale Meteorology Branch (RAMMB, Mark DeMaria)• Center for Multiscale Modeling of Atmospheric Processes (CMMAP – NSF, Dave Randall)• Western Governor’s Association (WGA, Tom Moore)• Western Regional Air Partnership (WRAP, Tom Moore)

Other CIRA Partnerships and Collaborations• National Aeronautics and Space Administration (NASA)• Institute for the Environment at the University of North Carolina, Chapel Hill• Sonoma Technologies, Inc.• Air Resource Specialists, Inc. (ARS)• ENVIRON• Multiple Universities

Groups, Partnerships, and Collaborations

Architects, Developers, and Operational Managers for:• The Visibility Information Exchange Web System (VIEWS): http://views.cira.colostate.edu• The WRAP Technical Support System (TSS): http://vista.cira.colostate.edu/tss• The Federal Environmental Database (FED): http://views.cira.colostate.edu/fed• The EPA Air Toxics Data Archive (ATDA): http://vista.cira.colostate.edu/atda

Principal Investigators for:• The NASA Satellite Data Extension to VIEWS/TSS (beta): http://views.cira.colostate.edu/nasa• The NAAPS Data Extension to VIEWS/TSS: http://views.cira.colostate.edu/web/projects/naaps.aspx• The EPA Ozone Data Pilot Project: http://views.cira.colostate.edu/web/projects/epaozone.aspx

Other Active Projects and Collaborations:• The Emissions and Air Quality Modeling Data Management System for the Southeastern States Air

Resource Managers (SESARM): (URL pending)• Operational exemplar (VIEWS/TSS) for the Group on Earth Observations’ (GEO) Global Earth

Observation System of Systems (GEOSS): (URL pending)• EPA’s Cyber-Infrastructure (CyAir) project: http://www.cyair.net/

Development Consultation and Operational Hosting for:• The WRAP Emissions Data Management System: http://www.wrapedms.org• The WRAP Fire Emissions Tracking System: http://wrapfets.org

Systems, Projects, and Collaborations

VIEWS/TSS Data and Processing Flowchart

Modeling Data and Processing Flowchart

High-level Architecture of our System of Systems

Database and Data Warehouse Management

Data Import and Integration

VIEWS/TSS Decision Support Tools

VIEWS Website Traffic and Usage

Data Management: Extracting and Filtering

Data Management: Transforming and Reformatting

Data Management: Structuring and Reconciling

Data Management: Derived and Intermediate Products

• Uniformly formatted netCDF files– Optimized for long term storage– More easily consumed by end users– Facilitates interoperability and standardization– Better enables derived product generation

• Static graphical images– Can be used by a wide variety of visualization tools– Transparency of *.PNG images is ideal for Google and Bing Maps– Enables quick browsing and identification of air quality events– Enables smooth client-side animations and searching

• KML and KMZ files– Makes it possible to store feature and topology data– Can be used by current and popular visualization tools– Enables direct data and metadata query– Facilitates sharing and interoperability

“Interoperability”: The ability of diverse systems and organizations to work together to exchange and utilize information.

To facilitate this, VIEWS has been designed with the following services and features:

• Discovery, retrieval, and exchange of data and metadata

• On-the-fly transformation and formatting of data and metadata

• Upload and management of user-supplied data and metadata

• Generation of visualization and analysis products

• Availability of “embeddable” components and tools

• Support of OGC web standards for spatial data exchange

Data Management: Standards and Interoperability

Interoperability: Enabling Systems of Systems

VIEWS/TSS Decision Support Tools

Metadata Exploration: Dataset Inventory

Metadata Exploration: Monitoring Site Information

Exceptional event metadata (such as fires) can be stored and dynamically associated with data, photos, and imagery at run time in order to provide quick cross-referencing and discovery of related data. Thus, satellite and modeled data can be quickly associated with aerosol events once these linkages are complete.

“Bisquit” fire impacting Crater Lake in 2002

Metadata Exploration: Air Quality Events

Data Exploration: File Explorer tool

Data Exploration: Aerosol Composition Analysis

Data Exploration: Image Browser and Animation Tool

Data Exploration: Dataset Timeline and Parameter Inventory

Mapping: CAMx Ozone from CIRA – April 20, 2006

Mapping: CMAQ Ozone from UNC – April 20, 2006

Mapping: Aura/OMI Ozone Column – 4/20/2006

Mapping: Aura/OMI Radiative Cloud Fraction

Mapping: IMPROVE Ammonium Sulfate

Mapping: Three layers combined

Mapping: Google Earth

Comments and Recommendations

• Expect greater complexity (than you do right now)

• Ensure that software engineers lead the Data Warehouse development (not scientists)

• Leverage existing work (e.g. WestJump AQMS, WRAP work, etc.)

• Modularize functional areas

– Modeling

– Data Warehouse

– Configuration Management (versioning, lineage, pedigree, and change tracking)

– End User Tools

• Create a “data broker” position to manage data flows between functional areas

• Buffer modelers and system developers from decisions about data

• Organize an approach to disagreements

• Don’t let the perfect become the enemy of the good

• Plan for delays

Thanks.VIEWS website: http://views.cira.colostate.edu

TSS website: http://vista.cira.colostate.edu/tss

Shawn McClure: mcclure@cira.colostate.edu