Environmental Monitoring: Database and Beyond
description
Transcript of Environmental Monitoring: Database and Beyond
Environmental Monitoring:Database and Beyond
Chengyang ZhangComputer Science Department
University of North Texas
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
Overview
Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
System Architecture1
Database Server
Internet
Client Data Browsingand Processing
Base Station Node
Wireless Sensor Nodes
GPRS Modem
Gateway (Single Board Computer)
GPRS Link
Data loggers
1 Jerry’s slide
Current Data Flow
City of Denton FTP
DB serverUV Machine
Greenbelt Station
PullCOD Daemon
PushUV Daemon
Wireless Data Daemon
DBChecker Daemon
Pull/Push
Raw Data Format COD FTP Data
Directory Structure: YEAR/MONTH/cod_date_time 2007/9/cod_2007-9-21_8.30
File Format: Station_Name/Gate_Name, date, time, rain_guage_reading/gate_status RG1, 9/21/2007, 8:30:57 AM, 0.00 HC_N_Gate_STATUS, 9/21/2007, 8:30:57 AM, UP
UV Data Directory Structure: YEAR/MONTH/uv_date_time.txt
2007/9/uv_2007-9-15_12.30 File Format: Date, Time, SUV #1, UVA #2, Temp 1, Temp2
“15.09.2007”, “12:30”, 1.367, 0.000, 32, 0
Raw Data Format(cont.) Wireless Data
Data Transmission Rate every hour (send 4 data packets) can be adjusted
Data Packet Format
Directory Structure: YEAR/MONTH/greenbelt_date_time File Format: Date, time, value #1, value #2, ……
datetime Sensor Data
4 byte 18 * 4 bytes or More
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange EML SensorML KML
DBMS Selection Free/Open Source v.s. Commercial Database
We choose Free!
PostgreSQL v.s. MySQL We choose PostgreSQL!
Data Integrity Check Server-side Programming Spatial Cartridge Support(PostGIS)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange EML SensorML KML
Database Schema Design
Stations
PK StationID
StationName Location Coorinate GaugeType IPAddress Description
Sensors
PK SensorID
FK1 StationID SensorName Manaufacturer SensorType Pheneomenon Input Output Unit ValueRange CollectionFrequency Longitude Latitude CollectionDevice CollectionMethod
*
UVData
PK,FK1 SensorIDPK Date
SUV Temp Rawdata
*
Gate
DateFK1 SensorID
Status
* Rain
DateFK1 SensorID
Value rawdata*Clam
PK Date
clam validation rawdata
*
Humidity
PK,FK1 SensorIDPK Date
Humidity Rawdata
*
Streams
PK,FK1 SensorIDPK Date
StreamData rawdata
*
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
Web Interface (Powered by PHP)
Web Interface (Powered by PHP)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
Virtual Earth Interface
Google Map Interface (KML)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
Multi-Layer System Structure*
Telemetry Flood Gates
Water Quality
Sondes Clams Water Quality
Other 11 Stations
Doyle and FreemanNorth Lakes
Pecan Creek WWTPSouth Lakes
Hickory Creek Life Station
Wired SensorsWeather
Soil Moistuire
Greenbelt ParkField Research Station
Wireless Sensor Network
Soil Moisture(200 motes)
Local SensorsUV
Local SensorsOzone
Local Data Logger
EESAT337
In situ DataData ServerEESAT 341
Field UNT Campus
L2 FTP
L3Wireless Modem
Data Collection Gateway
EESAT 341
L1Wireless Modem
LAN LAN Local GIS Data
Repository
Web ServiceEarth Science Data
Physical Data Layer
Logical Data Layer
Classic Web PortalWeb Presentation
Layer
User Layer
Sensor ML / O&M
K-12 Education Outreach
Download Data
(Real Time)
Download Data(non Real
Time)
AnalysisSynthesisModeling
WMSWFSWCS
OGC
Retrieve
Publish
Semantic Layer
Ontology
Semantic Service
Catalog Service
Semantic Web Portal
Semantic Level Query/Analysis
Wireless Sensor Network
Soil Moisture(10 motes)
SOSSASSPS
* From CRI proposal
Data Exchange Motivation
Retrieve data from Web-enabled Sensor Data Repositories Publish our sensor data as web-enabled service
Technology Behind the Scene XML Web Service
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
XML Derivatives
• KML (Keyhole Markup Language )• A markup language for displaying
geographical data in an earth browser*
• Example<kml><Placemark><name>City of Denton: Doyle and Freeman</name><description>Rain Gauge Reading: 0 inch</description><Point><coordinates>-97.132,33.214</coordinates></Point></Placemark></kml>
* http://code.google.com/apis/kml/documentation/index.html
XML Derivatives (Cont.)
• EML (Ecological Metadata Language)• A markup language developed by/for the
ecological discipline*
* http://knb.ecoinformatics.org/software/eml/
eml
packageId: UNTCSE.1.1
system: knb
dataset
title: Greenbelt Wireless Soil Moisture Dataset
creator
individualName
contact
surName:Chengyang
surName: Shu
individualName
XML Derivatives (cont.)
• GML (Geography Markup Language )*
• A markup language defined by the OGC to express geographical fatures
• Example<gml:Polygon><gml:outerBoundaryIs><gml:LinearRing
><gml:coordinates>0,0 100,0 100,100 0,100
0,0</gml:coordinates> </gml:LinearRing> </gml:outerBoundaryIs> </gml:Polygon> <gml:Point>
<gml:coordinates>100,200</gml:coordinates> </gml:Point><gml:LineString> <gml:coordinates>100,200 150,300</gml:coordinates>
</gml:LineString>
* http://en.wikipedia.org/wiki/Geography_Markup_Language
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
XML Derivatives (cont.)
• SensorML (Sensor Model Language )*
• An OGC standard o describe processes of sensor measurements and observations
• discover sensors• obtain sensor information • access sensor observations • receive alerts
* http://vast.uah.edu/SensorML/
SensorML (cont.)
• SensorML (Sensor Model Language )*
* http://vast.uah.edu/SensorML/
O&M
• O&M (Observations & Measurements)*
* www.buyya.com/papers/SensorWebChapter.pdf
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
Web Services*
Web Services – A program that can be executed through HTTP and XML SOAP (Simple Object Access Protocol)
Request / Response WSDL (Web Service Description Language)
Describes how to connect and query information UDDI (Universal Description Discovery and Integration)
Service Registration And Discovery
* http://en.wikibooks.org/wiki/XML:_Managing_Data_Exchange/Web_services
Web Services(cont.) SOAP Examples
Request<m:SoilMoistureDataRequest xmlns:SOAPSDK1="http://www.unt.edu/cri/xml/wsdl/SoilMositure.wsdl"> <latitude xsi:type="xsd:decimal">33.214</latitude> <longitude xsi:type="xsd:decimal“>-97.132</longitude><startDate xsi:type="xsd:date"></startDate> <numDays xsi:type="xsd:integer">1</numDays> <format>24 Hourly</format> </m:SoilMoistureDataRequest >
Response<SoilMoistureDataResponse xmlns:SOAPSDK1=" http://www.unt.edu/cri/xml/wsdl/SoilMositure.wsdl "> <dwmlByDayOut xsi:type="xsd:string">.....</dwmlByDayOut>
</ SoilMoistureDataResponse >
Web Services(cont.) WSDL Examples
<definitions xmlns:SOAP-ENV=“http://schemas.xmlsoap.org/soap/envelope/” …><message name="SoilMoistureDataRequest"> <part name="latitude" type="xsd:decimal"/> <part name="longitude" type="xsd:decimal" /> <part name="product" type="typens:productType" /> <part name="startTime" type="xsd:dateTime" /> <part name="endTime" type="xsd:dateTime" /> <part name="weatherParameters"
type="typens:weatherParametersType" /> </message> <message name=" SoilMoistureDataResponse "> <part name="dwmlOut" type="xsd:string" />
</message> <service name=“criSoilMoisture"> <port name=“criXMLPort" binding="tns:criXMLBinding"> <soap:address location=" http://www.unt.edu/cri/xml/SOAP_server/criserver.php" />
</port> </service> </definitions>
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement (SWE)
Sensor Web Enablement*
Decision Support Tools
- vendor neutral- extensive
- flexible- adaptable
Heterogeneous sensor network
In-Situ monitors
Bio/Chem/RadDetectorsSurveillance
Airborne Satellite
- sparse- disparate
- mobile/in-situ- extensible
Models and Simulations
- nested- national, regional, urban- adaptable- data assimilation
Sensor Web Enablement- discovery- access- tasking- alert notification
web services and encodings based on
Open Standards(OGC, ISO, OASIS,
IEEE)
* vast.uah.edu/SensorML/presentations/SWE_SensorML_Botts.ppt
SWE Services SOS (Sensor Observation Services)
Formerly SCS fetch observations Response to queries
WNS (Web Notification Services) asynchronous notification of sensor events two-way
SPS (Sensor Planning Services)
SWE Deployment Database (Spatial Database Preferred) Intermediate Program(Middleware)
that converts sensor schema and readings to SensorML/O&M javabeans etc,
XML Parser SAX etc.
SOAP Toolkit Apache soap toolkit etc.
Thank you!