Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University...
Transcript of Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University...
![Page 1: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/1.jpg)
Environmental Monitoring:Database and Beyond
Chengyang ZhangComputer Science Department
University of North Texas
![Page 2: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/2.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 3: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/3.jpg)
Overview
Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 4: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/4.jpg)
System Architecture1
Database Server
Internet
Client Data Browsingand Processing
Base Station Node
Wireless Sensor Nodes
GPRS Modem
Gateway (Single Board Computer)
GPRS Link
Data loggers
1 Jerry’s slide
![Page 5: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/5.jpg)
Current Data Flow
City of Denton FTP
DB serverUV Machine
Greenbelt Station
PullCOD Daemon
PushUV Daemon
Wireless Data Daemon
DBChecker Daemon
Pull/Push
![Page 6: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/6.jpg)
Raw Data Format COD FTP Data
Directory Structure: YEAR/MONTH/cod_date_time 2007/9/cod_2007-9-21_8.30
File Format: Station_Name/Gate_Name, date, time, rain_guage_reading/gate_status RG1, 9/21/2007, 8:30:57 AM, 0.00 HC_N_Gate_STATUS, 9/21/2007, 8:30:57 AM, UP
UV Data Directory Structure: YEAR/MONTH/uv_date_time.txt
2007/9/uv_2007-9-15_12.30 File Format: Date, Time, SUV #1, UVA #2, Temp 1, Temp2
“15.09.2007”, “12:30”, 1.367, 0.000, 32, 0
![Page 7: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/7.jpg)
Raw Data Format(cont.) Wireless Data
Data Transmission Rate every hour (send 4 data packets) can be adjusted
Data Packet Format
Directory Structure: YEAR/MONTH/greenbelt_date_time File Format: Date, time, value #1, value #2, ……
datetime Sensor Data
4 byte 18 * 4 bytes or More
![Page 8: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/8.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange EML SensorML KML
![Page 9: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/9.jpg)
DBMS Selection Free/Open Source v.s. Commercial Database
We choose Free!
PostgreSQL v.s. MySQL We choose PostgreSQL!
Data Integrity Check Server-side Programming Spatial Cartridge Support(PostGIS)
![Page 10: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/10.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange EML SensorML KML
![Page 11: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/11.jpg)
Database Schema Design
Stations
PK StationID
StationName Location Coorinate GaugeType IPAddress Description
Sensors
PK SensorID
FK1 StationID SensorName Manaufacturer SensorType Pheneomenon Input Output Unit ValueRange CollectionFrequency Longitude Latitude CollectionDevice CollectionMethod
*
UVData
PK,FK1 SensorIDPK Date
SUV Temp Rawdata
*
Gate
DateFK1 SensorID
Status
* Rain
DateFK1 SensorID
Value rawdata*Clam
PK Date
clam validation rawdata
*
Humidity
PK,FK1 SensorIDPK Date
Humidity Rawdata
*
Streams
PK,FK1 SensorIDPK Date
StreamData rawdata
*
![Page 12: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/12.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 13: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/13.jpg)
Web Interface (Powered by PHP)
![Page 14: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/14.jpg)
Web Interface (Powered by PHP)
![Page 15: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/15.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 16: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/16.jpg)
Virtual Earth Interface
![Page 17: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/17.jpg)
Google Map Interface (KML)
![Page 18: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/18.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 19: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/19.jpg)
Multi-Layer System Structure*
Telemetry Flood Gates
Water Quality
Sondes Clams Water Quality
Other 11 Stations
Doyle and FreemanNorth Lakes
Pecan Creek WWTPSouth Lakes
Hickory Creek Life Station
Wired SensorsWeather
Soil Moistuire
Greenbelt ParkField Research Station
Wireless Sensor Network
Soil Moisture(200 motes)
Local SensorsUV
Local SensorsOzone
Local Data Logger
EESAT337
In situ DataData ServerEESAT 341
Field UNT Campus
L2 FTP
L3Wireless Modem
Data Collection Gateway
EESAT 341
L1Wireless Modem
LAN LAN Local GIS Data
Repository
Web ServiceEarth Science Data
Physical Data Layer
Logical Data Layer
Classic Web PortalWeb Presentation
Layer
User Layer
Sensor ML / O&M
K-12 Education Outreach
Download Data
(Real Time)
Download Data(non Real
Time)
AnalysisSynthesisModeling
WMSWFSWCS
OGC
Retrieve
Publish
Semantic Layer
Ontology
Semantic Service
Catalog Service
Semantic Web Portal
Semantic Level Query/Analysis
Wireless Sensor Network
Soil Moisture(10 motes)
SOSSASSPS
* From CRI proposal
![Page 20: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/20.jpg)
Data Exchange Motivation
Retrieve data from Web-enabled Sensor Data Repositories Publish our sensor data as web-enabled service
Technology Behind the Scene XML Web Service
![Page 21: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/21.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 22: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/22.jpg)
XML Derivatives
• KML (Keyhole Markup Language )• A markup language for displaying
geographical data in an earth browser*
• Example<kml><Placemark><name>City of Denton: Doyle and Freeman</name><description>Rain Gauge Reading: 0 inch</description><Point><coordinates>-97.132,33.214</coordinates></Point></Placemark></kml>
* http://code.google.com/apis/kml/documentation/index.html
![Page 23: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/23.jpg)
XML Derivatives (Cont.)
• EML (Ecological Metadata Language)• A markup language developed by/for the
ecological discipline*
* http://knb.ecoinformatics.org/software/eml/
eml
packageId: UNTCSE.1.1
system: knb
dataset
title: Greenbelt Wireless Soil Moisture Dataset
creator
individualName
contact
surName:Chengyang
surName: Shu
individualName
![Page 24: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/24.jpg)
XML Derivatives (cont.)
• GML (Geography Markup Language )*
• A markup language defined by the OGC to express geographical fatures
• Example<gml:Polygon><gml:outerBoundaryIs><gml:LinearRing
><gml:coordinates>0,0 100,0 100,100 0,100
0,0</gml:coordinates> </gml:LinearRing> </gml:outerBoundaryIs> </gml:Polygon> <gml:Point>
<gml:coordinates>100,200</gml:coordinates> </gml:Point><gml:LineString> <gml:coordinates>100,200 150,300</gml:coordinates>
</gml:LineString>
* http://en.wikipedia.org/wiki/Geography_Markup_Language
![Page 25: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/25.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 26: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/26.jpg)
XML Derivatives (cont.)
• SensorML (Sensor Model Language )*
• An OGC standard o describe processes of sensor measurements and observations
• discover sensors• obtain sensor information • access sensor observations • receive alerts
* http://vast.uah.edu/SensorML/
![Page 27: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/27.jpg)
SensorML (cont.)
• SensorML (Sensor Model Language )*
* http://vast.uah.edu/SensorML/
![Page 28: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/28.jpg)
O&M
• O&M (Observations & Measurements)*
* www.buyya.com/papers/SensorWebChapter.pdf
![Page 29: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/29.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement(SWE)
![Page 30: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/30.jpg)
Web Services*
Web Services – A program that can be executed through HTTP and XML SOAP (Simple Object Access Protocol)
Request / Response WSDL (Web Service Description Language)
Describes how to connect and query information UDDI (Universal Description Discovery and Integration)
Service Registration And Discovery
* http://en.wikibooks.org/wiki/XML:_Managing_Data_Exchange/Web_services
![Page 31: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/31.jpg)
Web Services(cont.) SOAP Examples
Request<m:SoilMoistureDataRequest xmlns:SOAPSDK1="http://www.unt.edu/cri/xml/wsdl/SoilMositure.wsdl"> <latitude xsi:type="xsd:decimal">33.214</latitude> <longitude xsi:type="xsd:decimal“>-97.132</longitude><startDate xsi:type="xsd:date"></startDate> <numDays xsi:type="xsd:integer">1</numDays> <format>24 Hourly</format> </m:SoilMoistureDataRequest >
Response<SoilMoistureDataResponse xmlns:SOAPSDK1=" http://www.unt.edu/cri/xml/wsdl/SoilMositure.wsdl "> <dwmlByDayOut xsi:type="xsd:string">.....</dwmlByDayOut>
</ SoilMoistureDataResponse >
![Page 32: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/32.jpg)
Web Services(cont.) WSDL Examples
<definitions xmlns:SOAP-ENV=“http://schemas.xmlsoap.org/soap/envelope/” …><message name="SoilMoistureDataRequest"> <part name="latitude" type="xsd:decimal"/> <part name="longitude" type="xsd:decimal" /> <part name="product" type="typens:productType" /> <part name="startTime" type="xsd:dateTime" /> <part name="endTime" type="xsd:dateTime" /> <part name="weatherParameters"
type="typens:weatherParametersType" /> </message> <message name=" SoilMoistureDataResponse "> <part name="dwmlOut" type="xsd:string" />
</message> <service name=“criSoilMoisture"> <port name=“criXMLPort" binding="tns:criXMLBinding"> <soap:address location=" http://www.unt.edu/cri/xml/SOAP_server/criserver.php" />
</port> </service> </definitions>
![Page 33: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/33.jpg)
Overview Data Transmission and Storage
Database Design DBMS Database Schema Design
Data Presentation Traditional Web Interface Virtual Earth / Google Map
Data Exchange XML Derivatives SensorML/O&M Web Services Sensor Web Enablement (SWE)
![Page 34: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/34.jpg)
Sensor Web Enablement*
Decision Support Tools
- vendor neutral- extensive
- flexible- adaptable
Heterogeneous sensor network
In-Situ monitors
Bio/Chem/RadDetectorsSurveillance
Airborne Satellite
- sparse- disparate
- mobile/in-situ- extensible
Models and Simulations
- nested- national, regional, urban- adaptable- data assimilation
Sensor Web Enablement- discovery- access- tasking- alert notification
web services and encodings based on
Open Standards(OGC, ISO, OASIS,
IEEE)
* vast.uah.edu/SensorML/presentations/SWE_SensorML_Botts.ppt
![Page 35: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/35.jpg)
SWE Services SOS (Sensor Observation Services)
Formerly SCS fetch observations Response to queries
WNS (Web Notification Services) asynchronous notification of sensor events two-way
SPS (Sensor Planning Services)
![Page 36: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/36.jpg)
SWE Deployment Database (Spatial Database Preferred) Intermediate Program(Middleware)
that converts sensor schema and readings to SensorML/O&M javabeans etc,
XML Parser SAX etc.
SOAP Toolkit Apache soap toolkit etc.
![Page 37: Environmental Monitoring: Database and Beyond Chengyang Zhang Computer Science Department University of North Texas.](https://reader035.fdocuments.us/reader035/viewer/2022070409/56649e975503460f94b9a71f/html5/thumbnails/37.jpg)
Thank you!