Instruments and Sensors as Grid Services Donald F. (Rick) McMullen 1 Kenneth Chiu 2 John C. Huffman...
-
Upload
david-mckenna -
Category
Documents
-
view
214 -
download
2
Transcript of Instruments and Sensors as Grid Services Donald F. (Rick) McMullen 1 Kenneth Chiu 2 John C. Huffman...
Instruments and Sensors as Grid Services
Donald F. (Rick) McMullen1
Kenneth Chiu2
John C. Huffman1
Kia Huffman1
Randall Bramley1
1Indiana University2SUNY Binghamton
2
Motivations
• Instruments and sensors are not well integrated into grids
• Data is acquired, processed and stored before it “hits the grid”.
• Need methodology for interacting with instruments and sensors in real time from grid applications
• Some abstraction of sensor and instrument functionality is needed to make grid applications that use them more robust and flexible.
3
Goals
• Integrate instruments and sensors (e.g. real-time data sources) into a Grid computing environment with Grid services interfaces.
• Abstract instrument capabilities and functions to reduce data acquisition and analysis applications’ dependence on specialized knowledge about particular instruments.
• Move production of metadata as close to instruments as possible and facilitate the automatic production of metadata.
• Develop a standard, reusable methodology for “grid enabling” instruments.
• Collaborate with scientists in academia and industry in a broad range of disciplines who either develop instruments or whose work depends on the details of using them.
4
Increasing number of sensors
Increasingbandwidth per sensor
Increasing real-time application
X-Ray Crystallography
Electron Microscope
Radio Telescope
Ground motion sensor array
Traffic sensors
Wireless ‘mote’sensor network
• Simple 2-D analysis of instrument taxonomy• At lease five dimensions identified in more detailed analysis• Project must address enough points (classes) to assure breadth of applicability
5
Initial Applications• High brilliance X-ray crystallography
– Large instrument application– Deeply integrated into bio and medical discovery research methodology– Mature analysis software and large user community
• Robotic telescopes– Small numbers of sensors: CCD, environmental; some control aspects:
filters, aiming, dome.– Global coordination needed for scheduling– Aggregation of disparate sensors into a “composite”
instrument• Small sensors
– Minimal memory and CPU– Wireless connectivity– Developing parallel project to use ad hoc/swarm networks for data
collection for real-time simulation and prediction– Updating of data flows in response to sensor/network reconfiguration.
6
CIMA implementation targets
MICA Mote wireless sensor/controller board
PC104 industrialcontroller board
Synchrotron beamline
(APS/ALS)
Large scientific instruments
Embedded sensorsand controllers
verylarge
systems,few
elements
verysmall
systems,many
elements
7
MMSF Automated Telescope
• Typical remote access automated observatory
• System has 33 distinct “sensors”, 12 controllers– Open/close of roof based on a
Polaris transparency monitor and rain detector – simple grid of wires detecting rain drops
– Telescope direction and dome control
– Filter selection and telescope focus
– Liquid nitrogen fills of the CCD dewar jars ….
8
MMSF Observatory Features
• Instruments producing data without units– Temperature, humidity cutoffs determined empirically
as resistances, not degrees or %
• Hierarchical and co-located instruments– Single platform holds three instruments, so orienting
one changes orientation of all
• Updates to equipment occurs frequently• Data transferred via 28k modem line –
middleware needs to work locally, directly between instruments and sensors
9
Observatory Data Architecture• Control Data
– object control – instrument control
• Object Data (i.e., object of scientific interest)– Full spectrum from raw unitless data to derived data
artifacts• Instrument package
– system package data (multiple attributes output)– system sensor data (single attribute output)– nonsystem sensor data (weather data from NOAA)– calibration data– access protocols
10
X-Ray Crystallography
InstrumentServices
PortalInstrumentManager
DataArchive
WAN LAN
LAN
Non-grid service
Grid service
Persistent
Non-persistent
Proxy Box
11
LTER
• Automatic updating of flows• Provenance• SOAP/ARTS (Antelope Real-Time System)
Data-logger
QAAgent
Env.Agent
QAAgent
Oracle
WebServer
WebBrowser
Config. Event (CIMA)
Other Connection
Trout Lake Station
University ofWisconsin Campus
Buoy
ORB
ARTS Connection
Sensors
Env. Event (CIMA)
JDBC/ODBC Connection
WebBrowser
Config.Agent
Config.Agent
Config.Agent
Config.Agent
1
2
3
3
4
Other Locations
12
Further applications currently being explored
Robotic telescopes- Bradford robotic telescope, Oxenhope observatory,Faulkes telescope project
Electron Microscopy-U. Queensland EM group – regional scale, multiple instruments - KISTI 2nd largest EM (after Osaka)
Industrial monitoring and control, e.g. Train axles- Ore trains – Km long, derailment is very expensive to fix- Temperature sensors on axles monitor bearing status, anticipate wheel failure
Environmental monitoring- Water use- contaminants
13
Architecture
14
Common Instrument Middleware Architecture (CIMA) Elements
• Schema for instrument functionality (and ontology for schema attributes);
• Data model for representing instrument metrics and calibration;• A small, high performance, embeddable Web Services stack, initially in
Java, including Proteus support for multi-protocol, multimodal transport;
• Service implementation for accessing the instrument’s functionality and metrics via the Proteus-mediated interface;– Ability to dynamically insert new protocols into running instances
• OGSA and WS-RF compliant functions to register with a location service, authenticate users, provide access control to instrument controls and data, send and receive events, and co-schedule the instrument into a Grid computing and storage context.
15
Overview
Physical Network Transport
Data Pipeline
AcquisitionComponent
AcquisitionCode
InstrumentAccess
AnalysisComponent
AnalysisCode
InstrumentAccess
CurationComponent
CurationCode
InstrumentAccess
Instrument
Sensor 1
Controller
Sensor 2
InstrumentPresentation
Scientist
InstrumentAccess
RemoteAccess
GUI
Device-Independent Application Module
Device-Dependent Virtualization Module
Shared Implementation
16
InstrumentHardware
Controller
ServiceImplementation
InstrumentAccess
InstrumentPresentation
OGSI
Proteus
SOAPBinaryFormat
TCP/IPOther Network
Protocols
Instrument-Specific CIMA
Instrument-Independent CIMA
Existing Instrument Code
17
Example CIMA minimal knowledge bootstrap procedure
Application CIMA instrument
“send description”
“RDF description”
Proteus/SOAP calls
Application parses description for ports to
read for calibration and voltage
“read calibration port”
“calibration”
“read thermocouple port”
“thermocouple voltage”
Service Implementation (SI) returnsdescription of itself and instrument
SI returnsstored calibration
SI calls controller function to read and return voltage
Application reads thermocouple voltage
then computes and displays a
temperature
Globus init, user authentication, and instrument lookup
18
Instrument
Services• Instrument
– get– set
• Sensor– get– set
• ChannelSource– get– set– register
• ChannelSink– receive– get– set
Sensor
ChannelSource
ChannelSource
Sensor
SensorChannelSource
ChannelSource
ChannelSink
19
Parcels
• Wish to unify our data models, etc.• Toolkit must be application-independent as
much as possible.• Attributes
– Type (string)– Globally unique ID (string)– Encoding
• CDATA, Binary, ASCII, Base64– Location
• Inline, URL, Other
• Parcel Sets– Special data used for connectivity information.
20
Technologies
21
Technologies
• Web services– XML, SOAP, WSDL, binary XML
• Grid services– OGSA/OGSI, WS-RF, DAIS
• Axis C++ gSOAP
• Proteus (SC 2002)
• XBS (HPC 2004)
• Schema-specific parsing
22
Proteus Motivation
• Web Services for scientific computing– SOAP performs well as a lingua franca
• But suffers from performance problems for scientific data
– Solution: establish initial communication with SOAP, and then switch to a faster protocol.
• Grid intermediaries
23
Proteus Overview
• Provides multiprotocol RMI system to applications– Can wrap existing protocol implementations with dynamic
invocation
• Facilitates use of SOAP as common language– Switch to faster protocols if supported by both sides.
• Mediates between protocol providers and applications– Applications use Proteus client API– Providers use Proteus provider API
• Allows a new provider to be added (at run-time) without changing application
• Generic serialization/deserialization allows marshalling code to be reused for multiple protocols
24
Multiprotocol
Network
Proteus
Client
ProviderA
ProviderB
Proteus
Client
ProviderA
ProviderB
Protocol A
Protocol B
Process 1 Process 2
25
Proteus
Protocol A Provider
Application
Protocol B Provider
Proteus APIs
Client API
Provider API
26
Schema-Specific Parsing
• XML processing stages (conceptual)– Well-formedness
• Lexical and syntactic, defined by core XML specification.
– Validity• Conformance to a schema, mainly structural
– Application
27
Merging Stages
Application
Validation
Well-formedness
Application
Validation
Well-formedness
Application
Validation
Well-formedness
User written
Merged
Fully articulated Implicitly validated Schema-specific parsing
28
Compiler-Based Approach
• Front-end parses schema into intermediate representation.
• Back-end generates code from intermediate representation.
• Intermediate representation is a generalized automata.
XMLSchema
IR
RELAXNG
C (fast)
C(low-pow)
Java
29
Summary
• Create standards for accessing broad spectrum instruments and sensors.
• Incompatible components should still have some base level of interoperability.