OOI CyberInfrastructure Common Operating...
Transcript of OOI CyberInfrastructure Common Operating...
OOI CyberInfrastructureCommon Operating Infrastructure
• CI
Overview
• Context of COI• Usage scenario• Work plan• Early Results• Logical Architecture• Deployment Architecture• Prototype
Scenario• A scientist is trying to setup up a
facility out of resources (instruments, computing capabilities, storage) spread out over a variety of authority domains
• Challenges–– Resource discovery Resource discovery
(instruments, storage, (instruments, storage, computation)computation)
–– Resource access (seamlessly Resource access (seamlessly across infrastructure)across infrastructure)
–– Resource lifecycle Resource lifecycle (adding/removing an (adding/removing an instrument, ...)instrument, ...)
–– Authentication, authorization, Authentication, authorization, and other policies, and other policies,
–– GovernanceGovernance–– Capability presentationCapability presentation
COI
Instruments Storage Computing
Various Authority Domains User
Legend
ProvidedCapabilities
• COI–– integration platform, integration platform, orchestratororchestrator, communication conduit, communication conduit–– handles crosshandles cross--cutting issues including identity / policy / governance / cutting issues including identity / policy / governance /
resource management => focus of Y1 activitiesresource management => focus of Y1 activities
Development Plan
Date Deliverable StateSep 30 Requirements Document New
Oct 31
Domain Model (AD) NewStandards (SSEP) NewRequirements Document RevisedCOI Prototype with data streams and basic instrument control for MBARI instruments New
Nov 30 Requirements Document RevisedDomain Model Revised
Dec 31 Report on identified Interaction Patterns NewLanguage for defining policies (AD) New
Jan 31
Policy authoring tool (Prototype) NewInteraction Patterns UpdatedLanguage (AD) UpdatedRequirements document (RD) RevisedArchitecture Document (AD) Revised
Feb 27
Software Mapping (SSEP) NewPolicy authoring tool (Prototype) UpdatedRequirements document (RD) RevisedArchitecture Document (AD) Revised
• Calit2 – Model and design the COI Architecture
• NCSA – Policy/Identity federation
• NCSU – Policy/Governance• MBARI – Instruments &
Infrastructure
• Tasks1.1. Requirements ElicitationRequirements Elicitation2.2. Standards AnalysisStandards Analysis3.3. Domain ModelingDomain Modeling4.4. Architecture and DesignArchitecture and Design5.5. Integration and PrototypingIntegration and Prototyping6.6. Documentation and PrototypingDocumentation and Prototyping
Development Plan
• Calit2 – Model and design the COI Architecture
• NCSA – Policy/Identity federation
• NCSU – Policy/Governance• MBARI – Instruments &
Infrastructure
• Tasks1. Requirements Elicitation2. Standards Analysis3. Domain Modeling4. Architecture and Design5. Integration and Prototyping6. Documentation and Prototyping
Release 1• Federated Facility Services• Enterprise Service Bus & Container
• Presentation Framework• Governance Framework• Service Framework• Resource Framework• Distributed State Management• Communication Infrastructure
• Identity Management Services• Resource Lifecycle Services• Resource Catalog and Repository
Services
Release 2• Federated Facility Services• Enterprise Service Bus & Container• Resource Activation Services• Resource Collaboration Services
Technologies
• ESB infrastructure–– MULEMULE–– Hibernate, Spring, Groovy, JMXHibernate, Spring, Groovy, JMX
• Messaging–– ActiveMQActiveMQ/AMQP/AMQP
• Policy/Governance–– GridShibGridShib, , GridGroupperGridGroupper, , myVocsmyVocs
• Leverage existing –– BIRN/BIRN/TelescienceTelescience ATOMICATOMIC–– SDSC Storage Resource Broker (SRB)SDSC Storage Resource Broker (SRB)
Early Results• Results informed by the following models:
–– IEEE1451 IEEE1451 –– Instrument models, relationship with Instrument models, relationship with SensorMLSensorML, , TransducerMLTransducerML
–– ebBPebBP -- Business transaction modelsBusiness transaction models–– Liberty Alliance Liberty Alliance –– Identity managementIdentity management–– SAML SAML –– Identity managementIdentity management–– ……
• Domain models created:–– DoDAF OVDoDAF OV--2, OV2, OV--77–– Focus currently on policy & governance aspectsFocus currently on policy & governance aspects
• Interaction patterns considered:–– Liberty Alliance identity federationLiberty Alliance identity federation
• Early prototype development
CyberPoP Software Platform
Testing prototype:• Hardware: Dual
Xeon• Main OS: Linux• VM: Vmware Server• VM Guest OS: Linux
Security - Authentication and Authorization
ESB BackboneEnd-to-End Data Transformation
Web Portal BPEL Web
ServicesJ2EE/EJB/
Servlet SAP IBM AS400
JBI (JSR-208)
File/FTP/SFTP
JMS, MQ Series,
ActiveMQTCP,
MCAST, SSL
Caching (Distrib.)
Frameworks(Spring)
GRID,JavaSpace
E-CommEmail, IM
Software configuration:• ESB: MULE• Framework: Spring• Messaging: ActiveMQ/AMQP• History: MySQL + JDBC• Feeds: RSS• Visualization: HTML + JavaScript +
Google Maps API
Thank you
From Requirements & State of the Art technologies to CI
From Requirements & State of the Art technologies to CI
From Requirements & State of the Art technologies to CI
From Requirements & State of the Art technologies to CI
From Requirements & State of the Art technologies to CI
Resource
Subject AttributeAttribute
Value Assertion
**
Property
Attribute Authority Attribute
Authority Policy
* Policy
Consent Policy
protects
Responsible Organization
Identity Service
Attribute Authority Manager
*Mapping
configures
configures
Principaldefines
*Rule Context ref
User
Client Application
accessesClient Attributes Requirements
Declaration
Deployment Manager
edits
Developer
creates
ref
reads
ref
Privacy Auditor
checks
gets attributes from
Liberty Alliance
From Requirements & State of the Art technologies to CI
Resource
Subject AttributeAttribute
Value Assertion
**
Property
Attribute Authority Attribute
Authority Policy
* Policy
Consent Policy
protects
Responsible Organization
Identity Service
Attribute Authority Manager
*Mapping
configures
configures
Principaldefines
*Rule Context ref
User
Client Application
accessesClient Attributes Requirements
Declaration
Deployment Manager
edits
Developer
creates
ref
reads
ref
Privacy Auditor
checks
gets attributes from
Liberty Alliance
OOI CI Model (simplified)
InstrumentInterface
Serv
ice/
Dat
a C
onne
ctor
Cyb
er-
Infr
astr
uctu
re
CO
IM
ess
agin
g,
Da
ta D
ist,
Po
licy
Dat
a/St
orag
eIn
terfa
ce
Computation/GridInterface
Con
trol
Mod
elin
g
Inst
rum
ent
Dat
a
CE
IPr
oces
sing
InstrumentInterface
Serv
ice/
Dat
a C
onne
ctor
Cyb
er-
Infr
astr
uctu
re
CO
IM
ess
agin
g,
Da
ta D
ist,
Po
licy
Dat
a/St
orag
eIn
terfa
ce
Computation/GridInterface
Con
trol
Mod
elin
g
Inst
rum
ent
Dat
a
CE
IPr
oces
sing
InstrumentInterface
Serv
ice/
Dat
a C
onne
ctor
Cyb
er-
Infr
astr
uctu
re
CO
IM
ess
agin
g,
Da
ta D
ist,
Po
licy
Dat
a/St
orag
eIn
terfa
ce
Computation/GridInterface
Con
trol
Mod
elin
g
Inst
rum
ent
Dat
a
CE
IPr
oces
sing
InstrumentInterface
Serv
ice/
Dat
a C
onne
ctor
Cyb
er-
Infr
astr
uctu
re
CO
IM
ess
agin
g,
Da
ta D
ist,
Po
licy
Dat
a/St
orag
eIn
terfa
ce
Computation/GridInterface
Con
trol
Mod
elin
g
Inst
rum
ent
Dat
a
CE
IPr
oces
sing
OOI CI Model
Inst
rum
ent
Inte
rface
Service/Data Connector
Cyber-Infrastructure
COIMessaging, Data Dist, Policy
Data/StorageInterface
Com
puta
tion/
Grid
Inte
rfaceControlModeling
Instrument Data
CEIProcessing
CO
I Interaction & M
ediation
Process Definition
Observation R
equest
Process Definition
Data R
epresentation
CO
I Interaction & Mediation
CO
I Interaction & M
ediation
CO
I Interaction & Mediation
CO
I Interaction & M
ediation
CO
I Interaction & Mediation
Service Agreement
ProposalProcessing Status,
Data Product
Service Agreement
Proposal,Process D
efinition
Data Product,Process
Ontology
Registration, C
omm
unication
COI Detailed needlines and relationships
COI Services Network (internal details)
GPS Clock O10Policy Validator O9
Policy Enforcement O3Identity Management O1
Authentication O2 State Management O4 Logging O6
Router/Interceptor O7
Messenger/Communicator O8
ID Inform
ation O
N1, O
N2
Authentication O
N3, O
N4
Policy, E
vent O
N5, O
N6
State, E
ventO
N7, O
N8
Event ON
11
Time
ON
16, ON
17
Policy
ON
14, ON
15
Instrument SN I0Data SN D0 Modeling SN M0Control SN C0Processing SN P0
Science User/Operator Interface U0
MessageON12, ON13
Service/Data Connector O13
Message ON22, ON23
ON32, ON33
ON24, ON25
ON26, ON27ON28, ON29
ON30, ON31
ON34,ON35 Coastal-Global Scale Node A2 (*)
Regional Scale Node A1 (*)
Research Laboratory A4 (*)
Classroom Facility A5 (*)
Governance O5
Service Registry O12
CEI SN E0
ON36, ON37
Service ID
O
N20, O
N21
Conversation
ON
9, ON
10
Registration, C
omm
unication
CO
I Interaction & M
ediation
AN4, AN9
AN3, AN8
AN2, AN7
Inst
rum
ent
Inte
rface
Service/Data Connector
Cyber-Infrastructure
COIMessaging, Data Dist, Policy
Data/StorageInterface
Com
puta
tion/
Grid
Inte
rfaceControlModeling
Instrument Data
CEIProcessing
Deployment Strategy
Service/Data Connector
CI Core
COIMessaging, Data Dist, Policy
Data/StorageInterface
ControlModeling
Instrument Data
CEIProcessing
Compute Cloud
Data Storage Cloud
Coastal/Global Scale Node
SDC
Regional Scale Node
SDC
Other Observatories/Laboratories
SDC
Science Portals and Applications
OOI Domain Model Overview• Focus is on
Process and Control Networks
• Modeling, Data and Instrument Networks are summarized
Source figure: Exhibit 5 of Ov7
<<resource>>Instrument
<<policy enforcer>>Instrument Proxy
1
1
<<resource>>Process Instancecommunicate via
communicate
*
<<resource>><<policy enforcer>>
Communication Infrastructure
*
<<resource>><<policy enforcer>>
Execution Engine
executes
<<resource>>Process Definition
reads
1
*
*
*
<<resource>>Computation
Node
runs*
*
Definition Languageis specified in
understands
1
1
specifies
CEI Services Network
<<policy enforcer>>Computation
Scheduler
setup
Observation Plan
<<policy enforcer>>Resource Planner
*1
Observation Request
receives
creates
*
1
Science Ontology
expressed in
<<policy enforcer>>Instrument Plannersetup
Instrument Services Network
Service AgreementProposal
Resource Setup Protocol
plays consumer
exchanges
plays provider plays provider
Control Services Network
Modeling Services Network
Interaction Role
*
*
plays
constraints
<<resource>>Interaction
Specification
*
*
Communication Channel
*
2..*
communicates over
*
Message Sequence
constraints
1
<<resource>>Message
*Data Message
Command Message
Engineering Data Message
Science Data Message
Data Product Message
Raw Data Message
COI-CoreData Services Network
Interaction
*
1..*
<<policy enforcer>>Dispatcher
<<policy enforcer>>Process Controller
Process Plan
Process Status
<<resource>>Communication
Facility
*
produces
consumes
consumes
produces
consumes
provides
*
Process Services Network
<<policy enforcer>>Data Planner
plays provider
Policy Model
• Informed by Shibboleth/ GridShib, Java Security Model, Liberty Alliance Models, …Source figure: Exhibit 10 of Ov7
Policy
Authentication AuthorizationAuthentication Controller
Interaction Role
Shibboleth Authentication
Controller
Principal Authorization Role
*
Authorization Controller
Shibboleth Authorization
Controller
plays controller plays
* *
Permission
Credential *
*
*
*
Tokenissues
<<resource>>
*
Capabilityaccess
Scope
1
1..*
*
Authentication Domain
1..*
playsprincipal
authenticated by
authenticateplays
1..*
<<policy enforcer>>
check
Governance ModelSource figure: Exhibit 9 of Ov7
Process Model
• COI will provide support to CEI to implement the Process Model
Source figure: Exhibit 11 of Ov7
<<resource>>Process Instance
<<resource>>Communication
Facilitycommunicate via
**
<<resource>><<policy enforcer>>
Communication Infrastructure
*
<<resource>><<policy enforcer>>
Execution Engine
<<resource>>Process Definition
reads
1
*
*
*
<<resource>>Computation
Node
runs
**
Definition Language
is specified in
understands
1
1specifies
<<policy enforcer>>Computation
Scheduler
delegates
Resource Setup Protocol
plays provider
Interaction Role
*
1..*
plays
<<resource>>Interaction
Specification*
Interaction
Setup
Engineering Data Message
produces
<<resource>>Message
Data Product Message
produces
consumestransmits
Communication Setup Strategy
creates
setup
Model Process Repository
Instrument Process
Repository
Process Definition
Repositoryqueries
queries
queries
Repository
Kepler BRTT Antelope / Unix Java VM ISI Pegasus Matlab
<<RIS>>Policy
*
1..*
emerges
Virtualization Engine:
Xen, Vmware, …
*
<<policy enforcer>>Dispatcher
<<policy enforcer>>Process Controller
Process Plan
Process Status
consumes
consumes
produces
consumes
*
executes
uses
1..*
Logical Model for PrototypeSe
rvic
e/D
ata
Con
nect
or
CTD
/Fl
orom
eter
Con
nect
or
Deployment Model for Prototype