Grid and Collaboration work at Community Grids Laboratory
description
Transcript of Grid and Collaboration work at Community Grids Laboratory
Grid and Collaborationwork at Community Grids
LaboratoryAFRL DaytonAugust 4 2004
Geoffrey FoxIndiana University
Bit levelInternet
ServiceInternet
Layered Architecture for Web Services and Grids
Application Specific GridsGenerally Useful Services and Grids
Workflow WSFL/BPELService Management XGSP
Service Discovery (UDDI) / InformationService Internet Transport Protocol
Service Interfaces WSDL
Base Hosting EnvironmentProtocol HTTP FTP DNS …
Presentation XDR …Session SSH …
Transport TCP UDP …Network IP …
Data Link / Physical
ServiceContext
HigherLevelServices
XGSP Web Service MCU Architecture
SIP H323 Access Grid Native XGSPAdmire
Gateways convert to uniform XGSP Messaging
High Performance (RTP)and XML/SOAP and ..
Media ServersFilters
Session ServerXGSP-based Control
NaradaBrokeringAll Messaging
Use Multiple Media servers to scale to many codecs and manyversions of audio/video mixing
NB Scales asdistributed
WebServices
NaradaBrokering
Global-MMCS Community Grid We are building an open source protocol independent Web
Service “MCU” which will scale to an arbitrary number of users and provide integrated thousands of simultaneous users collaboration services.
The function of A/V media server is distributed using NaradaBrokering architecture.• Media Servers mix and convert A/V streams
Open XGSP MCU based on the following open source projects• openh323 is basis of H323 Gateway• NIST SIP stack is basis of SIP Gateway• NaradaBrokering is open source messaging• Java Media Framework basis of Media Servers• Helix Community http://www.helixcommunity.org for Real
Media http://www.globalmmcs.org open source “non advertised”
release
Break up into Web Services Monolithic MCU becomes many different “Simple Services”
• Session Control• Thumbnail “image” grabber• Audio Mixer• Video Mixer• Codec Conversion• Helix Real Streaming• PDA Conversion• H323/SIP Gateways
As independent can replicate particular services as needed• Codec conversion might require 20 services for 20 streams
spread over 5 machines 1000 simultaneous users could require:
• 1 session controller, 1 audio mixer, 10 video mixers, 20 codec converters, 2 PDA converters and 20 NaradaBrokers
Support with a stream optimized Grid Farm in the sky• Future billion way “Video over IP” serving 3G Phones and home media
centers/TV’s could require a lot of computing
GlobalMMCS and NaradaBrokering All communication – both control and “binary” codecs are
handled by NaradaBrokering Control uses SOAP and codecs use RTP transport Each stream is regarded as a “topic” for NB Each RTP packet from this stream is regarded as an “event” for
this topic Can use replay and persistency support in NB to support
archiving and late clients Can build customized stream management to administer replay,
and who gets what stream in what codec NaradaBrokering supports unicast and multicast Use firewall penetration and network monitoring services in NB
to improve Q0S
NaradaBrokering can Support• Service level Internet with Software Overlay (ad-hoc)
Network• Virtualize inter-service communication• Federate different Grids Together• Scalable pervasive audio-video conferencing – “Video
over IP”• General collaborative Applications and Web services • Build next generation clients interacting with messages
not method-based user interrupts (Message-based MVC)• Unify peer-to-peer networks and Grids• Handle streams as in “media or sensor Grids”• Handle events as in WS-Notification• Agent protocols as XML Messaging (Semantic Web)
Minicomputer
Firewall
ComputerServer
PDA
Modem
Laptop computerWorkstationPeers
Peers
Audio/VideoConferencing Client
Audio/VideoConferencing Client
NaradaBrokering BrokerNetwork
NaradaBrokering
Web Service B
Stream
Server-enhanced Messaging
NB supports messagesand streams
Queues
Multiple transport supportIn publish-subscribeParadigm with differentProtocols on each link
Transport protocols supported include TCP, Parallel TCP streams, UDP, Multicast, SSL, HTTP and HTTPS.Communications through authenticating proxies/firewalls & NATs. Network QoS based Routing
Subscription Formats Subscription can be Strings, Integers, XPath queries, Regular Expressions, SQL and tag=value pairs.
Reliable delivery Robust and exactly-once delivery of messages in presence of failures
Ordered delivery Producer Order and Total Order over a message type Time Ordered delivery using Grid-wide NTP based absolute time
Recovery and Replay Recovery from failures and disconnects.Replay of events/messages at any time.
Security Message-level WS-Security compatible security
Message Payload options
Compression and Decompression of payloadsFragmentation a nd Coalescing of payloads
Messaging Related Compliance
Java Message Service (JMS) 1.0.2b compliant Support for routing P2P JXTA interactions.
Grid Application Support NaradaBrokering enhanced Grid-FTP. Bridge to the Globus TK3.
Web Service reliability Prototype implementation of WS-ReliableMessaging
Current NaradaBrokering Features
NaradaBrokering Service IntegrationS1 S2P2P1
S1 S2
S1 S2
Proxy Messaging
Handler Messaging
Notification
S? Service P? Proxy
NB Transport
Internal to Service: SOAP Handlers/Extensions/Plug-ins Java (JAX-RPC) .NET Indigo and special cases: PDA's gSOAP, Axis C++
Standard SOAP Transport
Any Transport
Virtualizing Communication Communication specified in terms of user goal and Quality of
Service – not in choice of port number and protocol Protocols have become overloaded e.g. MUST use UDP for A/V
latency requirements but CAN’t use UDP as firewall will not support ………
A given communication can involve multiple transport protocols and multiple destinations – the latter possibly determined dynamically
Dial-upFilter
SatelliteUDP
FirewallHTTP
A
B1
Hand-HeldProtocol
FastLink
Software MulticastB2
B3NB Broker Client Filtering
NB Brokers
Performance Monitoring Every broker incorporates a Monitoring service that
monitors links originating from the node. Every link measures and exposes a set of metrics
• Average delays, jitters, loss rates, throughput. Individual links can disable measurements for
individual or the entire set of metrics. Measurement
intervals can also be varied
Monitoring Service, returns measured metrics to Performance Aggregator.
BrokerNode
LinkData
BrokerNode
LinkData
Performance AggregationService
Control MessageExchange
Aggregates infofrom nodes in acertain domain
MonitoringService
Pentium-3, 1GHz, 256 MB RAM100 Mbps LAN
JRE 1.3 Linux
hop-3
0
1
2
3
4
5
6
7
8
9
100 1000
Tra
nsit
Del
ay
(Mill
isec
onds
)
Message Payload Size (Bytes)
Mean transit delay for message samples in NaradaBrokering: Different communication hops
hop-2
hop-5 hop-7
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
1000 1500 2000 2500 3000 3500 4000 4500 5000
Sta
nd
ard
De
via
tion
(M
illis
eco
nd
s)
Message Payload Size (Bytes)
Standard Deviation for message samples in NaradaBrokering Different communication hops - Internal Machines
hop-2hop-3hop-5hop-7
NaradaBrokering and JMS (Java Message Service)
Low Rate; Small Messages (commercial JMS)
NaradaBrokering and JXTA Federation
Based on hybrid proxy that acts as both Rendezvous peer (JXTA routers) and NaradaBrokering end-point.
No changes to JXTA core or constraints on interactions• Change made to Rendezvous
layer Peers are not aware that
they interact with a Narada-JXTA proxy or Rendezvous peer.
NARADA-JXTA proxy
NARADAbroker cloud
Peers
JXTARendezvousPEER
Dynamic/fluidpeer groups
High end "long lived"/persistent resources
NB provides JXTA guaranteed long distance delivery NB federates multiple JXTA Peer Groups
Based on Message Level SecurityMessages organized into topicsEach topic has a separate key; Topics can be organized into sessions
Functionality I WebSphere MQ
(formerly MQSeries)Pastry NaradaBrokering
Maximum number of nodes hosting the messaging infrastructure
Medium (MQ is based on the point-to-point model. There is a limit on the effectiveness of this mode in large configurations).
Very large Very large
JMS Compliant Yes No Yes
Guaranteed Messaging (Robust)
Yes Yes Yes
Support for routing P2P Interactions
No Yes JXTA and later Gnutella
Support for Audio/Video Conferencing & raw RTP clients
No No Yes
Communication through proxies and firewalls
Yes No Yes
Support for XPath queries/ subscriptions
No Yes Yes
end-to-end Security Yes No Yes
Network Performance Monitoring
No No Yes
Functionality II WebSphere MQ
(formerly MQSeries)Pastry NaradaBrokering
Workflow Support Yes No NoSupport for P2P distributed caching
No Yes (Squirrel) No
Platforms or Hosting Environments
35 different OS/ platforms supported. Also supports the Java Platform.
Supported on platforms which support C# (Microsoft) or Java (Rice).
Platforms supporting Java 1.4 (tunneling C++)
Maturity of Software
Extremely mature, with very robust diagnostic information
Fair Fair with some “production” testing
Transport Protocols Supported
TCP, HTTP, Multicast, SSL, SNA etc.
TCP, UDP TCP (Blocking and non-blocking), Parallel TCP, UDP, Multicast, HTTP, SSL, RTP, (GridFTP)
Multiple transport protocols over multiple hops.
Yes No Yes
Broker Network Design Interface
No No In Progress
WS-Reliability & WS-RM • There are two rival reliable messaging
specifications for Web Services that provide reliable delivery between two endpoints.
• Both the specifications use positive acknowledgements to ensure reliable delivery. WSRM recently has incorporated support for NACKS.
• Both specifications include support for faults
• WS-Reliability is a SOAP based protocol• WS-ReliableMessaging provides an XML
schema for reliable messaging. – Includes a SOAP binding.
NaradaBrokering & Reliable Delivery specifications
• We will provide support for both these specifications – In NaradaBrokering we provide reliable delivery
from multiple points to multiple points
• We have identified issues that will allow federation between these specifications– Sequence numbering, fault mappings,
numbering rollovers, quality of service guarantees
• Federation would allow– WSRM sender & WS-Reliability receiver– WS-Reliability sender & WSRM receiver
NaradaBrokering, WS-Notification & JMS• NaradaBrokering is JMS compliant• Topics in NaradaBrokering could be based on XML,
String(as in JMS), Plain text, Integers, and (tag=value) tuples.– Subscriptions could be XPath queries, SQL queries, Regular
expressions, Strings and integers
• Almost all the primitives needed in WS-Notification are available in NaradaBrokering– Exception: Entities never communicate directly with each
other, as proposed in WS-Notification.– We are either allow such direct communication or mimic in
NB – no performance overhead!
• We are currently building a prototype implementation of WS-Notification
• Need to relate WS-Notification with WS-Eventing and WS-Events
NaradaBrokering and NTP• NaradaBrokering includes an implementation of the
Network Time Protocol (NTP)• All entities within the system use NTP to
communicate with atomic time servers maintained by organizations like NIST and USNO to compute offsets– Offset is the computed difference between global time and
the local time.– The offset is computed based on the time returned from
multiple atomic time servers.• The NTP algorithms weighs results from individual time clocks
based on the distance of the atomic server from the entity.
• This ensures that all entities are within 1 millisecond of each other.
• The timestamps account for clock drifts that take place on machines– Time returned is based on software clocks which can slow
down with increased computing load on the machine.
Collaboration and Web Services Collaboration has
a) Mechanism to set up members (people, devices) of a “collaborative sessions”
b) Shared generic tools such as text chat, white boards, audio-video conferencing
c) Shared applications such as Web Pages, PowerPoint, Visualization, maps, (medical) instruments ….
b) and c) are “just shared objects” where objects could be Web Services but rarely are at moment
• We can port objects to Web Services and build a general approach for making Web services collaborative
a) is a “Service” which is set up in many different ways (H323 SIP JXTA are standards supported by multiple implementations) – we should make it a WS
Shared Event Collaboration All collaboration is about sharing events defining state changes
• Audio/Video conferencing shares events specifying in compressed form audio or video
• Shared display shares events corresponding to change in pixels of a frame buffer
• Instant Messengers share updates to text message streams
• Microsoft events for shared PowerPoint (file replicated between clients) as in Access Grid
Finite State Change NOT Finite State Machine architecture Using Web services allows one to expose update events of all
kinds as message streams Need publish/subscribe approach to share messages (NB) plus System to control “session” – who is collaborating and rules
• XGSP is XML protocol for controlling collaboration building on H323 and SIP
0
10
20
30
40
50
60
0 200 400 600 800 1000 1200 1400 1600 1800 2000
De
lay
(Mill
ise
con
ds)
Packet Number
Average delays per packet for 50 video-clients NaradaBrokering Avg=2.23 ms, JMF Avg=3.08 ms
NaradaBrokering-RTP JMF-RTP
0
1
2
3
4
5
6
7
8
0 200 400 600 800 1000 1200 1400 1600 1800 2000
Jitte
r (
Mill
ise
con
ds)
Packet Number
Average jitter (std. dev) for 50 video clients.
NaradaBrokering Avg=0.95 ms, JMF Avg=1.10 ms
NaradaBrokering-RTP JMF-RTP
Polycom, Access Grid and RealVideo views of
video-mixed streams using GlobalMMCS
Integration of PDA, Cell phone and Desktop Grid Access
NB Support for optimizedPDA Communication
GlobalMMCS Futures I• 1. New Collaboration tools
– SVG game ( stable )– Whiteboard ( stable )– e-Sport ( prototype)– Jabber IM client ( prototype)
• 2. JMF Audio/Video client ( stable)– performance enhancement finished– new codec ( MPEG4 finished; try H.264)– support different platform ( Linux, Mac )– support NAT/firewall– screen codec for shared display ( prototype )
• 3. Replay & Archive (prototype)– Replay Engine based on NaradaBroker Storage Service– XGSP-RTSP gateway – Extend RTSP and NaradaBrokering for Instant Replay
• 4. Web Server ( stable)– Standard calendar service ( iCalendar, vCalendar)– Flexible conference management
GlobalMMCS Futures II• 5. Conferencing Media Processing Service ( Stable)
– Fix the bugs and add scheduling– Support new codec ( MPEG4 )
• 6. H.323 Gateway ( Stable)– Import it to Linux platform
• 7. RealStreaming Gateway ( Stable )– Import it to Linux– Support Mobile clients
• 8. Global-MMCS deployment & test– Test under the setting of multiple NaradaBroker and
NAT/Firewall– support deployment for AFRL, NASA, DOE portals– test with remote sites
• 9. PDA Clients (prototype)