Communication Patterns Using Data-Centric Publish/Subscribe
-
Upload
sumant-tambe -
Category
Technology
-
view
5.198 -
download
1
description
Transcript of Communication Patterns Using Data-Centric Publish/Subscribe
Sumant Tambe, Ph.D.
Senior Software Research Engineer
Real-Time Innovations, Inc.
www.rti.com
Agenda
• Communication Models
• Data Distribution Service Standard
• Data-Centricity 101 – Demo
• Communication Patterns – Request/Reply
– Guaranteed Delivery
10/11/2012 2
RPC Vs. Pub-Sub/Messaging/Data-Distribution
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 4
• RPC (WebServices, CORBA, DCOM) offer a remote method abstraction
– Familiar OO programming model
– Results in a tightly-coupled system • Forces synchronous invocations
• Imposes global object model
• Limited QoS(appearance of local method call)
• Lack robustness: cascading points of failure
– Typically built on top of TCP: • Impacts scalability and time-determinism
– Best-suited to smaller, closely-coupled systems
• Pub-Sub (Messaging Data-Distribution) offer a queue-based and/or replicated-data model
– Subsystems are decoupled in time, space, and synchronization • Contracts established by verifying QoS compatibility
– Supports a variety of transports including multicast UDP
– Better suited for high-performance and real-time
Queue Vs. Pub/Sub
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 5
• Queue – Message sent to Queue
– Multiple readers can read from the queue
– Each message is delivered to ONLY one reader
• Readers “affect each other”
– Apps:
• Job Scheduling, Load Balancing, Collaboration
• Pub/Sub – Message sent to Topic
– Multiple readers can subscribe to Topic with or without filters
– Each message delivered to ALL subscribers that pass filter
• Readers are decoupled
– Apps:
• Notifications, Information Distribution
Pub/Sub Vs. Data-Distribution
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 6
• Pub-Sub – Only messages no concept of data
– Each message is interpreted without context
– Messages must be delivered FIFO or according to some “priority” attribute
– No Caching of data
– Simple QoS: filters, durability, lifespan
• Data-Distribution – Messages represent update to data-objects
– Data-Objects identify by a key
– Middleware maintains state of each object
– Objects are cached. Applications can read at leisure
– Smart QoS
• Ownership, History (per key), Deadline
– Subsumes Pub-Sub
Data Bus
DDS: Standards-based Integration Infrastructure for Critical Applications
© 2009 Real-Time Innovations, Inc.
Streaming Data
Sensors Events
Real-Time Applications
Enterprise Applications
Actuators
Systems that interact with the Real World
• Must adapt to changing environment • Cannot stop processing the information • Live within world-imposed timing Beyond traditional interpretation of real-time
© 2010 Real-Time Innovations, Inc.
RPC over DDS
2013
DDS Security
2012
Family of Specifications
© 2009 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL 10
DDS Implementation
Network / TCP / UDP / IP
App
DDS Implementation
App
DDS Implementation
DDS Spec
2004
DDS Interoperablity
2006
UML Profile for DDS
2008
DDS for Lw CCM
2009
DDS X-Types
2010 2010
DDS-STD-C++ DDS-JAVA5
App
© 2009 Real-Time Innovations, Inc. 11
Broad Adoption
• Vendor independent – API for portability – Wire protocol for interoperability
• Multiple implementations – 10 of API – 8 support RTPS
• Heterogeneous – C, C++, Java, .NET (C#, C++/CLI) – Linux, Windows, VxWorks, other
embedded & realtime • Loosely coupled
DDS Real-Time Publish-Subscribe Wire Protocol (RTPS)
Middleware
DDS API
Cross-vendor portability
Cross-vendor interoperability
DDS adopted by key programs
• European Air Traffic Control – DDS used to interoperate ATC centers
• UK Generic Vehicle Architecture – Mandates DDS for vehicle comm.
– Mandates DDS-RTPS for interop.
• DISR (DoD Information Technology Standards Registry) – Mandates DDS for Pub-Sub API and Interop.
• US Navy Open Architecture – Mandates DDS for Pub-Sub
• SPAWAR NESI – Mandates DDS for Pub-Sub SOA
© 2010 Real-Time Innovations, Inc.
Everyday Example: Schedule Meeting via Emails
Alternative Process #1 (message-centric):
1. Email: “Meeting Monday at 10:00.”
2. Email: “Here’s dial-in info for meeting…”
3. Email: “Meeting moved to Tuesday”
4. You: “Where do I have to be? When?”
5. You: (sifting through email messages…)
14
Everyday Example: Schedule Meeting Using a Calendar
Alternative Process #2:
1. Calendar: (add meeting Monday at 10:00)
2. Calendar: (add dial-in info)
3. Calendar: (move meeting to Tuesday)
4. You: “Where do I have to be? When?”
5. You: (check calendar. Contains consolidated-state)
15
The difference is state! The infrastructure consolidates changes and maintains it
What’s the Difference? State.
• Objects have identity and attributes – The meeting will run 1:00–
2:00 in the conference room. – My friend’s phone number is
555-1234 his email is… – The car is blue and is
traveling north from Sunnyvale at 65 mph.
• …whether they exist in the real world, in the computer, or both
• …whether or not we observe or acknowledge them
“State” (“data”) is a snapshot of those attributes and characteristics.
If the infrastructure maintains the state, the application does not need to re-construct it…
Why is it better to have the (data-centric) middleware manage the state?
• Reconstructing the state of an object is hard – Must infer based on all previous messages – Maintaining all these messages is expensive – Each app makes these inferences duplicate effort
• Reconstructing state is not robust – Many copies of state may be different bugs vs. Uniform operations on state fewer bugs – Any missing change compromises integrity
• State awareness results in better performance – Middleware can be smart about what to send and when
• Data-type awareness simplifies programming – Middleware supports direct definition and instantiation of the data-
model
DDS Real-Time Quality of Service (QoS) Attributes
QoS Policy
DURABILITY
HISTORY
READER DATA LIFECYCLE
WRITER DATA LIFECYCLE
LIFESPAN
ENTITY FACTORY
RESOURCE LIMITS
RELIABILITY
TIME BASED FILTER
DEADLINE
CONTENT FILTERS
Vo
lati
lity
Use
r Qo
S
De
live
ry
Pre
sen
tation
R
ed
un
dan
cy
Infr
astr
uct
ure
Transp
ort
QoS Policy
USER DATA
TOPIC DATA
GROUP DATA
PARTITION
PRESENTATION
DESTINATION ORDER
OWNERSHIP
OWNERSHIP STRENGTH
LIVELINESS
LATENCY BUDGET
TRANSPORT PRIORITY
Why Patterns
• From app programming POV
– Patterns solve specific problems nicely
• From platform or systems POV
– Handling each pattern in independent ad hoc ways results in brittle, poorly performing systems
• Solution: Build platform in terms of fundamental patterns
– Derive high-level patterns from basic patterns; expose those to apps
© 2012 RTI 22
Introduction to Request/Reply
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 24
• Publish/Subscribe (Pub/Sub) – Fundamental pattern in RTI Connext – Pub/Sub shines where there is
• Push model • Unidirectional stream of data • One-to-many (multicast)
• Request/Reply – Client-server model built on top of pub/sub – Request/Reply shines where there is
• Pull model • Bidirectional communication • A need for remote code execution
– E.g., Get/set configuration, send commands
Request-Reply
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
Requestor Replier
Request (Foo)
Request Topic
Reply Topic Reply
(Bar)
Correlation
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
Replier Requests
Replies
3
3
2 1
2 1
Message ID
Correlation ID
1
1
Requestor
Correlation
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 27
How Does Correlation Work
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 27
Requester App Replier App
Data Writer (VGUID1)
Data Reader
Data Reader
Data Writer (VGUID2)
VGUID1 VSN1 Foo
VGUID2 VSN2 Bar VGUID1 VSN1
sample_id1
sample_id-id2 related_sample_id1
Content-based Filter “@related_sample_id.VGUID == VGUID1”
Read
Bar
Index Condition “@related_sample_id.VSN == VSN1”
Write Foo
VSN1
Foo
Bar
Processing
VGUID1
Filtering Implementation details
Part of SampleInfo
Single-Request Multiple-Reply
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
Requestor Replier
Replies
3 2 1
Sequence ID
Request
Multiple Repliers
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
Requester
Replier A
Replier C
Replier B
Request
Reply
Replier “Hello, World!” (Server-side)
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 30
using namespace connext; DDS::DomainParticipantFactory factory = DDS::DomainParticipantFactory::get_instance(); DDS::DomainParticipant * participant = factory->create_participant(domain_id, DDS::PARTICIPANT_QOS_DEFAULT, NULL, DDS::STATUS_MASK_NONE); std::auto_otr<Replier<Foo, Bar> > replier (new Replier<Foo, Bar>(participant, "TestService")); Sample<Foo> request; // Receive one request bool received = replier->receive_request(request, MAX_WAIT); if (!received) { std::cout << "Requests not received" << std::endl; return false; } // A WriteSample automatically initializes and finalizes // the data using the TypeSupport (in this case BarTypeSupport) WriteSample<Bar> reply; if (request.info().valid_data) { sprintf(reply.data().message, "Reply for %s", request.data().message); // Send a reply for that request replier->send_reply(reply, request.identity()); // Note: a replier can send more than one reply for the same request }
Requester “Hello, World!” (Client-side)
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 31
using namespace connext;
// Create a DomainParticipant
DDS::DomainParticipant * participant = ...;
// Create a Requester
std::auto_ptr<Requester<Foo, Bar> > requester (new Requester<Foo, Bar>(participant, "TestService")); // Send request
WriteSample<Foo> request;
strcpy(request.data().message, "A Request");
requester->send_request(request); if (requester->wait_for_replies(2, MAX_WAIT, request.identity())) { LoanedSamples<Bar> replies = requester->take_replies(request.identity());
// If there are no replies, the container is empty
for(LoanedSamples<Bar>::const_iterator it = replies.begin();
it != replies.end();
++it) {
if (it->info().valid_data) {
std::cout << "Received reply: " << it->data().message << std::endl;
}
} }
Implementing Request/Reply Pattern
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 32
Define request and reply types Manual Manual
Steps With Vanilla DDS API With BIGPINE API
Create Participant Manual Manual
Pollute application data model with a correlation - id
Manual Not necessary
Create publisher/subscriber Manual Automatic
Create datareader / datawriter Manual Automatic
Register request/reply types Manual Automatic
Create request/reply topics Manual Automatic
Setup CFT to filter unwanted replies Manual Automatic
Correlation of request/reply samples Manual Semi - Automatic
Memory management of samples Manual Automatic
Lifecycle m anagement of entities Manual Automatic
Language friendly (idiomatic API) Partially Yes
Scalable Depends on impl Yes
Salient Features of Connext Request/Reply
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 33
• New API, built on top of DDS
• Data-centric
• Scalable
• Simple to use, abstracts many DDS details
• Idiomatic, language-friendly API (C++, Java, and C#)
• It’s not RPC, though can be the foundation for an RPC solution
Scalable Request-Reply
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
Replier
Requester 1
Requester N
Requester 2
Replies
Reply 1
Reply 2
Reply N
…
Thousands of clients!
Combining Patterns
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
Requestor Replier
Request Request
Topic
Reply Topic Reply
Subscriber
Subscriber
Wire Tap
The Guaranteed Delivery Problem
© 2012 RTI • COMPANY CONFIDENTIAL
• Delivery of samples from a Producer to the Intended Consumers such that the delivery is robust to multiple kind of failures
• Guaranteed delivery does not imply complete certainty Continuum of Delivery Guarantees
The Guaranteed Delivery Scenario
© 2012 RTI • COMPANY CONFIDENTIAL
Producer ‘P’ (DataWriter)
Consumer ‘C’ (DataReader)
Sample ‘S’
• Producer ‘P’ publish a Sample ‘S’ that should be received by Consumer ‘C’
Producer process failure
(the producer process crashes after the sample has been written)
Network and transport failures (packets are lost due to temporary
network failures or transport/middleware buffer overflows)
Late joiner consumer (the consumer is not running when
the sample is written)
Consumer process failure (the process crashes after the sample is
received by the middleware and before it is processed by the application)
Guaranteed Delivery
© 2011 Real-Time Innovations, Inc. COMPANY CONFIDENTIAL
Publisher
Message
Disk
Message
Message
Subscriber
Durable Subscriber
Message
App-level ack
DDS Durability QoS Primer
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 40
DDS Durability controls data availability with respect to late joining datareaders
• VOLATILE
– No need to keep data samples for late joiners
• TRANSIENT_LOCAL
– Data instances available for late joiners
– Data does not survive beyond datawriter
• TRANSIENT
– Data is not tied to the lifecycle of the datawriter.
– A separate service (persistence service) keep data in memory
• PERSISTENT
– Data is kept on permanent storage, so that they can outlive a system session
• DataWriter can be configured with:
– a list of required subscriptions • Required subscriptions are named subscriptions identified by a
role name
• DataWriter must store a sample until both:
– Acknowledged by all active reliable readers
– Acknowledged by all required subscriptions
• DataReader identifies itself:
– As serving a required subscription
– Uniquely via a virtual GUID
Beyond DDS: Required Subscriptions
• Concept
– It is a required subscription with durability >= TRANSIENT
– Features that allows to keep data until received by durable subscriptions for which readers may or may not exist at any given time
– Data is not lost even if the DataWriter crashes
• Benefits and Use Cases
• In combination with other features durable subscriptions provides guaranteed messaging
• Requires persistence service – To persist data
– To persist existence of durable subscriptions and their stats
Beyond DDS: Durable Subscriptions
• Concept: – Multiple DataWriters can relay samples from a common
stream (data source). The DataReaders reconstruct the original stream and deliver to the application the complete set of samples
• Benefits & Use cases: – High availability and fault tolerance
• Multiple DataWriters publishing the same samples
– Load balancing and data partition • Multiple DataWriters publishing different sets of samples from the
same source
– Scalability • Partition across different machines and cores
Beyond DDS: Collaborative DataWriters
Delivery Models: Best-Effort to Guaranteed!
© 2012 RTI • COMPANY CONFIDENTIAL
Use Case Features in RTI Connext Messaging 5.0.0
Periodic data from live producers Best effort reliability
Event data from live producers Reliable reliability
Last value cache from live producers (entire state of the datawriter cache)
Transient local durability + Reliable reliability
Durable last value cache (datawriter may come and go)
Persistent durability + Reliable reliability + Persistence Service
Guaranteed messaging from live producers Transient local durability + Reliable reliability + Required subscriptions + Application level ACK
Durable Guaranteed messaging Persistent durability + Reliable reliability + Collaborative DataWriters + Persistence Service + Durable subscriptions + Application level ACK
References
10/11/2012 © 2012 RTI • COMPANY CONFIDENTIAL 45
• Real-Time Innovations, Inc. – www.rti.com
• RTI Connext DDS – www.rti.com/products/dds
• RTI Connext Request-Reply API – Just Google!
• OMG DDS Specifications – www.omg.org/technology/documents/dds_spec_catalog.htm