SunGuide TM Center-to-Center Functionality What “it is” and How to “use it” November 12,...

SunGuideSunGuideTMTM Center-to-Center Functionality Center-to-Center FunctionalityWhat “it is” and How to “use it”What “it is” and How to “use it”

November 12, 2008November 12, 2008

Purpose of this class…Purpose of this class…

Part 1 – C2C Fundamentals:– Explain the two “data channels” of C2C:

• Status• Command / Control

– How C2C “works”– What does it do and how does it do this– How to “deploy” C2C

Part 2 – Developer / Support Training:– Test tools available to the developer / support staff– Configuring C2C to “work”– How to write a “plug-in”– How to troubleshoot

November 2008C2C Concepts 2


C2C Fundamentals


Why Utilize a C2C Standard?Why Utilize a C2C Standard?

Allows data to be collected across TMCs

Allows assets to be used across TMCs

Provides TMC “software independence”

TMCs are not “equal”:– Different equipment– Different data from different TMCs (all do not provide

the same)

A tool to collect regional or statewide data (that could be used by a statewide ATIS system)


C2C Infrastructure Concept:C2C Infrastructure Concept:Functional Areas AddressedFunctional Areas Addressed

Roadway network:– Layout (GIS based)– Current conditions

Events (detailed data) Lane Closure Emergency Evacuation Field Devices:

– DMS– LCS– CCTV– Ramp Meter– HOV– Traffic Signals– Environmental Sensors

Railroad Crossings

HAR Parking Lots School Zones Reversible Lanes Dynamic Lane Assignments Transit (Bus):

– Schedules– Location

Commuter/Light Rail:– Schedules– Location

Signals:– Intersection– Artery/Section/Corridor

Vehicle Priority

Addressing both Status and Command/Control


C2C Infrastructure Concept:C2C Infrastructure Concept:Conceptual ViewConceptual View


C2C Infrastructure Concept:C2C Infrastructure Concept:Deploying a C2C “Cloud”Deploying a C2C “Cloud”


TMDD Network ModelTMDD Network Model(TMDD = Traffic Management Data Dictionary)(TMDD = Traffic Management Data Dictionary)

The network model is the underlying data model for C2C Travel times / traffic conditions are published as “link” data Each SunGuide deployment is its own “C2C network” Equipment must be associated with a “C2C network”

During the following slides…During the following slides…

Please consider the following as we move forward:– C2C was designed to be a generic carrier of C2C data– C2C was NOT designed specifically for FDOT SunGuide– C2C was NOT designed specifically for FDOT FL-ATIS

The C2C can be deployed in MANY ways to achieve different objectives

C2C is being deployed in a very specific fashion to satisfy SunGuide requirements

Please do NOT jump to “conclusions” as to how C2C is used until the end of the presentation! Just sit back and “soak it in”!


C2C Design ConceptsC2C Design Concepts

Status:– Data revolves around a “network id” (each deployment

will have a UNIQUE “network id”– When a ‘deployment’ comes on-line, the data associated

with the deployment is ‘discovered’ and published (added) to the C2C infrastructure

– ‘Deployments’ pace how often status data is updated– When a ‘deployment’ goes off-line it is ‘erased’ (deleted)

out of the C2C immediately– Status is like a “party line” where all participants can

hear each other simultaneously

Command:– Point-to-point connection created as needed


Brisco, John S.

Fixed spelling of UNIQUE and how.

Two “Paths” Through C2CTwo “Paths” Through C2C

When a C2C environment is “Designed” (configured), all participants need to decide what is to be exchanged (and what the ground rules are)

Design decisions include:– SunGuide:

• Status Data– Outbound: Published to C2C as soon as changes are made (i.e.

DMS changed, traffic conditions updated, event updated, etc.)– Inbound: Received and posted upon arrival

• Commands:– Outbound: must be sent with a username/password– Inbound: must have a local username/password to process

– FL-ATIS:• Status Data

– Outbound: Publishes Floodgate data– Inbound: Receives DMS, CCTV, EM Locations, Roadways, Links

• Commands:– Outbound: none– Inbound: accepts Floodgate and Event requests


Data Flow Through C2CData Flow Through C2C

Two types of data flows:– Status (broadcast in nature, one-to-many)– Command/Control (point-to-point, one-to-one)

GENERIC infrastructure to exchange XML (there are some basic formatting guidelines) – does NOT have to be “ITS data” – could be “stock prices”)


Defining XML for C2CDefining XML for C2C

All participants on a C2C “network” need to ‘agree’ to what the messages that will be exchanged contain

Since XML is not a “protocol”, an exchange protocol must be agreed to (in C2C it is SOAP)


Adding Plug-insAdding Plug-ins

Plug-ins “glue” (or convert) between system specific data definitions and the C2C XML definitions

Note that systems connecting to the C2C can be from different vendors and their internal “architecture” is transparent to the C2C


Startup / Inventory and statusStartup / Inventory and status

How does a Center tell others about “what inventory has”:

– On initial connection to the C2C infrastructure a center will “declare” its inventory

– This typically includes:

• Names

• Locations

• Attributes (usually a limited set, e.g. rows/cols) After inventory is published, status is provided for the inventory:

– Current status (e.g. current message, current speed)

– Pace of “status” publication is entirely based on the SENDER

– C2C propagates changes as soon as they are received

When a center CONNECTS to the C2C and asks for “status” it will receive an inventory and then status updates.

Status can only be published for something previously declared in an inventory


Focus On How Status Focus On How Status Information Is TransmittedInformation Is Transmitted

To ‘integrate’ multiple centers a decision on WHAT data will be sent must be agreed to

Inventory data is sent ONE time Status data is sent as changed (pace is established by the

‘source’


Behavior of C2C ComponentsBehavior of C2C Components

Each component has a different “purpose”– Provider – getting data into C2C– Collector – repository for data– Extractor – retrieving data from C2C


Brisco, John S.

The text to the left of the C2C Extractor box should be "One Connection INTO an Extractor". Same for slides 18, 19, and 20.

C2C Provider StartupC2C Provider Startup

C2C Provider:– Plug-in connects to provider and provides inventory data– Inventory data forwarded to any connected downstream

Collector or Extractor– As status updates received, they are passed downstream– Data stored for future connection requests


C2C Collector StartupC2C Collector Startup

C2C Collector:– Connects to any configured Collectors or Providers– Inventory data forwarded to any connected downstream

Collector or Extractor– As status updates received, they are passed downstream– Data stored for future connection requests


C2C Extractor StartupC2C Extractor Startup

C2C Extractor:– Connects to any configured Collectors or Providers– Inventory data forwarded to any connected plug-in– As status updates received, they are passed to plug-in– No storage of data


Rack and Stack Status Rack and Stack Status ComponentsComponents

There is not one “right way” to deploy C2C Deployment is based on what data is to be delivered where:

– If you want all users to get all data…– If you want some users to only get data from 2 sources…– Etc…


Status: Subscribing for Status: Subscribing for SunGuide DataSunGuide Data

Configuration files specify WHAT data is “subscribed for” Subscription is at “network” (center) level, is not used to

“filter” data EACH C2C component “can” have a different configuration Can be used to create “custom” data feeds


Status Information from Status Information from SunGuideSunGuide

Mapping of SunGuide subsystems to C2C status data

Note: SunGuide does NOT publish “Floodgate Status” (FL-ATIS does)


Status Data From FL-ATISStatus Data From FL-ATIS

Note that SunGuide does NOT publish the Floodgate status information – this comes from FL-ATIS

The FL-ATIS plug-in will update the status of the Floodgate data after a command is received.


Tracing a Single Status Data Tracing a Single Status Data Type: DMSType: DMS

Each center “publishes” their DMS data A Collector will “gather” the data A consumer of data can get all the data from ONE source Note a TMC only needs to send status once for multiple

consumers to benefit (performance issue)


Network DeletesNetwork Deletes

In “status world” of C2C, when a TMC goes “offline” then ALL STATUS data is PURGED in the downstream C2C components.

Offline is the result of:– Plug-in stopping– Center being “rebooted”– Loss of network connectivity

When a center comes back online, inventory is first published and then status publishing resumes

Intermittent network connections can generate large transfers of data


Tracing a Single Status Data Tracing a Single Status Data Type with a Center “fails”: DMSType with a Center “fails”: DMS

Assume that District 5 has a failure, this could include:– DMS Subsystems stops– All of SunGuide Fails (or key component such as Data Bus)

Data associated with the failure is deleted THROUGHOUT the C2C (the consuming application must also decide “what to do” – in SunGuide the data is purged))


Tracing a Single Status Data Type Tracing a Single Status Data Type with a Collector “fails”: DMSwith a Collector “fails”: DMS

A collector fails (either process of “communications”) Districts are still publishing data (they are ‘healthy’) Downstream ‘consumers’ do not see ANY data During troubleshooting you have to know where to look…


Command DataCommand Data

Included with a command:– Command “data”– Userid / password that must be validated by the

RECEIVING application (C2C performs no validation) Commands must provide ‘response’ to sender


Putting it all together:Putting it all together:How to connect 2 SunGuide DeploymentsHow to connect 2 SunGuide Deployments

Each sends out status information Each retrieves status information ‘of interest’ Command are send a issued by operator Note: RWIS and HAR data are not exchanged via C2C


Brisco, John S.

Sg supports publish and subscribe for both RWIS and HAR.

Putting it all together:Putting it all together:Connecting multiple SunGuide Deployments to FLAITSConnecting multiple SunGuide Deployments to FLAITS

Data from ALL Districts and other providers (e.g. FTE) will go to a single “collector”

Note: EACH deployment will have a “command channel” to FLATIS to deliver events and Floodgates


SunGuide C2C Data:SunGuide C2C Data:Status Data and CommandStatus Data and Command

SunGuide can be configured to process status data from other centers and send commands:– DMS– CCTV– HAR– RWIS– Events (status only)


Carefully Configure SunGuide:Carefully Configure SunGuide:Preventing un-desired status data and commandsPreventing un-desired status data and commands

Configure what SunGuide received (both status and commands) Configure what data SunGuide sends Note: NO command will be processed unless a valid

userid/password for the command type is received


FLATIS Conceptual Data Flow:FLATIS Conceptual Data Flow:(Only one way data flow is shown)(Only one way data flow is shown)


How to “Test”…How to “Test”…

C2C Test Suite that allows:– Data to be repeatedly “injected” (excellent for IV&V or

performance testing)– Data to be “viewed” (excellent for verification and

debugging the infrastructure)


Performance Testing for FL-ATISPerformance Testing for FL-ATIS

Load testing should use the network links FL-ATIS will use Target system should use production hardware for C2C

infrastructure and FLATIS plug-in/database C2C test tools can be used rather than SunGuide to provide

data– Tools should be run in actual Districts to test firewalls

and networks– Test tools allow highly repeatable and predictable data

to be delivered Loading comments:

– Status data is by far the largest user of bandwidth/CPU– Command data is “very small” – Data files to “mock up” an operational District take a

day or less to create


Performance Testing for FL-ATIS: Performance Testing for FL-ATIS: continuedcontinued

Run “Test Tools” in each District

Test files built on “real” data

Utilize FL-ATIS comm. Links

One way testing


Performance Testing for FL-ATIS: Performance Testing for FL-ATIS: Test ToolsTest Tools

For “status” load testing:– Create a “status” file from (a deployed) SunGuide data

stream– Send base file on startup– Send regular updates


Performance Testing for FL-ATIS: Performance Testing for FL-ATIS: Test Tools - continuedTest Tools - continued

For sending FL-ATIS commands using XML Tester:– Send an “event”– Send a “floodgate”



Questions?


Developer / Support Training

OutlineOutline

Key to understanding the data: C2C ICD

Installing and configuring C2C

Testing/validating data: C2C Test Suite

Something is “broken”: C2C Troubleshooting

Plug-in Development


C2C ICDC2C ICD

You HAVE to understand the XML to understand C2C


C2C ICD – Examine the XMLC2C ICD – Examine the XML

XML Spy will be used to examine the XML…


C2C InstallationC2C Installation

Perform a C2C installation… Install:

– C2C Provider– C2C Collector– C2C Extractor– C2C Command/Receiver– Test Suite


Test SuiteTest Suite

Test Server– Publishes status data to C2C Provider– Accepts command requests from C2C Command

Receiver– Validates received commands using C2C XML schemas– Selectable XML-based test data sets

Status Client– Subscribes for status data from C2C Extractor– Validates received status data using C2C XML schemas

Command Client– Interacts with C2C Command Receiver to retrieve device

status and submit command requests– Validates received device data and command responses

using C2C XML schemas


Test ServerTest Server


Test ServerTest Server

Useful for providing event and device status data during development of a “subscriber” plug-in component Useful for accepting command requests during development of a “command sender” plug-in component Configurable Provider connection Selectable XML-based status data sets Manual and automatic publishing of current status, status updates, and status deletes Configurable Command Receiver connection Data validation of incoming commands using C2C XML schemas View published status data, commands received, and command responses Built-in HTML help


Sample Data File for ServerSample Data File for Server

VERY used for “IV&V” or regression testing

Allows “repeatable” data to be used in testing

A file can be created to send an entire TMC

Files for ‘n’ TMCs could be produced and all sent to the C2C


Sample Data File for Server – Sample Data File for Server – continuedcontinued

Send Floodgate:

Send Event:


Status ClientStatus Client

Useful for consuming event and device status data during development of a “publisher” plug-in component

Configurable Extractor connection

Data validation using C2C XML schemas

View entire status tree, update messages, and update elements

Built-in HTML help


Command ClientCommand Client

Useful for delivering command requests during development of a “command receiver” plug-in component

Configurable Command Receiver connection

Data validation using C2C XML schemas

View commands sent and responses received

Built-in HTML help


Web Service ConfigurationWeb Service Configuration

Status Logger server host and port Extractor

– Server list contains a single Collector or Provider URI Collector

– Server list contains one or more Collector and/or Provider URIs (blank delimited)

– Subscription data types Provider

– Subscription data types Command Receiver

– XML server port value

Note: Web service “web.config” settings documented in SunGuide VDD


Example Configuration FileExample Configuration File

Extractor application settings from its Web.Config file:



Collector application settings from its Web.Config file:



Provider application settings from its Web.Config file:



C2C Test Suite

Demonstration

C2C TroubleshootingC2C Troubleshooting

IIS– ASP.NET Version, Enabled, “aspnet_regiis.exe”– Application pools: disable recycling– Web service properties

Web service “web.config” files– Valid Status Logger values– Valid URI’s (Extractor and Collector)– Correct subscription data types (Provider and Collector)

Network connectivity– Telnet test– IE web service access test


C2C Troubleshooting (cont’d)C2C Troubleshooting (cont’d)

Plug-in configuration (SunGuide)– Publisher configuration

• Valid C2C Provider host and web service• Valid C2C command port• Correct “statusList” and “commandList” elements

– Subscriber configuration• Valid C2C Extractor host and web service• Valid Extractor update port• Correct Extractor subscription list• Valid C2C Command Receiver host and web service• Valid Command Receiver center ID and credentials


Plug-in DevelopmentPlug-in Development

Determine what support your system needs:– Status

• Publish event and/or device status data• Subscribe for event and/or device status data

– Command/Control:• Receive command/control requests• Transmit command/control requests

Develop “glue” (aka plug-in) to interface YOUR system to the infrastructure using the C2C ICD (Interface Control Document) and C2C XML schemas

Plug-in utilizes web services to exchange data with the infrastructure


Publisher Plug-in DesignPublisher Plug-in Design

Establish functionality:– Identify local device and event data types to be

supported by plug-in– Use C2C Status ICD and XML Schemas to identify

required and optional data elements for each data type to be supported

Code Development:– Code that retrieves data from local TMC components– Code that parses and stores local device and event data– Code that interfaces with the C2C Provider using the

“ProviderClient” interface– Code that generates C2C XML messages for current

status, status updates, and status deletes– Code that manages local TMC connection and C2C

Provider connection


Publisher Plug-in DesignPublisher Plug-in Design

C2C Provider “ProviderClient” interface:


Publisher Plug-in OperationPublisher Plug-in Operation

Normal Flow:– Connect to local TMC resources; gather device and

event data from TMC server and database; store in local cache; subscribe for device and event updates

– Initialize connection to C2C Provider and deliver current device and event data for configured types

– Deliver status updates and status deletes when supplied by local TMC

Network Interruptions:– Loss of TMC connection should cause send of Network

Delete to C2C Provider; actions after reconnection are the same as at startup

– Loss of C2C Provider connection should cause Provider reset; actions after reconnect include send of Network Delete followed by startup actions


Subscriber Plug-in DesignSubscriber Plug-in Design

Establish functionality:– Identify C2C device and event data types to be supported by

plug-in– Use C2C Status ICD and XML Schemas to identify required

and optional data elements for each data type to be supported Code Development:

– Code that parses C2C device and event data for current status, status updates, and status deletes

– Use the C2C device and event data elements as new local data types, OR

– Convert the C2C device and event data into local TMC equivalents

– Code that delivers remote device and event data to local TMC components

– Code that interfaces with the C2C Extractor using the “ExtractorServer” interface

– Code that manages local TMC connection and C2C Extractor connection


Subscriber Plug-in DesignSubscriber Plug-in Design

C2C Extractor “ExtractorServer” interface:


Subscriber Plug-in OperationSubscriber Plug-in Operation

Normal Flow:– Connect to local TMC resources– Start listener socket for connection by C2C Extractor

and initialize connection to Extractor– Parse C2C current status messages as they are

received and store at least data type and data identifiers to local cache

– Convert to designated local format and deliver to TMC– Process C2C status update messages and C2C status

delete messages the same way– Process C2C network delete messages by sending local

delete messages for each cached device and event instance

Network interruptions:– Loss of C2C Extractor connection should result in

delete messages to local TMC for all device and event instances

– Actions after reconnection are the same as at startup


Command Handler Plug-in Command Handler Plug-in DesignDesign

Establish Functionality:– Identify local device types to be supported for remote

commands by plug-in– Use C2C Command ICD and XML Schemas to identify

required and optional command elements for each device type to be supported

– Usually developed to be part of C2C Status Publisher Plug-in for local data cache sharing

Code Development:– Code that generates local TMC command messages for

supported device types– Code that accepts messages from the C2C Command

Receiver– Code that parses the C2C Command request messages– Code that generates C2C Command response

messages– Code that manages local TMC connection and C2C

Command Receiver connection (listener socket)


Command Handler Plug-in Command Handler Plug-in OperationOperation

Setup:– Connect to local TMC resources. Perform any necessary

authentication– Initialize socket connection for C2C Command Receiver

Command Processing:– Parse received commands and convert to local TMC

format. Check for presence of device in local data cache– Submit command to local component for processing– Parse received responses and convert to C2C

Command response format– Return formatted response on same socket used for

command receipt


Command Sender Plug-in Command Sender Plug-in DesignDesign

Establish functionality:– Identify remote device types to be supported by plug-in– Use C2C Command ICD and XML Schemas to identify required

and optional command elements for each device type to be supported

– Usually developed to be part of C2C Status Subscriber Plug-in for local data cache sharing

Code Development:– Code that parses local device command request messages– Code that generates C2C Command request messages– Code that interfaces with the C2C Command Receiver– Code that parses C2C Command response messages– Code that generates local device command response

messages– Code that manages local TMC connection and C2C Command

Receiver connection


Command Sender Plug-in Command Sender Plug-in DesignDesign

C2C Command Receiver interface:


Command Sender Plug-in Command Sender Plug-in OperationOperation

Normal Flow:– Connect to local TMC resources; perform any necessary

authentication– Initialize C2C Command Receiver connection– Parse received commands and convert to C2C

Command format; check for presence of device in local data cache

– Submit command to designated C2C Command Receiver for delivery to remote TMC

– Parse received responses and convert to local TMC response format

– Return formatted response to submitting TMC component


General ConsiderationsGeneral Considerations

Allocation of plug-in components to applications based on needs of local system

Connection between plug-in and Provider or Extractor web service uses IIS sessions:– Session identifiers contained in “cookies”– Plug-in needs to define a cookie container and assign it

to the web service proxy– Plug-in needs to call the web service “KeepAlive” web

method regularly to prevent session timeout Plug-in that publishes status data should issue “network

delete” and “shutdown” to its Provider when shutting down Plug-in that subscribes for status data should “logout”

from its Extractor when shutting down


Sample Source Code:Sample Source Code:“Help” to start a status publishing plug-in“Help” to start a status publishing plug-in


Sample Source Code:Sample Source Code:“Help” to start a status subscribing plug-in“Help” to start a status subscribing plug-in


DocumentationDocumentation

C2C Infrastructure and Test Suite installer documented in SunGuide VDD

C2C Infrastructure web service configuration settings documented in SunGuide VDD

C2C ICD and XML Schemas available from SunGuide Project website (sunguide.datasys.swri.edu)

Example C2C Plug-in project (C#, Visual Studio 2005) Sample C2C XML status data fragments included with Test

Server


Resources…Resources…

Most current C2C ICD is on the project web site in the Reading Room

Sample source code to create a plugin:– http://sunguide.datasys.swri.edu/ReadingRoom/Etc/C2C

%20Class/C2CExample.zip

Sample data to use with the Test Suite:– http://sunguide.datasys.swri.edu/ReadingRoom/Etc/C2C

%20Class/FLATIS-Samples.zip



Questions?

SunGuide TM Center-to-Center Functionality What “it is” and How to “use it” November 12,...

Documents

Transcript of SunGuide TM Center-to-Center Functionality What “it is” and How to “use it” November 12,...