Compartmentalized Common Operational Picture Sc2 Op

Sync’d Compartmentalized Common Operational Picture: SC2OP

1

SAW Concepts LLC Modified Presentation to NYC Computer

Measurement Society, Summer 2011

S.O.S

SYNCSEED_CLOUD

Situational Awareness CDRLS

.0001 .05 .01 .1 5 15 30 99 .0001 .05 .01 .1 5 15 30 99

SYNCSEED_CLOUD

The “SC2OP”

Avada Confidential - Feedback Monitoring - 2010-11

2

A complete, integrated, secure, collaborative portal which

facilitates logical, virtual environments to aid with problem

determination, administration, monitoring, testing, auditing

& statistical reporting for Message Oriented Middleware

(WebSphere MQ, Tibco, & any JMS base solution)

* Providing collaboration between business units

* Providing collaboration btwn different MOM Environments

Avada Software Markets Infrared360

SC2OPSynchronized, filtered

Single Integrated Compartmentalized

Operational Picture derived from the Family of

Interoperable Operational Pictures (FIOP).

Role based / filteredCOMPARTMENTALIZED

Common Operational Picture


3

Metrics For Business Units

Feedback Monitoring – Working with Business Units to Determine Business Application SLA Criteria

Topics:1.Dangerous assumptions planning THRESHOLDS for new business applications2.Best practices for planning threshold management for new business applications

Feedback Monitoring:-Heartbeat msg update: y/n-Update = new netconfig

planning thresholds for new business applications> / < threshold by intensity, duration, magnitude… threshold rules dictate alert and event feedback to mitigate / adjust & reform orgs..

DECISION

CAPXML


4

Business Scenario

Airline plans promotion via new business partner.

New business partner is actually subsidiary and previously booked through airline, but not by leveraging their real-time systems.

Business unit announces new partnership, but not details of new business application.

Business unit understands that middleware group has handled this type of data transaction before.*

Business unit and middleware group do not meet to review new application criteria, only details of middleware objects needed.

Business unit assumes similar thresholds will monitor this application the same way; “this one’s not too different!”

DoD systems: driven by Structured military message Threads that along with theIndividual FFURNS and FFUDS[data elements as 3 / 4 digit codes] Determine funding and developmentSchedule and pace – often, single data elements take over a year to NOT change – e.g., the infamous redThread where an AVIATION data element was the fly in the ointment between a ground system and a intelligence system – true story!

SoS

SABRE

Avada Confidential - Feedback Monitoring - 2010-11 5

Example Application Environment

Multiple Queue Managers with multiple Queues associated with “each” different business application.

Different business applications exist on the same Queue Managers - in order to leverage server capabilities.

Different applications built by different business units, different development staff, different business requirements, different system footprint - all on same server!

Administrators assume similar thresholds will monitor all of the different applications the same way; “this one’s not too different!”

The Heart Beacon involves pre and post configuration management of the network and is neutral to systems and middleware. What Vinton Cerf and his team described metaphorically as the heartbeat is actually a interval in time where (state meta) data is gathered and placed into temporary holding areas, queues, file structures for onward delivery by middleware software such as agents, bots, motes, scripts. The heartbeat is an opportunity in time to synchronize data harvesting protecting bandwidth on low capacity networks prior to entry into portals and fusion centers – the point of no return in context with event time stamping. After transition to the middleware or cloud layer, it is too late to try and re-establish timing of event / alert chain of custody. Many systems report the same event at different time intervals. Too fast updates tend to saturate the network. Too slow refresh rates cause the data to be too “stale” for targeting systems. Operational discipline derived from years of military testing and simulation is reused across the entire portfolio of assets / applications.

The Heartbeacon is neutral to middleware – HOWEVER! Not all Middleware is created with the same design / functionality. The Heart Beacon involves gathering state meta data via the heartbeat andPlacing this data (state meta data snapshots) into temporary QUEUES And other temporary structures for onward delivery to the System Administrations (S-6’s) who concatenate this data into heartbeat messages. A change in the platform / user state must be monitored andUpdated and is THRESHOLD sensitive – a user / platform that hasn’t responded is said to be “stale” and shown in the application accordingly

Queues // Thresholds

6

Fares

Crew

Departure

Cargo Reservations

Ticketing

CodeShare

Mechanics

Qmgr - AAA

Qmgr - BBB

Qmgr - CCC

Qmgr - DDD

Only one problem:

This is only 1

server in 1 airport!

Example Application EnvironmentHeart Beacon option: The TCP/IP sub-protocol’s heartbeat function acts as a mini publish-subscribe mechanism that is set to gather state meta data from cloud-subscriber-user at pre-defined intervals by cloud-subscriber-administrator. State meta data is temporarily transferred to files, QUEUES, folders, caches and other temporary data storages awaiting pick up by transport-agent to the cloud-management-broker that aggregates cloud-subscriber-user state management data into heartbeat state management snapshots (REST Representational State Transfer compliant) for onward replication (copy) to other cloud-subscriber-administrators to effect inter and intra cloud infrastructure changes accommodating group adhoc spontaneous integration and cloud-subscriber-GROUP (described by Organization Identification ORG ID) affiliation / de-affiliation based on current mission, operation, scenario goal.

QUEUE MANAGER = Perfect for the Heart Beacon

7

Qmgr - AAA

Qmgr - BBB

Let’s Multiply!

Fares

Ticketing

Fares

Ticketing

Fares

Ticketing

Fares

Ticketing

Fares

Ticketing

Qmgr - BBB

Qmgr - DDD

Crew

Mechanics

Crew

Mechanics

Crew

Mechanics

Crew

Mechanics

Qmgr - DDD

Fare Quotes are being processed by all endpoints.

Typically Overlooked:

Are there other apps [Blue

Force Tracker] running on those

endpoints?

Departure

Cargo

Departure

Cargo

Departure

Cargo

Departure

Cargo

Qmgr - AAA


Net Centric Warfare Challenge – propagate Sync delta message to update Concept of operations across many networks / subnetsThat are in flux – constantly monitoring end points / platforms / sensors / soldiers

More than 1 type of Blue Force Tracker! Land Warrior/ Grenadier Brat.. Apps for different business units


Qmgr - AAA

Qmgr - BBB

Fares

Ticketing

Fares

Ticketing

Fares

Ticketing

Fares

Ticketing

Fares

Ticketing

Qmgr - BBB

Typically Overlooked:

Which endpoints more closely match the PROFILE of travel portal customers or

GEOGRAPHY of portal

customers ?

Departure

Cargo

Departure

Cargo

Departure

Cargo

CodeShare

Reservations

Qmgr - AAA


User Profile: Role By OrganizationalType – ORG IDOr type of device:URN: Uniform Resource Name

Each Heartbeat Interval, the geo-location or 10 digitGrid coordinate isSampled. Change in geo-loc isExtrapolated into State – moving, halt, stale, in-op..

9

Fares

Qmgr -EWR

The MQ Admins get ready for a long night because alerts will be firing often based on old traffic patterns. The Biz units get calls wondering why customers are calling in because of slow response times for quotes. Are the servers tuned and optimized? Yes – that is not the issue.

Qmgr - CCC

TravelPortal

Qmgr - DDD

AirlinePortal

Fares

Qmgr - HPN

Fares

Qmgr - LGAEach Unit has its own assumptions of typical business transactions.


Beacon Tech / Avada = millisecond response

Disciplined process & temporal sampling discipline protects bandwidth And limited physical resources e.g., server cycles & available time slots Traffic patterns change = network changes


10

Collaborative, Delegated Management

Win2k

OS/390 AIX

AS400

HPUX

Linux

http http

Solaris

HP-nonStop

MQ Adminteam

LGA Qmgrs HPN QmgrsEWR QmgrsJFK Qmgrs ISP-Qmgrs

The MQ Admin has scope to all Qmgrs, Queues, etc. & partial authority to change attributes and settings.

Infrared360 enables role-based visibility into application environments

ROLEBASED

FCC Command &

Control Management

Framework

Heartbeat message#73334 accept / delay

ORG ID / URN

“Rose Colored” Glasses

11


LGA Qmgrs

Win2kOS/390 AIX

AS400

HPUX

Linux

http http

HPN QmgrsEWR Qmgrs

Solaris

JFK Qmgrs ISP-Qmgrs

HP-nonStop“Invent”

Fares app team = “TF Bling-Bling”

MQ Adminteam

The Fares Biz Unit has scope to only the Fare application Objects, but no authority to modify them.

Infrared 360 enables role-based visibility into application environments

Get / put state meta data from cloud-subscriber-user at pre-defined intervals by cloud-subscriber-administrator. State meta data is temporarily transferred to files, queues, folders, caches awaiting pick up by transport-agent to the cloud-management-broker that aggregates cloud-subscriber-user state management data into heartbeat state management snapshots (REST Representational State Transfer compliant) for onward replication (copy) to other cloud-subscriber-administrators to effect inter and intra cloud infrastructure changes accommodating group adhoc spontaneous integration and cloud-subscriber-GROUP (described by Organization Identification ORG ID) affiliation / de-affiliation based on current mission, operation, scenario goal.

Access to target objects are secured by group and role permissions and audit trails mark every action taken

“The GRAIL”

SC2OPHeartbeat Sync Delta message

# N, N +1 accept / delay

12


Travel-Portal

LGA Qmgrs JFK Qmgrs

Win2k

OS/390 AIX

AS400

HPUX

Linux

http http

Solaris

HP-nonStop

Fares app team

Travel Portal team

MQ Adminteam

The Travel Portal Biz Unit has scope to only the interface to airline fare quote system.Infrared360 enables role-based visibility into application environments

“Trusted Spaces”

Unified Comms CenterEAC, EOC, DMOC,TOC…

COMMON SYMBOLS

III

THE BIG REDBUTTON

II

SYNCHRONIZEDEVENTS /

ALERTS

I

CAP

TASK FORCE-BLING $

vBET: Network OVERLAY / JBFSA MDL

Integrated System Control:

ISYSCON

UTOAdhocOrgs

Geospatial OVERLAY

GENI NET SNAPSHOT / Heart Beat Message

Easy ConfigReconfig byTask / Mission

NIEMPayload

CAP

OPS / Intel / Log / Civil Affairs.CFO / CEO / Security / Logistics

Visibility7By RoleOnly seeWhat youAllowedTo see

Heartbeat Sync DeltaMessage accept y/n?

13

An Interactive ApproachHow do these biz units collaborate to adjust monitoring thresholds for the new Fares application and associated promo?• Feedback Monitoring: “A practical approach to working with Business Units”

Situational Awareness CDRLS

.0001 .05 .01 .1 5 15 30 99 .0001 .05 .01 .1 5 15 30 99

SYNCSEED_CLOUD

THINK SYNCTHINK SYNC

THINK SYNC

SYNCSEED_CLOUD

SYNCSEED_CLOUD

Single, Authoritative, Reliable blip / event / alert filteredFrom n Situational Awareness systems

just say no to swivel chair

analysis!

Different view from same Common Picture Different view from same Common Picture- Uncleared soldiers on battlefield do not see Intel source

Ops sees OpsIntel sees Intel

A / B /C / D/ n.. -Pentagon- Wall Street- Global NOC

NISTFort Collins, CO

-IEPD -- Naming Design Rules NDR- Conformance Tool- Payload / data island to Common Alert Protocol

IRODS: Rules Repository / Engine across n domains

CDX NIEM TECHIn The Cloud

Infrared360™

CI-BER: CyberInfrastructure for Billions of Electronic Records

NARA / NCASTUNC CHAPEL HILL


Monitoring Scenarios are usually based upon:

Immediate Thresholds – historical assumptions of current SLAs (MQ Admins generally set these)

Statistical Measurements – capacity teams for each biz unit : MQ Admins, fares app. team, portal app. team)

From historical trends ¹

From test harnesses ²

From transaction measuring software ³


Current Methods

Patterns are usually based on historical volumes:

Assuming the same rate of business ... when your biz wants better?

Cost of analysis typically low = cost of error typically high!

Patterns are usually based upon previous offerings

But expectations are now based upon market condition? – different volume of users, the time of the offering?

If so, the previous patterns may not apply.


Feedback Monitoring:

Deriving Patterns and Behavior

Sometimes there are no previous base statistics so behavior patterns are not yet determined

Even more need for collaborative intermediate analysis• Pattern detection• Application behavior• Peaks & Valleys• Volumes• Timeframes


Supporting Agile Business

Corporations are in difficult economic conditions and/or competitive markets

Need for creativity and agility to create new offerings in order to compete

New offerings are business decisions

Outside of IT

Regardless of IT

Are time dependent to get market share

Are based completely on promised return on investment


Market Driven Examples:

Travel – special fare offerings

Banking – special rates accounts

Insurance – competitive rate offerings

Retail – special price + limited time offerings

Utilities – special rate offerings

Manufacturing – limited resource availability

All of the above – new partners, inventory, financial incentives &

liabilities


Problem Area for Threshold Planning

Issue: Historical basis not yet formed Threshold basis not yet determined Market conditions not certain Success of acceptance of new business app

Goal: Early pattern and behavioural recognition


Best Practices for Setting Thresholds

Modelling New Business Applications • Use real data! “hello world” is not a transaction

• Model transactions in test/QA/etc using the same payloads

as would be in Production

• Model them while running other transactions volumes on

the same Qmgr, other queues, as would be in Production

• Monitor the transactions in test/QA/etc the same as you

would in Production (better to know it now than later)


Modelling new business applications

Review findings with business units

Use comparatives – is this normal, not normal?• Fare app Trend Chart example

Offer opinions on ability to run & support • What are alternatives if predictions are wrong?

Analyze real peaks in application trends• seasonal, weekly, daily (peak levels for each)• your thresholds are built for the peaks – not the average!

Create 'stepped' warning levels for MQ Admins • at threshold A, at threshold B, etc?



Modelling New Business Applications • Review expectations and tactics with business units.• Gather statistics over short intervals during initial rollout.• Determine and set threshold values to watch – initial hours.• Gather statistics over medium intervals during initial rollout.• Determine and set threshold values for comfortable limits for

that day (8am-11am, 11am-2pm, 2pm-5pm, 5pm-8pm).• Gather statistics over longer intervals during initial rollout.• Determine and set thresholds values based upon daily totals, then based upon each daily pattern for that week.• Monitor thresholds for THIS APP should now be in place.

• Use same feedback loop in early production monitoring for each unique application !!!



Infrared360 Provides:

• Collaborative visibility to business applications environments• Short, mid, long term threshold alerting• Real-time middleware object detail statistics visibility• Comparative statistical charting of real-time middleware objects• Real-time administrative interface to problem area for quick problem resolution • Self-healing capabilities ties alerts to pushed Admin changes• Automated replay and volume testing of real transactions• Instant visibility to application environment without pre-set monitoring infrastructure

for Enterprise Middleware Environments

Infrared360 Value Proposition


Feedback Monitoring Best Practices

Thank you!

Compartmentalized Common Operational Picture Sc2 Op

Documents

Transcript of Compartmentalized Common Operational Picture Sc2 Op