March 2007Copyright 2007, RCI1 Top Ten Risks in Net- Centric Systems Donald J. Reifer Reifer...

March 2007 Copyright 2007, RCI 1

Top Ten Risks in Net-Centric Systems

Donald J. ReiferReifer Consultants, Inc.

P.O. Box 4046Torrance, CA 90510

Phone: (310) 530-4493Email: [email protected]

mailto:[email protected]


Introduction• Identify the “top ten” risks• Discuss what you can do

about them• Highlight others’ experiences• Recommend that you tackle

risks head-on• Tell you that “even with all

the headaches that the move to net-centric systems is worth the pain”


Net-Centric Defined • Net-centric - Participating as a part of a continuously-

evolving, complex community of people, devices, information and services interconnected by networks to achieve optimal benefit of resources and better synchronization of events and their consequences. – Involves more than interfaces and interoperability

– Views the network as an enabler and distribution channel

– Built on concepts like industry-wide standards and service-oriented architectures

– Facilitates collaboration and real-time access to knowledge

– Security, availability and quality of service directly impact the warfighter’s ability to succeed on the battlefield

Source: Wikipedia


Net-Centric Creates New Challenges

Commands

Navigation DataTargeting Data

Tracking Info

CooperativeEngagement

Data

Navigation Data

Shipboard CIC

COTS-BASED


Top Ten Risks (When Addressing these Challenges)

1. Poor system engineering

2. Inadequate security engineering

3. COTS incompatibilities

4. Impossible schedules

5. Malicious code nightmares

6. Incompatible/ immature processes

7. Unforeseen/unfunded requirements

8. Supply chain surprises

9. Unpredictable quality of service levels

10. High maintenance costs

Source: review of twelve major recent programs


1. Poor Systems Engineering• Risks

– Poorly defined operational requirements/architecture

– Inadequate attention given to non-functional details

– Lack of early attention on test and integration needs

• Mitigation Actions– Develop system use cases

(sys-ML) and scenarios• Map to capability

requirements


More Mitigation Actions• Embrace service-oriented

architectures to facilitate sharing and commonality

• Allocate non-functional requirements via end-to-end scenarios as quantitatively as possible using threads

• Consider elevating information operations to the subsystem level

• Tie threads to operational scenarios and then use them as the basis for regression test development

Source: Sun


2. Inadequate Security Engineering• Risks

– Networks unreliable– Networks subject to

infections/outages• Mitigation Actions

– Elevate Information Operations (IO) to subsystem status early in life cycle

– Design security into the products

– Focus on high return activities

– Address vulnerabilities early in design stage

Many Security Threats• Poor engineering practices• Regulatory hell

– New requirements/no money• Supplier chain vulnerabilities

– Rootkits and other malicious code attacks

• Bad guys getting smarter• New types of attacks

– Use of bots as a Parasitic malware (malicious code)

• Mobile threats in shared networks

• Evolving convergence threats


Vulnerabilities Everywhere You Look• Hardware vulnerabilities

– RFID used to track movement

– Physical security barriers– Protected links– Access controls/video

surveillance

• Software vulnerabilities– Authentication mechanisms– Principle of least privilege– Identity protection/

biometrics– Behavior patterns/alerts

• Network vulnerabilities– Authentication mechanisms– Traffic/usage patterns/alerts– Voice loggers/Voice-over-IP– Vulnerability scans/

penetration testing– Wireless stack protection– Properly configured firewalls

and intrusion detection devices

– Web filters/gateway restrictions

– Security settings updated as patches tracked and logged


More Mitigation Actions• Automation vulnerabilities

– Controls isolated, monitored and authenticated

– Synchronization by variable time clocks

• Tampering vulnerabilities– Difficult to reverse engineer

– Critical Program Information (CPI) protected

– Difficult to tamper with

• Incident handling & response– Strategies to recognize attacks

and pro-actively deal with them

• Budget for security– Involves much more than

administrative functions

– Cost for equipment and software is substantial

– Focus is on keeping networks operational

• Staff for security– Requires skilled engineers

who understand tradeoffs

• Implement a security as part of your culture– Lives are at stake


3. COTS Incompatibilities• Risks

– Suppliers unresponsive

– Plug-and-play becomes patch-and-pray

– Performance plagues

– High maintenance costs

– Licensing nightmares

• Mitigation Actions– Adopt and promote a modern,

standards-based architecture

– Make suppliers vested members of the team


More Mitigation Actions• Embrace those standards, both

current and future, that best support your needs– Maintain watch on evolving standards;

try to influence their development

• Address dynamic interplay between COTS systems used across the network by focusing on services that implement plug-and-play

• Understand the games vendors play

• Team with those COTS vendors that you trust and can work with


4. Impossible Schedules• Risks

– Scheduled operational need date impossible to meet

• Nobody has the guts to admit it

– Schedules do not focus on interim capabilities

• Mitigation Actions– Incrementally develop using

capability builds and spirals

– Conduct incremental demos to engage user community and maintain their confidence


More Mitigation Actions• Run models to determine if

the schedule continues to remain feasible – If not, adjust capability and

build plan accordingly• Monitor progress; assess

status; conduct periodic assessments to determine if schedules remain feasible

• Remember, the easiest way to proceed according to your schedule is to lie to yourself– Avoid this by believing the

metrics and indicators

ISSUES• Interfacing/integrating

with legacy systems always takes longer than expected

• Reuse of legacy turns into a pipe-dream

• Networks require you to pay constant attention to testing and refactoring

• Testing networks forces you to upgrade your bench and develop regression test baselines


5. Malicious Code Nightmares• Recognize that defense-in-depth

and at the perimeter still leaves holes in network defenses

– GOTS, COTS and ROTS often riddled with malware

• Much of this is unintentional

– New vulnerabilities occur hourly that must be addressed

• Mitigation Actions– Check for malicious code in

GOTS, COTS and ROTS


More Mitigation Actions• Whenever possible, use components that

are on the certified products list (per common criteria)

• Keep all of your critical software up-to-date• Address false alarm rates by properly

configuring your intrusion detection and/or prevention devices

• Keep abreast of new and known vulnerabilities by monitoring the CVE

• Initiate alerts/alarms using a situation awareness display via the network operations center

• Design your systems to prevent insider as well as outsider attacks


6. Incompatible/Immature Processes• Risks

– Current CMMI-compatible processes do not address many of the processes used for network-centric warfare

– C&A and DIACAP add time and effort to the mix

– Supply chain dynamics may not be in synch

• Mitigation Actions– Employ agile processes for

integration and test


More Mitigation Actions• Focus on the processes that drive your

cost and schedule• Recognize that common processes for

the “net-centric” must be agreed upon by its many sources

• Try to manage the network as it evolves using a annual build and release process

• Use a demo-driven process to increase confidence in the releases and reduce risk

• Keep players involved via an ICWG• You cannot survive without common

CM/QC practices


Size Drivers

Exponential Scale Factors

Systems ofSystems

Definition and

IntegrationEffort

Calibration

• Interface-related equivalent KSLOC

• Number of logical interfaces at SoS level

• Integration simplicity• Integration risk

resolution• Integration stability• Component readiness• Integration capability• Integration processes

COSOSIMO Operational Concept

COSOSIMO

Source: Jo Ann Lane, University of Southern California, 2006


7. Unforeseen/Unfunded Requirements

• Risks– New opportunities/threats lead

to new requirements– Legacy and reuse shortfalls– More interfaces to mechanize

than anyone thought – Standards will change as will

interface specifications

• Mitigation Actions– Budget for a fixed level of

volatility and change


More Mitigation Actions• Maintain backup plans to cope with

legacy and reuse shortfalls • Incorporate changes into your build

plans– Prioritize capabilities/incrementally deliver

• Seek additional funds when changes are needed to support development of needed capabilities

• Maintain “visible” reserves to address contingencies

• Do not be a good guy and do things for others for nothing

• Manage your resources tightly


8. Supply Chain Surprises• Risks

– Other government agencies may not live up to their responsibilities

– Technology refresh

– Vendor surprises

• Mitigation Actions– Maintain technology and vendor

watch functions

– Develop Plan B’s (and C’s)


More Mitigation Actions• No matter what you do to prevent it,

supplier issues will dominate– Vendors may go out of business – Government supplier may not live up to

their obligations• Negotiate a two-tier support agreement

with critical vendors• Look for fallback positions for critical

items• Maintain good relationships with your

suppliers• Be capable of maintaining government

software organically as a last resort


9. Unpredictable Quality of Service Levels

• Risks– Poor performance (real or perceived)

– Unreliable service (real or perceived)

– High false alarm rates

• Mitigation Actions– Define quality of service expectations

– Define metrics and measures that quantify your expectations

– Develop benchmarks/assess usage

– Identify heavy usage profiles/patterns


Metrics Important to Network Operations

Percent of population thinking metric is important

1 10 100

Accuracy

Availability

Delay

Latency

MTBF

MTTR

Response Time

Thruput

Other

Enterprise

Service


10. High Maintenance Costs• Risks

– Lack of sufficient resources during maintenance

– Service degradation and lack of support

– Finger-pointing over who bears responsibility

• Mitigation Actions– Develop WBS for net-centric ops

– Adequately budget for WBS tasks

– Recognize COTS/GOTS is not free


Why Worry about NCO Risks?

Rand, Net-Centric Ops Case Study, Stryker Brigade Combat Team, 2005

Light Infantry Brigade

Stryker

Brigade

Quality of individual and shared information

- 10% - 80%

Speed of command 48 hours 3 hours

Ability to control the speed of command

No Yes

Blue-Red Casualty Ratio 10:1 1:1


Summary and ConclusionsSummary• We identified the “top ten”

net-centric risks and how to mitigate them

• In doing this, we highlighted past experiences and showed you how to capitalize on them to reduce risk

• We highlighted the fact that even though there was a lot of pain involved that the move was worthwhile

Conclusions• The path to network-centric

warfare is paved but has many potholes

• Most of the risks discussed are managerial– Sound management practices

should help eliminate them

• Many of the risks are inherent in any large system development– The mitigation actions

discussed however are not


Today’s Net-Centric Trends Lead to Tomorrow’s Challenges

• Today’s Trends– Convergence

– Interoperability

– Legacy operations

– Service-oriented architecture-based

• Applications servers

• Publish and subscribe protocols

• Standards-based

– Spiral acquisition

• Tomorrow’s challenges– Security

– Operational architecture

– Layered frameworks

– Net-ready standards• Uniform measures of

performance

• Measures of effectiveness

• Measures of readiness

• Measures of compliance

– Spiral incentives


Parting Thoughts• If development/fielding of

such networked-centric systems were easy, everyone would have them– Would not be a differentiator

in the battlefield

• The future battlefield raises the ante because getting enough bandwidth and performance remain major issues – Still lots of fun to have

solving these problems

• Many other challenges– Information overload

– Appropriate levels of automation

– Adaptive automation

– Distributed decision-making and team coordination

– Decision biases

– Mitigating complexity

– Security

– Trust and reliability

– Accountability


Contact Information

Donald J. Reifer, PIReifer Consultants, Inc.Phone: (310)[email protected]

mailto:[email protected]

March 2007Copyright 2007, RCI1 Top Ten Risks in Net- Centric Systems Donald J. Reifer Reifer...

Documents

Transcript of March 2007Copyright 2007, RCI1 Top Ten Risks in Net- Centric Systems Donald J. Reifer Reifer...