The AASHTO Guide to Systems Operations & Management NCHRP 3-94 Steve Lockwood – PB Phil Tarnoff...

The AASHTO Guide toSystems Operations & Management

NCHRP 3-94Steve Lockwood – PB

Phil Tarnoff – University of MarylandRich Margiotta, Erin Flanigan – CSI

John Conrad – CH2MHill

Scott Rawlins – SSOM and Panel Chair

Basic Mitigation Strategy

2

Effective Strategies are evolving to maximize performance of existing system

Impact of Operations Strategy (Best Practice) on Total Delay

Flow control/ramp metering

5-6%

Traffic responsive signals

1%

Incident management 5-6%

Work zone traffic management

1-2%

Weather information 1%

Traveler information 1%

These impacts are comparable to many capacity investments and can be applied on a large proportion of the total network.

Causes Solutions

2

3

State of the Practice is Advancing1. Performance Management critical for maximizing

mobility – especially as capacity lags

2. SO&M will significantly reduce delays and disruptions

3. Large gap between average applications and best practice

4. Review of DOT experience: absence of appropriate processes & institutional arrangements is a major barrier to effective SO&M

5. A formalized core SO&M program is the key to success

3

4

Supporting Effectiveness Through Maturing Processes and Institutional Arrangements

ProgramA needs-responsive, performance-driven, comprehensive C/E statewide SO&M program

ProcessesThe business processes and systems required to facilitate program qualities above

InstitutionsThe values, capabilities and arrangements and resources required to support and sustain of the required business process

ProgramA needs-responsive, performance-driven, comprehensive C/E statewide SO&M program

ProcessesThe business processes and systems required to facilitate program qualities above

InstitutionsThe values, capabilities and arrangements and resources required to support and sustain of the required business process

SO&M Program PerformanceA needs-responsive, performance-driven, comprehensive

and cost-effective statewide SO&M program

Necessary Technical and Business ProcessesBusiness & technical processes and systems required to facilitate

program qualities above

Supportive Institutional & Organizational ArrangementsThe values, capabilities and arrangements and resources required to support and sustain of the

required business process

4

5

The Need for SO&M “Green Book”• Objective: maximizing the effective use

(performance) of existing system

• Research shows that top management actions regarding processes and institutional arrangements are key to effectiveness

• Guidance to Indentify steps managers can/should take designed to improve SO&M programs

• Develop existing material into an accessible and user-friendly product

• Web-based approach with self-evaluation point of departure

5

6

The Operations Capability Framework: Key Elements

Business/Technical Process Capabilities:

• Scope of Activities• Business Processes

• Technology/Systems

• Performance

Measurement

Institutional/Organizational Arrangements:

• Culture/Leadership

• Organization/Staffing

• Resources

• Partnerships

6

7

Concept of Capability Maturity Levels

Ad Hoc

Transitioning

Managed

Mainstreamed

Level 1

Level 2

Level 3

Level 4

• Some strategies• No performance• Fragmented

organization

• Full range• Processes standard • Formal program• Some measurement

• Strategies integrated• Outcomes known• Continuous improvement• Accountability

• Agency commitment

• Wide understanding• Formal program

Most of today’s agencies

Goal for the future

Today’s Best practice

Possible target

7

8

Maturity Model Framework(Has now been applied in practice)

• State DOT experience indicates that certain process & institutional features are closely linked to effective programs and their maintenance of continuing improvement

• Key is internalizing continuous and measurable improvement in customer-related performance across complete range of operations needs

• Point of departure self-evaluated

• Defines next level of arrangements to improve effectiveness (stepwise) and strategies to achieve it

8

9

Substantive Structure:PROGRAM MATURITY Needs-basis

Customer needs related

Network/context related

Support Infrastructure Sharing across

applications Functional integration

Applications Aggressiveness Technology and

procedures balance Benchmarking Continuous

improvement Comprehensiveness

and Consistency Standardization Applications

consistency Application range

PROCESS MATURITY Business Processes

Planning Programming and

budgeting Procurement Operations

Systems and Technology Regional

architectures Project Systems

engineering process Standards/

interoperability Testing and

validation Performance

Measures definition Data acquisition

management Measures utilization

INSITUTIONAL MATURITY Culture

Mission Understanding Leadership Program Status DOT Authority Continuous Improvement

Organization and Staffing Executive Level Organizational &

accountability Structure Core competencies

Resource Allocation Formal. program-level

budget estimate Sustainable resourcing from

state funds Project Investment trade

offs Partnerships

Operational roles and procedures with PSAs, LG, MPOs

Rationalize staff versus outsourcing

9

10

Levels of Maturityfor Each of the 3 Dimensions

• Level 1 (L1) - characterizes agencies that have not yet begun to develop an SO&M program

• Levels 2 (L2) and 3 (L3) - describe the level of maturity found in the top 10 percent of State DOTs

• Level 4 (L4) - an ideal target for achievement in terms of institutionalized, sustainable, continuously improving SO&M activities

10

11

Complete functions statewideConsistent application SWStandardized by typeNo standardComprehensive/consistent

Performance-drivenOptimized to locationState of the practiceNot consideredApplication aggressiveness

Cost-effective linkagesApplications optimizedIntegrated on

Systematic basis

Minimal supportSupport Infrastructure

Synergies appliedFull range by contextC/E Priorities

indentified

Ad hoc, limitedNeeds-driven

Level 4Level 3Level 2Level 1PROGRAM ELEMENTS




Systematic basis



indentified



Mission-related outcomes utilized and archived

Out put date used for evaluation and reporting

Some measurements via outputs with limited use

No performance measurementPerformance

Architectures and standards updated

SE & standards employed statewide

SE employed for conops & arch

Unique to projectTechnology and systems

Processes streamlined & updated

Processes standardized & documented

Functions and criteria for processes identified

No custom-tailored processesBusiness processes

Level 4Level 3Level 2Level 1PROCESS ELEMENTS














Agency level coordination and accountability SW

Rationalization by formal agreements and laws

MOU for specific functionsBased on temporary personal relationships

Partnerships

Operations as trade-off with other investments

Transparent sustainable line item resource allocation

Ad hoc annual allocation to

projects

Ad hoc for occasional projects

Resource allocation

Top level manager, CO & districts with accountability

Units aligned & staff professionalized

Need for coord & core capacities understood

Legacy fragmentation & limited capacities

Organization and staffing

Customer service mission key focus as core program

Visible agency leadership & operations as formal program

SO&M appreciated by key staff heroes

Value of SO&M not understood

Operations Culture

Level 4Level 3Level 2Level 1INSTITUTIONAL ELEMENTS




Partnerships




projects


Resource allocation










Operations Culture


What types of processes (planning, systems engineering, performance measurement) are needed to improve program to the next level – and to continue to improve?

Note: these are the elements: each one is broken down into components for guidance purposes

11

12




Systematic basis



indentified






Systematic basis



indentified
































Partnerships




projects


Resource allocation










Operations Culture





Partnerships




projects


Resource allocation










Operations Culture

Level 4Level 3Level 2Level 1INSTITUTIONAL ELEMENTSWhat are the leadership, organization and staff features,Resources and relationships needed to support andmaintain the needed Processes?

12

13




Systematic basis



indentified






Systematic basis



indentified
































Partnerships




projects


Resource allocation










Operations Culture





Partnerships




projects


Resource allocation










Operations Culture


13

14 14

INSTITUTIONAL DIMENSIONS

L-1: AD HOCLegacy-based

L.2: RATIONALIZEDRestructuring

L.3:MAINSTREAMEDFully Supportive

Operations Culture

Legacy—Hero-drivenOperations acknowledged, including value of reliability) but without strategic commitment or top level leadershipAdherence to legacy roles among transportation and public safety entities

Championed/Internalized across disciplinesVisible agency leadership citing Operations leverage, cost-effectiveness and risks across disciplines -- Rationalization of responsibilities by formal agreements across institutions (transportation agency, PSAs, private)

Mobility CommittedCustomer mobility service commitment accessibility accepted as core programClear legal authority for operations roles, actions among transportation agency, PSAs, Local government clarified

Organization and Staffing for Operations

Fragmented, UnderstaffedSome fragmentation of key functions and boundaries - horizontal and vertical Reliance on key individual for technical knowledge and champions for leadership

Aligning, trainedTMC focus with Vertical/horizontal authority/responsibility alignment for operations including P/B/D/C/O/MCore capacities established with KSA specs, training and performance incentives

ProfessionalizedTop level management position with operations orientation established in central office and districts Professionalization and certification of operations core capacity positions

Resource Allocation to Operations

Project -levelFunds at project level, ad hoc, unpredictableAd hoc resource allocation with operations as secondary priority

Criteria-based programBudget allocation for operations driven by transparent criteria on life cycle needs basisOperations claim on agencies’ resources for mobility support established on timing, extent, cost-effectiveness

Sustainable Budget Line ItemOperations is formal visible sustainable line item in agencies’ budget -- capital, operating and maintenanceTrade-offs between operations and capital expenditure considered as part of the planning process

Partnerships for Operations

Informal, unalignedNon-transportation entities unaligned with transportation objectives, procedures relying on informal personal basisPrivate sector utilized for isolated functions

Formal, alignedTransportation agencies assert leadership in partnerships via formal written, agreements with PSA, EM, Private sector capabilities in technology, management tapped

ConsolidatedHigh level of operations coordination among owner/operators: state, local private with TMC consolidation Clear outsourcing role developed, while maintaining agencies’ core capacities

PROCESS DIMENSIONS

L-1TRANSITIONING”

L-2MANAGED

L-3INTEGRATED

Scope Narrow and OpportunisticAd hoc operations activities based on regional initiatives, with limited central office supportNarrow/ITS-project based, low hanging fruit

Needs-based and StandardizedOperations as needs mobility- based multi-strategy program Standardized agency programs or strategies related to specific problems, desired outcomes

Full range Core ProgramFull staged program of synergizing functionalitiesOperations as key trade-off investment with other improvements in terms of “mobility management”

Business Processes

Informal, undocumentedProjects/issues handled on fire fight basis with only modest formal regional/district planning i(but no standard template)Minimal conops, architecture; procedures ad hoc/no consistency

PlannedStrategic planning and budgeting of staged improvements including maintenance and construction implicationsArchitectures and related processes developed, including major communications structure

Integrated and DocumentedIntegrated operations-related planning, budgeting, staffing, deployment and maintenance both within operations and with SW and metro planningFull documentation of key conops, architecture, procedures and protocols

Technology and Systems

Qualitative, opportunistic Technologies selected at project level Limited understanding of operating platform needs

Evaluated platformsBasic stable technology for existing strategies evaluated on qualitative basisIdentification of standardized, statewide interoperable operating platforms and related procurement procedures

Standardized, interoperableSystematic evaluation/application of best available technology/p[procedure combinationsStandard technology platforms developed/maintained

Performance Outputs reportedMeasurement of outputs only with limited analysis/remediationOutput measures reported

Outcomes usedOutcome measures measured developed and used for improvementOutcome measures reported

Performance AccountabilityContinuous improvement perspective adopted (requires intra and interagency after action analysisAccountability and benchmarking at unit and agency level via regular outcome performance reporting – internal and public

PROOF OF CONCEPT

Guidance Based on Current Best Practice and Beyond

Informal, undocumented• Projects/issues handled on fire fight

basis with only modest formal regional/district planning (but no standard template)

• Minimal concepts of operations. Systems architecture; procedures ad hoc --no consistency

15

User-based Self-EvaluationIndicate User’s position (check one): Top management (CO or District)

Program manager (CO or District)

Project manager (CO or district)

Indicate agency current state-of-play (level) regarding selected element

Measures not defined or utilized

Output measures utilized only for some activities/unlinked to policy

Output measures for all activities/Linked to policy

Select capability element as point of departure (examples)

Performance measurement

Standardization/Documentation

Sustainable budget for planning

clear lines of responsibility

Aligned partnerships with PSAs

Guidance

Detailed Strategy Guidance forselected element to move to next level

15

16

“Black Box” Determines Users current level and next steps to improved capability

CURRENT LEVEL OF PERFORMANCE MEASUREMENT MATURITY

L-1 L-2 L-2 L-3 L-3 L-4

PM program based on output measures only

PM program based on output measures and a few outcome measures only

PM program based on full set of both output/outcome measures, linked to agency operations

16

17

Standard Strategy Templates for Each Element/Level combination (100+)

PERFORMANCE ELEMENTPerformance Measures Definition: Strategy from Level 1 to 2,

Issue Guidance

Function of component Development of performance metrics for operations activities and the impact operations activities have on users

Why important for CIP Development of an effective operations program must be based on matching needs to solutions. Performance measurement does this by cost-effectively directing operations investments. Having an ongoing performance process means that new deployments and policies can be continually evaluated, rather than having to undertake piecemeal evaluations.

Strategy (from Layer 3) Identify output performance measures for the selected operations activities

Detailed Description of activity to move up a level

A. The philosophy here is, “if you’re going to manage it, you’re going to measure it”. For Level 1 to Level 2, the limited number of operational activities that will be monitored must first be identified. For Level 2 to 3, all operations activities engaged in will have performance measures defined for them. Output measures for both Level transitions should be relatively simple and easy to collect. These are often referred to as “activity based” measures since they monitor the extent of activities undertaken, and their immediate consequences. Therefore, the measures selected will include at a minimum:incident durationincident and work zone characteristics (number, type, severity)operational activity (website hits, service patrol stops, messages)equipment locations and statusequipment downtimeResponsibility: Operations staffRelationships: N/AReferences: Guide to Benchmarking Operations Performance Measures, http://www.trb.org/TRBNet/ProjectDisplay.asp?ProjectID=1218NCHRP Web-Only Document 97, "Guide to Effective Freeway Performance Measurement: Final Report and Guidebook", http://www.trb.org/news/blurb_detail.asp?id=7477

17

http://www.trb.org/TRBNet/ProjectDisplay.asp?ProjectID=1218

http://www.trb.org/news/blurb_detail.asp?id=7477

18

Process is repeated for next element of capability

• “Model” uses user self-evaluation to determine current level

• Model defines next steps (based on current best practice

• User only sees material related to her agency level

Business & Technical Process Capabilities:

• Scope of Activities• Business Processes

• Technology/Systems

• Performance Measurement

Institutional/Organizational Arrangements:

• Culture/Leadership

• Organization/Staffing

• Resources

• Partnerships

18

19

Benefits of Web-Based Approach• Avoids lengthy paper documents with big

charts

• Relationships among elements built in

• Custom tailored to user

• Users self-evaluate SDOT state of play• Increasing levels of detail displayed on

automated basis• Hyperlinks to supporting documents

19

20

Need for Senior State DOT Input

• Importance of SO&M to your program?

• Your management objectives related to SO&M?

• Principle challenges you see

• Lessons you have learned to date

• Utility of Guidance to you

We look forward to follow-up with you20

The AASHTO Guide to Systems Operations & Management NCHRP 3-94 Steve Lockwood – PB Phil Tarnoff...

Documents

Transcript of The AASHTO Guide to Systems Operations & Management NCHRP 3-94 Steve Lockwood – PB Phil Tarnoff...