Traffic Records Assessment Assessor Training

October 2015

2

What & Where are the Criteria?

The Traffic Records Program Assessment Advisory:

• Provides guidance on the necessary contents, capabilities, and data quality measurements for a comprehensive traffic records system,

• Describes an ideal traffic records system, one that supports high-quality decisions that enable cost-effective improvements to highway and traffic safety,

• Poses a uniform set of questions that reveals the performance of the State traffic records system relative to the ideal.

3

Scope of the Assessment

• TRCC Management

• Strategic Planning

• Data Use & Integration

• Six Core Data Systems:

CRASH DRIVER ROADWAYVEHICLE CITATION/ADJUDICATON

INJURYSURVEILLANCE

4

Assessment QuestionsTRCC Management 19Strategic Planning 16Crash 44Driver 45Vehicle 39Roadway 38Citation / Adjudication 54Injury Surveillance 123*Data Use & Integration 13

Total 391* Injury Surveillance now includes sub-sections on EMS, Emergency Room, Hospital Discharge, Trauma Registry, and Vital Records

5

Standards of Evidence

The Advisory supplies a standard of evidence for each question.

• Describes the information needed to support State assertions that a traffic records system possesses the specific capability referred to in the question

• Determined by subject matter expert panelswith State, Federal, academic, and otherrepresentation.

6

Assessment Ratings

Upon review, assessors will make a determination for each question that reflects how the State’s traffic records systems are performing relative to the ideal detailed in the Advisory.

• MEETS the description of the ideal traffic records system

• PARTIALLY MEETS the description of the ideal traffic records system

• DOES NOT MEET the description of the ideal traffic records system

Assessment ScheduleFinalized Schedule Holiday

Sun Mon Tue Wed Thu Fri Sat

November 26 27 28 29 30 31 1

2 3 4 5 6 7 8 STRAP Training

9 10 11 12 13 14 15 Kickoff Meeting

16 17 18 19 20 21 22 Round 1: Data Collection

23 24 25 26 27 28 29

December 30 1 2 3 4 5 6 Round 1: Analysis

7 8 9 10 11 12 13

14 15 16 17 18 19 20 Round 2: Data Collection

12 22 23 24 25 26 27

January 28 29 30 31 1 2 3

4 5 6 7 8 9 10

11 12 13 14 15 16 17 Round 2: Analysis

18 19 20 21 22 23 24

25 26 27 28 29 30 31 Round 3: Final Data Collection

February 1 2 3 4 5 6 7

8 9 10 11 12 13 14 Round 3: Final Analysis

15 16 17 18 19 20 21

22 23 24 25 26 27 28 Facilitator Finalizes Report

March 1 2 3 4 5 6 7 Final Report Submitted

8 9 10 11 12 13 14

15 16 17 18 19 20 21 Report Out

8

Time and Knowledge Needed

Expect to spend no more than 40 hours working over the course of an assessment for an average module. (40-50 questions)

What qualifies someone as an expert? • “Expertise is a broad knowledge of how a system or process

works; skill, training and experience can give a person a level of expertise, but the ability to analyze various aspects of a program, solve problems and offer solutions is central to being a subject matter expert. This doesn't mean you have to know all there is to know about a subject, more that you are aware of the broader issues such as management principles that foster improvement and excellence.”

9

Average Assessor Time in STRAP

10

Average Hours Per Phase

9.7

5.5

6.1

National Average Assessor Time Spent by Phase

Round 1Round 2Round 3

STRAP Process Flow

State GR submits written request to NHTSA Region

State Respondents answer assigned questions & supply evidence

NHTSA Region forwards written request to NHTSA TR Team

Assessors review answers & evidence; provide findings & ratings

NHTSA TR Team confirms State request & schedules calls

NHTSA TRA team hosts initial call (~4 months prior to kickoff)

Facilitator leads ASSESSMENT KICKOFF MEETING

State Coordinator sends Respondent info & assigns questions

STRAP Tech Support hosts State Coordinator training webinar

STRAP Tech Support Sends State Coordinator STRAP tokens

STRAP Tech Support Launches Data Collection & Analysis Phases

Facilitator hosts call (1 month prior to kickoff)

STRAP Tech Support Sends Respondent STRAP tokens

STRAP Tech Support sends Assessor STRAP tokens

Facilitator review; forwards to NHTSA TR Team

NHTSA TR Team generates Final Report; sends to State

Facilitator leads ASSESSMENTREPORT OUTWEBINAR

Assessors confirm final findings, ratings and summaries

x3

Answers that meet the ideal are not returned to respondents in subsequent rounds

NHTSA TR Team

Facilitator

State Leadership

State Respondents

NHTSA Region

Assessors

STRAP Support

State Traffic Records Assessment Process (STRAP)

STRAP Overview

13

STRAP Overview

STRAP Overview

15

STRAP Overview

16

STRAP Overview

17

STRAP Overview – Module Leader

18

STRAP Overview – Respondent View

19

Sample Q & AModule: TRCC1. Does the State have both an executive and a technical TRCC?Evidence Requirement: Provide a charter and/or MOU. Also provide a roster with all members' names, affiliations, and titles for both the executive and technical TRCC.

State Response: Yes the State's TRCC, is organized as a 3-level tier. The top tier is the Executive level, the second tier is the technical level and the third are the working groups created to address current projects or challenges on a working level. See the attached TRCC charter and executive level roster.

Rating: Meets the Advisory Ideal - Both an executive TRCC and a technical TRCC are in place, and detailed rosters illustrating the structure at each level have been provided.

20

Advice• DO communicate with your co-assessor and/or Module

Leader. (Printouts)

• DON’T use outside information for Ratings/Findings.• Use only information provided by respondents.

• DO use outside information for clarification requests.• Ask the respondent to confirm/corroborate potential outside

information.

• DO use outside information for Module Summaries.• When applicable, note what was provided in-system did not match

external knowledge, but ratings had to be made based on what was actually submitted.

21

Advice

• DON’T restate the question or evidence requirement in the ballot and/or finding.

• DO Provide enough information so someone reading the report at a later date will understand how and why you reached your conclusion.

• DON’T write “State says no.”• Instead write: "The State's crash and driver systems are not linked."

22

Final Report

The result of the assessment is a report with the final question ratings and findings.

Each Module leader also creates a Module Summary similar to the previous assessment that lays out the opportunities and strengths for each module.

23

Final Report

Crash

Vehicle

Driver

Roadway

Citation /

Adjudication

EMS / Injury Surveillance

Description and Contents 96.4% 100.0% 70.0% 100.0% 77.2% 76.5%

Applicable Guidelines 100.0% 100.0% 100.0% 83.3% 75.4% 68.4% Data Dictionaries 73.3% 100.0% 91.7% 80.0% 79.4% 70.0%

Procedures / Process Flow 47.9% 87.9% 97.1% 83.3% 63.0% 67.2%

Interfaces 66.7% 97.0% 90.5% 100.0% 33.3% 42.9% Data Quality Control

Programs 44.2% 53.7% 47.9% 86.8% 46.2% 47.4%

Overall 64.3% 77.6% 77.0% 89.0% 63.0% 58.6%

Overall

Traffic Records Coordinating Committee 88.0% Strategic Planning for the Traffic Records System 76.2%

Data Use and Integration 52.5%

Recommendations are now automatically generated based on the score each module receives. Scores are calculated using a combination of the question importance and rating.

24

National Ratings

25

Becoming an Assessor

If you would like to be considered as an assessor for future assessments of other States’ traffic records systems, please fill out the form provided at the back of the room.

The form and other information is also available at:http://www.tsass.com/opportunities.html

All names are sent to NHTSA for vetting and then put into the Subject Matter Expert Pool for selection.

Questions?