The FMCSA Program Effectiveness Study

How Effective Are Compliance Reviews?

Why Measure Effectiveness?

• To address the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates federal agencies to measure the effectiveness of their programs as part of the budget cycle process

• To give FMCSA a means to measure how much of an impact the Compliance Review program activities have on avoiding crashes involving motor carriers and reducing the numbers of resulting injuries and fatalities

What Is a Compliance Review?

• An on-site examination of a motor carrier’s records of operation (including driver log books, maintenance and inspection records, driver qualification records, controlled substance and alcohol testing data,, financial responsibility information, crash data, hazardous materials documentation, etc.) to determine whether a motor carrier meets a safety fitness standard.

• FMCSA expects that through the Compliance Review (CR) process, carriers will improve the safety of their commercial vehicle operations, and, ultimately, reduce their crash rates.

Question:How does one measure the effectiveness of Compliance Reviews?

Answer:Examine the crash history of carriers both prior to, and after the CR process.

What Is the Compliance Review Impact Assessment

Model?An analytic model designed to measure the effectiveness of the Compliance Review in terms of:

– Crash Reduction = Crashes Avoided

– Injury Reduction = Injuries Avoided

– Fatality Reduction = Lives Saves

CR Impact Assessment Model Overview

The CR Impact Assessment Model – Shows the direct impact of compliance reviews on carrier

safety– It is based on “before and after” changes in the safety

performance of carriers that received CRs– It compares a motor carrier’s crash rate in a time period after

an on-site compliance review to its crash rate prior to that review

CR Impact Assessment Model Data Collection

• The model uses crash and mileage data collected during compliance reviews, and later stored in the FMCSA’s Motor Carrier Management Information System (MCMIS)

• During the CR, investigators obtain the number of recordable crashes in which the carrier was involved over the past 12 months, and the number of vehicle miles traveled (VMT)

• From these data, a “pre-Compliance Review” crash rate is calculated for all carriers receiving reviews.

CR Impact Assessment ModelCR Follow-up Survey Data

• To perform a “before and after” comparison, post-Compliance Review data are obtained from a Follow-up Survey.

• The Follow-up Survey collects motor carrier mileage and crash information for the 1-year period after the review.

• The percentage reduction in the recordable crash rate is then calculated for all carriers.

CR Impact Assessment ModelEffectiveness Calculations

• The number of crashes avoided the following calculation is performed:

Pre-CR average crash rate x crash rate reduction x VMT

• Historical data are then used to breakdown the total crashes avoided into the following categories: fatal crashes avoided, injury crashes avoided, and towaway avoided crashes.

• These numbers are then translated into injuries avoided and lives saved, also based on historical data.

How Effective Were the Compliance Reviews Conducted in 2000?

How Effective Were the Compliance Reviews Conducted in 2001?

Historical Effectiveness of Compliance Reviews

Based on CRs conducted in: 1998 1999 2000 2001

Number of Carriers Receiving CRs 6,055

8,877

11,340

8,924

Total Vehicle Miles Traveled (VMT) (million miles)

13,844

17,409

22,610

18,455

Pre-CR Average Crash Rate (crashes/million VMT)

.823 .804 .757 .715

Model Results Estimated for: 1999 2000 2001 2002

Total Number of Crashes Avoided 1,200 1,500 2,200 1,600

Number of Towaway Crashes Avoided 677 846 1,241 902

Number of Injury Crashes Avoided 480 600 880 690

Number of Fatal Crashed Avoided 43 54 79 58

Number of Injuries Avoided 822 1,028 1,395 1,105

Number of Lives Saved 51 64 91 67

Where Can I Find More Information on the FMCSA Program Effectiveness

Study?• The FMCSA website, A&I Online (www.ai.volpe.dot.gov),

presents information on the CR Impact Assessment Model. Access or download:

• Detailed Reports• One Page Summaries of the Model• Model Results• Overview Presentations (such as this presentation)

• Or contact Mr. Richard Gruberg, FMCSA Analysis Division, at 202-366-2959 or richard.gruberg@fmcsa.dot.gov for copies of the reports and related information about the Program Effectiveness Study

Where Can I Find Program Effectiveness on A&I Online?

• The A&I Program Measures Module includes Program Effectiveness as a sub-module