June 2006
FMCSA-RI-06-043
Prepared for:
Federal Motor Carrier Safety Administration
Office of Research and Analysis
Analysis Division, MC-RRA400
Seventh Street, S.W.
Washington, DC 20590
Prepared by:
John A. Volpe National Transportation Systems Center
Office of Surface Transportation Programs
Motor Carrier Safety Division, RTV-3E
Kendall Square
Cambridge, MA 02142
This report documents the methodology and results from a model, called the Compliance Review (CR) Effectiveness Model, that measures the effectiveness of one of the key safety programs of the Federal Motor Carrier Safety Administration (FMCSA), the compliance review. The model was developed for the FMCSA by the Research and Innovative Technology Administration's (RITA) John A. Volpe National Transportation Systems Center (the Volpe Center) in Cambridge, MA. This work is part of an effort to assess the effectiveness of the FMCSA's principal safety programs. The work also addresses the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates federal agencies to measure the results of their programs as part of the budget cycle process.
The CR Effectiveness Model is one of two models that provide a baseline of the effectiveness of FMCSA safety programs through the use of standard safety performance measures. This baseline allows the FMCSA to judge the relative performance of its programs on a periodic basis by reflecting the changes in benefits resulting from each program. The results of these analyses are also intended to provide a basis for FMCSA resource allocation and budgeting decisions that will more closely optimize the effectiveness and efficiency of its motor carrier safety programs.
In addition to the CR Effectiveness Model, another model, called the Intervention Model, has been developed to measure the effectiveness of and estimate benefits resulting from roadside inspection and traffic enforcement activities. These two models have been developed to estimate the benefits of these FMCSA safety programs in terms of crashes avoided, lives saved, and injuries avoided.
Thomas Keane, Chief of the Analysis Division in the Office of Research and Analysis, manages the project for the FMCSA. The Volpe Center project manager is Donald Wright, Chief of the Motor Carrier Safety Division in the Office of Surface Transportation Programs. The analysis was performed by Jon Ohman with assistance from Julie Nixon and Kevin Gay, all of the Volpe Center. Technical support was provided by Dennis Piccolo, Leon Parkin, and Richard Nguyen of Chenega Advanced Solutions & Engineering (CASE), LLC, under contract to the Volpe Center. Olu Ajayi of the FMCSA's Analysis Division deserves special thanks for his assistance in obtaining data that were used in the implementation of the model.
EXECUTIVE SUMMARY
1. INTRODUCTION
1.1. Background
1.2. Project Objective
1.3. Project Scope
2. COMPLIANCE REVIEW EFFECTIVENESS MODEL
2.1. Compliance Reviews
2.2. Methodology of the Model
2.3. Results of Implementation of Model for Carriers with Compliance Reviews in 2003
2.4. Extended Benefits of Compliance Reviews
2.4.1. Methodology
2.4.2. Extended Benefits of the Compliance Reviews Conducted in 2003
2.4.3. Extended Benefits of the Compliance Reviews Conducted in 2002
2.4.4. Summary of Results
3. ADDITIONAL ANALYSIS
3.1. Overview
3.2. Carrier Size
3.3. State of Domicile of Carrier
3.4. Carrier Safety Status
APPENDIX A. CALCULATION OF PRE-CR AND POST-CR AVERAGE CRASH RATES FOR CONTROL GROUP
APPENDIX B. ALLOCATION OF CHANGE IN AVERAGE CRASH RATE OF CONTROL GROUP TO CHANGE IN CRASH RATE AND OTHER UNDERLYING FACTORS
APPENDIX C. CALCULATION OF PROJECTIONS OF EXTENDED BENEFITS OF COMPLIANCE REVIEWS CONDUCTED IN 2003
ES-1. Estimated and Projected Crashes Avoided due to Compliance Reviews - 2002-2003 to 2006-2007
2-1. Compliance Review Effectiveness Model
2-2. Timeline for a Carrier with a Compliance Review on January 21, 2003
2-3. Estimated and Projected Crashes Avoided due to Compliance Reviews - 2002-2003 to 2006-2007
ES-1. Results of Implementation of Compliance Review Effectiveness Model for Carriers with Compliance Reviews in 2002 and 2003
ES-2. Crashes Avoided due to Compliance Reviews Conducted in 2002and 2003 - Estimated for 2002-2004 and Projected for 2004-2007
2-1. Results of Implementation of Compliance Review Effectiveness Model for Carriers with Compliance Reviews in 2002 and 2003
2-2. Percent Change in Average Crash Rate due to Compliance Reviews by Year and Benefit Period
2-3. Crashes Avoided due to Compliance Reviews Conducted in 2003 - Estimated for 2003-2004 and Projected for 2004-2007
2-4. Pre- and Post-CR Average Crash Rates for 2002 CR and Control Groups for Extended Benefit Period 1
2-5. Crashes Avoided due to Compliance Reviews Conducted in 2002 and 2003 - Estimated for 2002-2004 and Projected for 2004-2007
3-1. Results of Implementation of Model by Carrier Size
3-2. Results of Implementation of Model by State of Domicile of Carrier
3-3. Results of Implementation of Model by Carrier Safety Status
C-1. Percent Change in Pre-CR Average Crash Rate from Previous Year
C-2. Percent Change in Post-CR Power Units from Previous Year
C-3. Adjusted Percent Change in Pre-CR Average Crash Rate due to Compliance Reviews by Year
C-4. Crashes Avoided due to Compliance Reviews Conducted in 2002 and 2003 - Estimated for 2002-2004 and Projected for 2004-2007
This report documents the methodology and results from a model that measures the effectiveness of one of the key safety programs of the Federal Motor Carrier Safety Administration (FMCSA), the compliance review (CR) program. The research was conducted by the Research and Innovative Technology Administration's (RITA) John A. Volpe National Transportation Systems Center (the Volpe Center) in Cambridge, MA under a project plan agreement with the FMCSA. The work on the FMCSA Safety Program Effectiveness Measurement Project addresses the requirements of the Government Performance and Results Act (GPRA) of 1993, which obligates federal agencies to measure the results of their programs as part of the budget cycle process.
This report describes the methodology of the Compliance Review Effectiveness Model and presents the results of the implementation of the model for carriers receiving CRs in 2003. The benefits of the compliance review program are calculated in terms of crashes avoided, lives saved, and injuries avoided.
The on-site compliance review is perhaps the single greatest resource-consuming activity of the FMCSA. Thousands of CRs are conducted each year. In the year 2003, federal and state enforcement personnel conducted over 11,000 CRs on individual motor carriers. It is intended that through education, heightened safety regulation awareness, and the enforcement effects of the CR, carriers will improve the safety of their commercial vehicle operations, and, ultimately, reduce the number and severity of crashes in which they are involved.
The CR Effectiveness Model was developed to determine the effectiveness of the CR program. The model shows the direct impact of compliance reviews on carrier safety, but not the "deterrent" effects (i.e., the "threat" of having a CR for carriers that did not actually have a CR). The model is based on the individual and cumulative "before and after" changes in the safety performance of carriers that received CRs in a given year, e.g., year X. The model compares a motor carrier's crash rate following an on-site compliance review to its crash rate in the 12 months prior to that review. The model uses (1) crash data reported by the states and (2) power unit data reported by carriers or obtained during CRs, to calculate both before and after CR crash rates.
To eliminate the effects of underlying factors occurring in the general carrier population, a control group of carriers is selected. This Control Group consists of all carriers that did not receive CRs in year X. Any change in the average crash rate of the Control Group must be due to factors affecting the entire carrier population. Thus, the change in the average crash rate of the Control Group is calculated and then subtracted from the change in the average crash rate of the carriers that received CRs in year X. The difference resulting from this calculation represents the change in the average crash rate of the carriers that received CRs in year X that was solely the result of the CRs.
The CR Effectiveness Model has been used to estimate benefits only for carriers with CRs conducted in 2002 and 2003. The model succeeded the Compliance Review Impact Analysis Model, which was used to estimate the benefits for carriers with CRs in 1998, 1999, 2000, and 2001.1, See Endnotes The estimates produced by the CR Effectiveness Model establish new benchmarks and are not directly comparable to the estimates produced by the CR Impact Assessment Model.
The CR Effectiveness Model was implemented for carriers with CRs in 2003 to estimate the number of crashes (and associated fatalities and injuries) avoided in 2003-2004. The results for the first year following the review (2003-2004) are shown in Table ES-1. The results for 2002-2003 for carriers with CRs in 2002 are also shown in Table ES-1.
| Model Implementation for Motor Carriers with CRs in: | 2002 | 2003 |
|---|---|---|
| Compliance reviews conducted | 12,139 | 11,086 |
Motor carriers that received compliance reviews and:
|
9,172 | 8,587 |
| Estimated percentage reduction in average crash rate due to compliance reviews | 12.6 | 17.6 |
| Model Results (i.e., Benefits) Estimated for: | 2002-2003 | 2003-2004 |
| Crashes avoided | 1,426 | 2,276 |
| Fatal crashes avoided | 53 | 77 |
| Injury crashes avoided | 677 | 1,038 |
| Towaway crashes avoided | 696 | 1,161 |
| Lives saved | 62 | 90 |
| Injuries avoided | 1,087 | 1,651 |
It is likely that these benefits continue beyond the first year following the review based on a study being conducted at the Volpe Center in which a continued lower crash rate was observed for carriers receiving reviews in prior years. It appears from this study that there was a long-term reduction in the average crash rate of carriers that receive CRs lasts for several years after the CRs. (See below under "Extended Benefits of Compliance Reviews.")
The current methodology of the CR Effectiveness Model only estimates the benefits that occur in the 12 months following the CRs in a given year, i.e., the initial benefits. Based on the results of research being conducted at the Volpe Center, however, it appears that the reduction in the average crash rate of carriers continues for several years after the CRs. For this report, three groups of carriers were studied: (1) carriers that received CRs in 2000, (2) carriers that received CRs in 2001 and (3) carriers that received CRs in 2002. The average crash rate and the number of power units for each group was tracked over time using state-reported crash data and power unit data reported by carriers or obtained during CRs, which are the same data used in the CR Effectiveness Model.
For carriers with CRs in 2000 and 2001, the amount of crash rate reduction increased for each benefit period. For carriers with CRs in 2002, however, the crash rate reduction in extended benefit period 1 (i.e., the second 12 months after a carrier's CR) was less than the crash rate reduction in the initial benefit period (i.e., the 12 months after a carrier's CR). A methodology was developed to closely replicate these results and to project results for future years. This methodology involves projecting (1) the change in the average crash rate in each year and (2) the number of post-CR power units in each year. The projected number of crashes avoided is then calculated for each year using the same formula used in the CR Effectiveness Model.
Preliminary projections of the numbers of crashes avoided by carriers that received CRs in 2003 have been made for future periods (i.e., 2004-2005, 2005-2006, and 2006-2007). These projections should be regarded only as preliminary indicators of the extended benefits of compliance reviews rather than as official estimates. The research on this topic is still in progress. As more years of data become available for analysis and verification, this methodology will be refined to produce more accurate projections.
In addition, an estimate of the number of crashes avoided by the carriers that received CRs in 2002 has been calculated for the period 2003-2004. This estimate updates the projection of this quantity that was made in the report that described the implementation of the model for carriers with CRs in 2002,2, See Endnotes which was published in May 2005.
Table ES-2 shows the results of the implementation of the CR Effectiveness Model for CRs conducted in 2002 and 2003. Table ES-2 shows the latest estimates and projections of crashes avoided due to the CRs conducted in the two years, 2002 and 2003, for which the model has been run. Figure ES-1 presents a bar graph of the results.
| Compliance Reviews Conducted in | Crashes Avoided | ||||
|---|---|---|---|---|---|
| 2002-2003(Estimated) | 2003-2004(Estimated) | 2004-2005(Projected) | 2005-2006(Projected) | 2006-2007(Projected) | |
| 2002 | 1,426 | 1,171 | 1,475 | 1,526 | |
| 2003 | 2,276 | 1,876 | 2,191 | 2,531 | |
To further assess the effectiveness of the compliance review program, the preliminary results of the implementation of the model were broken out by (1) carrier size (i.e., number of power units), by (2) the state of domicile of the carrier and by (3) carrier safety status (i.e., the carrier's SafeStat3, See Endnotes category before receiving its CR in 2003).
During the 1980s, Congress passed several acts intended to strengthen motor carrier safety regulations. This led to the implementation of safety-oriented programs both at the federal and state levels. The Surface Transportation Assistance Act of 1982 established the Motor Carrier Safety Assistance Program, a grants in aid program to states to conduct roadside inspection and traffic enforcement programs aimed at commercial motor vehicles. The 1984 Motor Carrier Safety Act directed the U.S. Department of Transportation (U.S. DOT) to establish safety fitness standards for carriers. The U.S. DOT, in conjunction with the states, implemented the Motor Carrier Safety Assistance Program (MCSAP) to fund the roadside inspection and traffic enforcement programs and the safety fitness determination process and rating system (based on on site safety audits called compliance reviews).
It is expected that a major benefit of these programs has been and will continue to be an improved level of safety in the operation of commercial motor vehicles. Previously, however, there was no means to measure the benefits and effectiveness of these programs. The Safety Program Effectiveness Measurement Project was established to identify major functions and operations (programs) associated with the FMCSA mission and to develop results-oriented performance measures for those functions and operations, as called for in the Government Performance and Results Act (GPRA) of 1993.
Program evaluation should be viewed as a continuous management process that encourages the organization to reflect periodically upon how it is implementing its programs. Program effectiveness should be reassessed in light of the mission, available resources, changing requirements, political climate, technological change, public demands, and costs. Periodic review of the results of the evaluations will ensure that the activities are working, i.e., that they are delivering what was promised. This report is intended to satisfy the desire of the FMCSA to verify the effectiveness of one of its motor carrier safety programs, the compliance review program. The immediate objective of this effort is to measure how much of an impact the safety program activities have on avoiding crashes involving motor carriers and reducing resulting injuries and fatalities.
One of the main objectives of the Safety Program Effectiveness Measurement Project is to provide a baseline of the effectiveness of the selected programs through the use of standard safety performance measures. This baseline allows the FMCSA to judge the relative performance of its programs on a periodic basis by reflecting the benefits resulting from each program. The results of these analyses are intended to provide a basis for FMCSA resource allocation and budgeting decisions that will more closely optimize the effectiveness and efficiency of its motor carrier safety programs.
The scope of this overall effort is limited to the major identifiable operational FMCSA programs and their effectiveness in reducing crashes and avoiding injuries and fatalities. Currently the Safety Program Effectiveness Measurement Project includes the compliance review, roadside inspection, and traffic enforcement activities and programs performed and supported by the FMCSA. Two models have been developed to estimate the benefits of these programs: the Compliance Review Effectiveness Model and the Intervention Model (for roadside inspections and traffic enforcements). The benefits of these programs are calculated in terms of crashes avoided, lives saved, and injuries avoided.
An objective of the project is to continue to improve these models and update the results on a recurring basis. The models will serve the program-specific requirement to measure program effectiveness as well as the broader function of supporting annual budget requirements and helping to determine the best resource allocation among program elements.
This report describes the methodology of the Compliance Review Effectiveness Model and presents the results of the implementation of the model for carriers receiving compliance reviews (CRs) in 2003, including estimates of crashes avoided by carrier size and state of domicile. The report also contains projections of the extended benefits from the CRs conducted in 2003. In addition, the report contains an estimate of the extended benefits from the CRs conducted in 2002 for the period 2003-2004. This estimate updates the projection of this quantity that was made in the report that described the implementation of the model for carriers with CRs in 20021, See Endnotes, which was published in May 2005.
The CR Effectiveness Model has been used to estimate benefits only for carriers with CRs conducted in 2002 and 2003. The model succeeded the Compliance Review Impact Assessment Model, which was used to estimate the benefits for carriers with CRs in 1998, 1999, 2000, and 2001.2, See Endnotes The results from the two models are not directly comparable, because the estimates produced by CR Effectiveness Model will establish new benchmarks, which may differ from the level of the estimates produced by the CR Impact Assessment Model.
The on-site compliance review (CR) is perhaps the single greatest resource-consuming activity of the FMCSA. Thousands of CRs are conducted each year. In the year 2003, federal and state enforcement personnel conducted over 11,000 CRs on individual motor carriers. In addition to actually conducting CRs, the FMCSA invests in: extensive analysis of the requirements of the Federal Motor Carrier Safety Regulations (FMCSR), enhancements to the design of the CR to better assess safety performance and compliance with the FMCSR, continued safety investigator training, enhancements to prioritization methodologies such as SafeStat1, See Endnotes to determine who should receive CRs, and enhancements to information systems to report and store the results of the CRs that are conducted.
When performing CRs, FMCSA and state safety investigators spend many hours examining the safety records of individual motor carriers to assess their compliance and safety performance. The investigators also discuss their findings with the carriers' safety managers to improve understanding of their safety programs. After a review is completed, the carrier is assigned a safety rating (i.e., satisfactory, conditional, or unsatisfactory). If serious violations are discovered, an enforcement case is initiated and a fine may be imposed. The CR results are also incorporated, with other safety data (i.e., crashes, roadside inspection results, moving violations, and closed enforcement cases), into SafeStat to reassess the carrier's safety status. It is intended that through education, heightened safety regulation awareness, and the enforcement effects of the CR, carriers will improve the safety of their commercial vehicle operations, and, ultimately, reduce the number and severity of crashes in which they are involved.
The CR Effectiveness Model was developed to determine the effectiveness of the CR program. The model shows the direct impact of compliance reviews on carriers that received CRs, but not the "deterrent" effects (i.e., the "threat" of having a CR) on all carriers. In addition, the model was originally developed to estimate only the benefits that occur in the 12 months following a CR. The model is based on the individual and cumulative "before and after" changes in the safety performance of carriers that received CRs. The model compares a motor carrier's crash rate in the 12 months after an on-site compliance review to its crash rate in the 12 months prior to that review. The model uses (1) crash data reported by the states and (2) power unit data reported by carriers or obtained during compliance reviews, to calculate both crash rates. The data are stored in the FMCSA's Motor Carrier Management Information System (MCMIS).
A diagram of the CR Effectiveness Model, as implemented for carriers with CRs in 2003, is shown in Figure 2-1. The model estimates the number of crashes (and associated fatalities and injuries) avoided in the 12 months following the CRs. Thus, the benefits from the CRs conducted in 2003 occurred in both 2003 and 2004.
A step-by-step description of the implementation procedure follows. The step numbers (shown in parentheses) correspond to the numbers in parentheses in the diagram.
There were 8,587 carriers that received CRs in 2003, were still active 12 months after their CRs (i.e., throughout their post-CR periods), and met the following conditions:
The 8,587 carriers that received CRs in 2003 and were still active 12 months after their CRs had a pre-CR average crash rate of 5.290 crashes per 100 power units. This average was obtained by dividing the total number of carriers' state-reported crashes in the 12 months before their 2003 CRs by the total number of carriers' power units and then multiplying by 100. The power unit data came from the MCMIS Census File. The data were obtained from compliance reviews and updated Form MCS-150 information submitted by carriers. In the rate calculation for each carrier, the power unit data were taken from the SafeStat run for the month following the carrier's CR. That way, the power unit data used in the rate calculation would reflect the power unit data collected during the CR.
The 8,587 carriers that received CRs in 2003 and were still active 12 months after their CRs had a post-CR average crash rate of 4.848 crashes per 100 power units. This average was obtained by dividing the total number of carriers' state-reported crashes in the 12 months after their 2003 CRs by the total number of carriers' power units and then multiplying by 100. The power unit data came from the MCMIS Census File. In the rate calculation for each carrier, the power unit data were taken from the SafeStat run one year after the run used to calculate the carrier's pre-CR crash rate.
For example, if a carrier had a CR on January 21, 2003, then power unit data from the February 2003 SafeStat run would have been used to calculate its pre-CR crash rate, and power unit data from the February 2004 SafeStat run would have been used to calculate its post-CR crash rate. The carrier's pre-CR period (i.e., the 12 months prior to the CR) would have been January 21, 2002 to January 20, 2003, while its post-CR period (i.e., the 12 months after the CR) would have been January 22, 2003 to January 21, 2004. This information is shown in the timeline in Figure 2-2.
4a.Calculate the reduction using the data for the carriers with CRs in 2003.
The percentage change in the average crash rate of carriers with CRs in 2003 was calculated as follows:
Percentage Change in Average Crash Rate of Carriers with CRs in 2003
Post-CR Average Crash Rate - Pre-CR Average Crash Rate4b. Adjust the reduction for underlying factors in the general carrier population.
The change in the average crash rate of the carriers that received CRs (i.e., the CR Group) was not adjusted for underlying factors occurring in the general carrier population. For example, if the average crash rate of all carriers had decreased during the same period in which the CR Group's average crash rate decreased, then the reduction in the CR Group's average crash rate calculated in Step 4a would have been exaggerated. That is, not all of the reduction would have been the result of the CRs. Conversely, if the average crash rate of the general carrier population had increased during this period, then the reduction in the CR Group's average crash rate calculated in Step 4a would have been less than the actual crash rate reduction due to the CRs.
Another underlying factor that must be considered in the analysis of carriers that received CRs in 2003 is improved crash reporting. Over the past several years, the FMCSA has made a concerted effort to improve the timeliness and completeness of crash reporting by the states. As a result, crashes are being reported earlier and more completely. This improved crash reporting will tend to increase the post-CR average crash rate and produce a smaller crash rate reduction in the CR Group's average crash rate than actually occurred.
To eliminate the effects of underlying factors, a control group of carriers was selected. This Control Group consisted of all carriers that did not receive CRs in 2003. Any change in the average crash rate of the Control Group must have been due to factors affecting the entire carrier population. Thus, the change in the average crash rate of the Control Group was calculated and then subtracted from the change in the average crash rate of the carriers that received CRs in 2003. The difference resulting from this calculation represents the change in the average crash rate of the carriers that received CRs in 2003 that was solely the result of the CRs.
To be eligible for the Control Group, a carrier had to meet the following conditions:
There were 414,998 carriers that met these criteria.
The change in the average crash rate of the Control (i.e., non-CR) Group was calculated as follows:
Percent Change Post-CR Average Crash Rate - Pre-CR Average Crash Rate
in Average = ---------------------------------------------------------------------- X 100
Crash Rate Pre-CR Average Crash Rate
The pre-CR crash rate is the average crash rate for the entire pre-CR period, i.e., 2002-2003, while the post-CR crash rate is the average crash rate for the entire post-CR period, i.e., 2003-2004. The pre-CR and post-CR average crash rates were calculated as follows:2, See Endnotes
Pre-CR Crashes in 2002 + Crashes in 2003
Average = ---------------------------------------------------------------------------
Crash Rate Power Units at the end of 2002 + Power Units at the end of 2003
Post-CR Crashes in 2003 + Crashes in 2004
Average = ----------------------------------------------------------------
Crash Rate Power Units at the end of 2003 + Power Units at the end of 2004
The 414,998 carriers in the Control Group had a pre-CR average crash rate of 1.880 crashes per 100 power units and a post-CR average crash rate of 2.054 crashes per 100 power units.
The percentage change in the average crash rate of the Control Group was calculated as follows:
Percentage Change in Average Crash Rate of Control Group
Post-CR Average Crash Rate - Pre-CR Average Crash Rate
= ------------------------------------------------------ X 100
Pre-CR Average Crash Rate
2.054 - 1.880
= ------------- X 100
1.880
= +9.26% (i.e., an increase of 9.26 percent)
This increase in the average crash rate of the Control Group (and therefore, the general carrier population) is the sum of the effects of (1) any change in the average crash rate of the general carrier population and (2) other underlying factors in the general carrier population (e.g., changes in crash reporting). To determine how much of the increase was due to each element, a separate set of calculations was performed. The calculations showed that there was a 3.49 percent decrease in the average crash rate of the general carrier population. Therefore, the 9.26 percent increase in the average crash rate of the Control Group (and therefore, the general carrier population) was the sum of a 3.49 percent decrease in the crash rate of the general carrier population and a 12.75 increase due to other underlying factors in the general carrier population (e.g., changes in crash reporting). These calculations are shown in Appendix B.
Therefore, the adjusted change in the average crash rate due to the CRs conducted in 2003 was:
Adjusted Change in Crash Rate due to CRs Conducted in 2003
= Percentage Change in Average Crash Rate of Carriers with CRs in 2003
− Percentage Change in Average Crash Rate of Control Group
= (-8.36) - (9.26)
= -17.6% (i.e., a decrease of 17.6 percent)
The 8,587 carriers that received CRs in 2003 and were still active 12 months after their CRs had a total of 244,444 power units 12 months after their CRs. This number was used to calculate the post-CR average crash rate in Step 3.
The estimated number of crashes avoided in 2003-2004 by the 8,587 carriers that received CRs in 2003 and were still active 12 months after their CRs was calculated as follows:
Crashes Avoided in 2003-2004 by Carriers with CRs in 2003
= Pre-CR Average Crash Rate X Crash Rate Reduction (%) X Post-CR Power Units
= 5.290 crashes per 100 power units X 17.6% X 244,444 power units
= 2,276 crashes
Next, estimates were made of the number of crashes avoided in 2003-2004 by the carriers receiving CRs in 2003 by severity, i.e., fatal, injury, and towaway.3, See Endnotes State-reported crash data in the MCMIS were used to compute these proportions. Of the crashes involving large trucks or motorcoaches (i.e., cross-country or intercity buses) in 2003-2004, the period in which the benefits of the CRs conducted in 2003 would occur, 3.4 percent were fatal crashes, 45.6 percent were injury crashes, and 51.0 percent were towaway crashes.
Applying these proportions to the estimate of 2,276 crashes avoided produced the following results:
Fatal crashes = 2,276 X 3.4% = 77
Injury crashes = 2,276 X 45.6% = 1,038
Towaway crashes = 2,276 X 51.0% = 1,161
The average number of fatalities per fatal crash was calculated from data from the Fatality Analysis Reporting System (FARS), which is maintained by the National Highway Traffic Safety Administration (NHTSA). The benefits of the CRs conducted in 2003 occurred in the period 2003-2004. For crashes in 2003-2004 involving large trucks or motorcoaches (i.e., cross-country or intercity buses), the ratio was 1.17 fatalities per fatal crash.
The number of injuries per crash involves fatal as well as injury crashes, since fatal crashes can also result in injuries. State-reported crash data from the MCMIS Crash File were used to compute the average numbers of injuries in fatal and injury crashes. For 2003-2004 large truck and motorcoach crashes, the averages were as follows:
The estimated number of lives saved in the crashes avoided in 2003-2004 by the carriers with CRs in 2003 was calculated as follows:
Number of lives saved in fatal crashes avoided in 2003-2004 by carriers with CRs in 2003
= Number of fatal crashes avoided X Average number of fatalities per fatal crash
= 77 X 1.17
= 90 lives saved
The estimated number of injuries avoided in the crashes avoided in 2003-2004 by the carriers with CRs in 2003 was calculated as follows:
Number of injuries avoided in crashes avoided in 2003-2004 by carriers with CRs in 2003
= Number of fatal crashes avoided X Average number of injuries per fatal crash + Number of injury crashes avoided X Average number of injuries per injury crash
= 77 X 1.09 + 1,038 X 1.51
= 1,651 injuries avoided
Table 2-1 summarizes the estimated benefits that occurred in 2003-2004 as a result of the CRs conducted in 2003 on the 8,587 carriers that were still active 12 months after their CRs and met the additional criteria listed in the table. The table also shows the estimated benefits from the CRs conducted in 2002 that occurred in 2002-2003.
| Model Implementation for Motor Carriers with CRs in: | 2002 | 2003 |
|---|---|---|
| Compliance reviews conducted | 12,139 | 11,086 |
Motor carriers that received compliance reviews and:
|
9,172 | 8,587 |
| Estimated percentage reduction in average crash rate due to compliance reviews | 12.6% | 17.6% |
| Model Results (i.e., Benefits) Estimated for: | 2002-2003 | 2003-2004 |
| Crashes avoided | 1,426 | 2,276 |
| Fatal crashes avoided | 53 | 77 |
| Injury crashes avoided | 677 | 1,038 |
| Towaway crashes avoided | 696 | 1,161 |
| Lives saved | 62 | 90 |
| Injuries avoided | 1,087 | 1,651 |
2.4.1. Methodology
The methodology of the CR Effectiveness Model was originally designed to estimate only the benefits that occur in the 12 months following the CRs in a given year, i.e., the initial benefits. Based on the results of research being conducted at the Volpe Center, it appears that the reduction in the average crash rate of carriers that receive CRs lasts for several years after the CRs.
For this report, three groups of carriers were studied: (1) carriers that received CRs in 2000, (2) carriers that received CRs in 2001 and (3) carriers that received CRs in 2002. The average crash rate and the number of power units for each group was tracked over time using state-reported crash data and power unit data reported by carriers or obtained during CRs, which are the same data used in the CR Effectiveness Model.
Table 2-2 shows the percent change from the pre-CR average crash rate by post-CR period for each group of carriers. The post-CR periods consist of:
| Year of Compliance Reviews | Percent Change in Crash Rate due to CRs | |||
|---|---|---|---|---|
| Benefit Period | ||||
| IB | EB1 | EB2 | EB3 | |
| 2000 | -8.46 | -9.09 | -11.92 | -16.75 |
| 2001 | -7.39 | -9.84 | -14.98 | ----- |
| 2002 | -12.59 | -10.71 | ----- | ----- |
For example, for carriers with CRs in 2000, the pre-CR period consisted of the years 1999 and 2000, while the initial benefit period consists of the years 2000 to 2001. Extended benefit periods 1, 2, and 3 are 2001-2002, 2002-2003, and 2003-2004, respectively.
For carriers with CRs in 2000 and 2001, the amount of crash rate reduction increased for each benefit period. For carriers with CRs in 2002, however, the crash rate reduction in extended benefit period 1 was less than the crash rate reduction in the initial benefit period. A methodology was developed to closely replicate these results and to project results for future years. This methodology involves projecting (1) the change in the average crash rate in each year and (2) the number of post-CR power units in each year. The projected number of crashes avoided is then calculated for each year using the same formula used in the CR Effectiveness Model.
The initial benefits of the CRs conducted in year X occur in the two-year period consisting of years X to X+1. The CR Effectiveness Model can be used to calculate the initial benefits when complete crash and power unit data for year X+1 become available. At that time, the model can also be used to calculate projections of the extended benefits occurring in the following two-year periods:
1) X+1 to X+2,
2) X+2 to X+3, and
3) X+3 to X+4.
In subsequent years, as complete data become available for years X+2, X+3, and X+4, the model can be used to calculate actual estimates (instead of projections) of the extended benefits occurring in the three two-year periods listed above.
In this report, estimates of the initial benefits of the CRs conducted in 2003, which occurred in 2003-2004, were calculated in Section 2.3. Projections of the extended benefits of the CRs conducted in 2003, which occurred (or will occur) in 2004-2005, 2005-2006, and 2006-2007, were calculated and are shown in Section 2.4.2. The methodology that was used to calculate these projections is described in Appendix C. Section 2.4.3 describes the calculation of the actual estimate of the extended benefits of the CRs conducted in 2002 that occurred in 2003-2004.
Section 2.4.4 contains a table of the results obtained in the two years that the CR Effectiveness Model has been in use.
Extended Benefits of the Compliance Reviews Conducted in 2003
Table 2-3 shows the number of crashes avoided in 2003-2004 by carriers that received CRs in 2003 that was estimated by the CR Effectiveness Model. Table 2-3 also shows the projected numbers of crashes avoided in the periods 2004-2005, 2005-2006, and 2006-2007.
| Benefit Time Period | ||||
|---|---|---|---|---|
| 2003-2004 (Estimated) | 2004-2005 (Projected) | 2005-2006 (Projected) | 2006-2007 (Projected) | |
| Estimated/Projected Crashes Avoided |
2,276 | 1,876 | 2,191 | 2,531 |
The projections in Table 2-3 should be regarded as preliminary indicators of the extended benefits of compliance reviews rather than as official estimates. The research on this topic is still in progress. As more years of data become available for analysis and verification, this methodology will be refined to produce more accurate projections.
2.4.3. Extended Benefits of the Compliance Reviews Conducted in 2002
Based on available complete crash and power unit data, the CR Effectiveness Model has produced the following estimates and projections of the benefits of the CRs conducted in 2002: (1) estimates for 2002-2003 and 2003-2004 and (2) projections for 2004-2005 and 2005-2006. This section describes the calculation of the estimate of the extended benefits for the period 2003-2004.4, See Endnotes
To calculate the initial benefits (for 2002-2003), the model compared a motor carrier's crash rate for the 12 months prior to a CR to its crash rate for the 12 months after that CR. To calculate the extended benefits for 2003-2004, the carrier's crash rate for the 12 months prior to the CR was compared to the crash rate for the second 12 months (i.e., months 13 to 24) after the CR.
To be eligible for the initial benefit calculation, a carrier in the CR Group had to be active with nonzero power units for 12 months before and 12 months after its 2002 CR. To be eligible for the extended benefit calculation, a carrier in the CR Group had to be active with nonzero power units for 12 months before and 24 months after its 2002 CR.
The Control Group adjustment for the initial benefit calculation compared the crash rate in the Control Group in 2001-2002 to its crash rate in 2002-2003. The Control Group adjustment for the extended benefit calculation compared the crash rate in the Control Group in 2001-2002 to its crash rate in 2003-2004.
To be eligible for the initial benefit calculation, a carrier in the Control Group had to be active with nonzero power units throughout the pre-CR period (i.e., January 2001 to December 2002) and the post-CR period (i.e., January 2002 to December 2003). To be eligible for the extended benefit calculation, a carrier in the Control Group had to be active with nonzero power units from the start of the pre-CR period (i.e., January 2001) to the end of the extended benefit period 1 (i.e., December 2004), i.e., a time span of four years.
When the model was implemented to calculate the initial benefits, 9,172 carriers in the CR Group and 396,478 carriers in the Control Group met the qualifying conditions. When the model was implemented to calculate the extended benefits, 8,616 carriers in the CR Group and 384,953 carriers in the Control Group met the qualifying conditions. These decreases are the result of carriers becoming inactive (e.g., going out of business) since the model was implemented to calculate the initial benefits.
Table 2-4 shows the pre-CR and post-CR average crash rates for the CR and Control Groups. The post-CR crash rates are for extended benefit period 1 (EB1).
| Group | Carriers | Pre-CR Crash Rate* (2001-2002) | Post-CR Crash Rate* (2003-2004) | Percent Change in Crash Rate |
|---|---|---|---|---|
| 2,276 | 8,616 | 4.020 | 4.157 | +3.41 |
| 2,276 | 384,953 | 1.721 | 1.964 | +14.12 |
| * - Crashes per 100 power units | ||||
Therefore, the adjusted change in the average crash rate due to the CRs conducted in 2002 was:
The 8,616 carriers in the CR Group had 272,143 power units two year after their CRs.
The estimated number of crashes avoided in 2003-2004 by the 8,616 carriers that received CRs in 2002 and were still active 24 months after their CRs was calculated as follows:
This estimate is 13.6 percent below the projection of 1,356 crashes avoided that was made in May 2005.
2.4.4. Summary of Results
Table 2-5 shows the results of the implementation of the CR Effectiveness Model for CRs conducted in 2002 and 2003. Table 2-5 shows the latest estimates and projections of crashes avoided due to the CRs conducted in the two years, 2002 and 2003, for which the model has been run. Figure 2-3 presents a bar graph of the results.
| Compliance Reviews Conducted in | Crashes Avoided | ||||
|---|---|---|---|---|---|
| Benefit Time Period | |||||
| 2002-2003 (Estimated) | 2003-2004 (Estimated) | 2004-2005 (Projected) | 2005-2006 (Projected) | 2006-2007 (Projected) | |
| 2002 | 1,426 | 1,171 | 1,475 | 1,526 | |
| 2003 | 2,276 | 1,876 | 2,191 | 2,531 | |
(Note: Data from Table 2-5 is displayed in the figure below")
The results of the implementation of the model were broken out by carrier size (i.e., number of power units), by the state of domicile of the carrier, and by carrier safety status (i.e., the carrier's SafeStat1, See Endnotes category before receiving its CR in 2003).
The results of these analyses revealed the types of carriers that will most likely respond positively to CRs. By focusing on carriers that are likely to respond positively to CRs, the effectiveness of the compliance review program may be improved. Alternative treatment approaches may be suggested for carriers that are at risk, but will most likely not respond positively to CRs.
The changes in the average crash rates of (1) the carriers that received CRs in 2003 and (2) the Control Group (i.e., the carriers that did not receive CRs in 2003) were calculated individually for each power unit group, state of domicile, and SafeStat category group.
The sums of the estimates of crashes avoided by power unit group, state of domicile, and SafeStat category group did not equal the estimate of 2,276 crashes avoided that was obtained in Section 2.3. Therefore, the estimates were prorated to sum to this number. The estimated numbers of crashes avoided and the percent changes in the average crash rates shown in Tables 3-1, 3-2, and 3-3 reflect this prorating procedure.
The results of the implementation of the model were broken out by carrier size as measured by the number of power units at the time of the CR, i.e., the number of pre-CR power units.
Table 3-1 shows the results of the implementation of the model for the four power unit groups:
Table 3-1 shows, for each power unit group, the number of carriers in the group that received CRs in 2003, the pre-CR average crash rate, and the adjusted percent change in the average crash rate after receiving the CRs. Table 3-1 also shows, for each power unit group, the estimated number of crashes avoided as a result of the CRs.
| Number of Pre-CR Power Units | Number of Carriers with CRs in 2003 | Pre-CR Average Crash Rate* | Percent Change in Average Crash Rate | Estimated Number of Crashes Avoided in 2003-2004 |
|---|---|---|---|---|
| 1 - 5 | 3,699 | 10.813 | -58.7 | 787 |
| 6 - 20 | 3,100 | 7.116 | -33.0 | 815 |
| 21-100 | 1,479 | 5.466 | -17.4 | 611 |
| ≥101 | 309 | 4.332 | -1.1 | 63 |
| All Carriers | 8,587 | 5.290 | -17.6 | 2,276 |
| * - Crashes per 100 power units | ||||
The smaller carriers, those with 20 or fewer power units, had the greatest reduction in the average crash rate as well as the largest number of estimated crashes avoided as a result of the program. For carriers with 1-5 power units, the post-CR average crash rate showed a decrease of 58.7 percent from the pre-CR average crash rate, resulting in 787 crashes avoided. For carriers with 6-20 power units, the crash rate decrease was 33.0 percent, resulting in 815 crashes avoided.
For carriers with 21-100 power units, the post-CR average crash rate showed a decrease of 17.4 percent, resulting in 611 crashes avoided. Carriers with 101 or more power units had a decrease of 1.1 percent in their average crash rate and a decrease of 63 crashes.
The results of this analysis are consistent with (1) the results of the analysis of data from the implementation of the model for carriers with CRs in 2002,2, See Endnotes and (2) the results of analyses of data from the previous model, the Compliance Review Impact Assessment Model.3, See Endnotes
Table 3-2 shows the results of the implementation of the model broken out by the carrier's state of domicile. For a state's results to be published in the table, it had to have at least 50 carriers with CRs in 2003. Eight states4, See Endnotes and the District of Columbia did not meet this requirement. Their data were combined and are shown in the row labeled "Other States". Since there were not enough Canadian or Mexican carriers receiving CRs in 2003 to summarize at the province/state level, these results were summarized at the national level.
Table 3-2 shows, for each state (or country), the number of carriers that received CRs in 2003, the pre-CR average crash rate, and the adjusted percent change in the average crash rate after receiving the CRs. Table 3-2 also shows, for each state (or country), the estimated number of crashes avoided as a result of the CRs. (Note: A number in parentheses indicates an increase in the number of crashes.)
Table 3-2 shows that one state, Arizona (247), had more than 200 crashes avoided in 2003-2004 due to CRs performed in 2003. Six other states (Illinois, Alabama, Texas, Georgia, Tennessee, and North Carolina) each had more than 100 crashes avoided. Four states showed increases in the number of crashes in 2003-2004 for carriers that received CRs in 2003.
There are several factors that affect the state estimates of crashes avoided. The equation that is used to calculate the number of crashes avoided consists of three factors: the pre-CR average crash rate, the percentage reduction in the average crash rate due to the CRs, and the number of post-CR power units. The states with the largest numbers of crashes avoided are usually among the states with the highest numbers of post-CR power units, which is a function of the number of carriers receiving CRs. The more carriers in a state that receive reviews, the greater the number of post-CR power units that results, which increases the potential for a large number of crashes to be avoided. For example, Texas had a reduction in its average crash rate of only 9.3 percent, but had 143 crashes avoided because it had 719 carriers with CRs in 2003. On the other hand, Connecticut had a reduction in its average crash rate of 66.3 percent, but had only 23 crashes avoided because it had only 65 carriers with CRs in 2003.
Another factor that influenced the state results was the proportion of the carriers with zero crashes in the pre-CR period in each state that received CRs in 2003. Of the total of 8,587 carriers that received reviews in 2003, 4,980, or 58.0 percent, had pre-CR crash rates of zero. Thus, the crash rates of these carriers could either stay the same or increase, but not decrease. If a state had an especially high percentage of these carriers, it would make it difficult for that state's average crash rate to decrease significantly. For example, the state of Washington had an increase in its average crash rate of 9.8 percent. One reason for this increase is that 69.6 percent of the carriers in Washington that received CRs in 2003 had pre-CR crash rates of zero.
In addition, the relatively low number of carriers in each state that received CRs in 2003 makes the state results subject to the influence of a few large carriers, i.e., carriers with large numbers of power units. As shown in Table 3-1, there were 309 carriers with 101 or more power units that received CRs in 2003. While these carriers made up only 3.6 percent of the 8,587 carriers being analyzed, they accounted for 54.5 percent of the total post-CR power units. Thus, the data from a single large carrier could greatly affect an individual state's results.
| State/Country of Domicile | Number of Carriers with CRs in 2003 | Pre-CR Average Crash Rate* | Percent Change in Average Crash Rate | Estimated Number of Crashes Avoided in 2003-2004 |
|---|---|---|---|---|
| Alabama | 207 | 7.600 | -28.5 | 157 |
| Arizona | 174 | 6.718 | -18.4 | 247 |
| Arkansas | 140 | 6.693 | -8.6 | 37 |
| California | 231 | 2.952 | +1.4 | (3) |
| Colorado | 208 | 4.202 | -0.1 | 0 |
| Connecticut | 65 | 3.786 | -66.3 | 23 |
| Florida | 116 | 5.100 | -6.0 | 9 |
| Georgia | 367 | 6.349 | -30.9 | 132 |
| Idaho | 83 | 5.000 | -25.7 | 15 |
| Illinois | 284 | 6.476 | -29.0 | 169 |
| Indiana | 325 | 4.931 | -21.6 | 87 |
| Iowa | 93 | 8.738 | -55.8 | 84 |
| Kansas | 263 | 4.979 | -29.0 | 76 |
| Kentucky | 192 | 6.047 | -40.8 | 84 |
| Louisiana | 104 | 5.346 | -23.4 | 28 |
| Maryland | 98 | 4.674 | -31.7 | 47 |
| Massachusetts | 87 | 4.854 | -35.9 | 28 |
| Michigan | 194 | 5.489 | -10.0 | 37 |
| Minnesota | 315 | 3.638 | -19.5 | 76 |
| Mississippi | 195 | 7.742 | -16.9 | 65 |
| Missouri | 438 | 4.474 | -20.3 | 73 |
| North Carolina | 227 | 9.177 | -39.1 | 101 |
| North Dakota | 57 | 5.500 | -41.0 | 25 |
| Nebraska | 95 | 6.657 | -22.1 | 31 |
| Nevada | 74 | 3.289 | -19.5 | 7 |
| New Jersey | 176 | 4.730 | +13.3 | (34) |
| New Mexico | 68 | 4.363 | -32.5 | 23 |
| New York | 148 | 5.195 | -28.9 | 57 |
| Ohio | 493 | 4.800 | -6.5 | 43 |
| Oklahoma | 159 | 5.375 | -1.2 | 3 |
| Oregon | 58 | 6.181 | -39.8 | 18 |
| Pennsylvania | 290 | 3.669 | -30.7 | 60 |
| South Carolina | 122 | 7.845 | -41.3 | 62 |
| South Dakota | 51 | 4.175 | -2.7 | 2 |
| Tennessee | 203 | 7.362 | -30.8 | 110 |
| Texas | 719 | 4.754 | -9.3 | 143 |
| Utah | 202 | 4.758 | +1.4 | (2) |
| Virginia | 73 | 5.098 | -31.1 | 28 |
| Washington | 257 | 3.738 | +9.8 | (22) |
| West Virginia | 79 | 7.256 | -77.1 | 51 |
| Wisconsin | 296 | 5.297 | -2.9 | 32 |
| Wyoming | 95 | 2.506 | -37.8 | 8 |
| Other States† | 228 | 4.738 | -22.0 | 40 |
| Canada | 120 | 4.338 | -13.3 | 16 |
| Mexico | 118 | 0.764 | -35. | 3 |
| Total | 8,587 | 5.290 | -17.6 | 2,276 |
|
* - Crashes per 100 power units † - Alaska, District of Columbia, Delaware, Hawaii, Maine, Montana, New Hampshire, Rhode Island, and Vermont |
||||
One of the primary methods of prioritizing carriers for CRs is to use SafeStat results. Carriers are assessed in four Safety Evaluation Areas (SEAs): Accident, Driver, Vehicle, and Safety Management. Carriers are placed in SafeStat categories if they are found to be deficient in one or more SEAs. Carriers with the most extensive deficiencies are placed in Categories A and B and are assigned the highest priority for CRs, followed by carriers in Category C, carriers in Categories D-G, and finally, carriers not in any category (i.e., carriers not deficient in any SEAs).
The purpose of the analysis in this section is to determine the impact of carrier safety status prior to CRs on crash rate reduction after the CRs. In other words, determine if carriers with the highest priority for CRs show the greatest improvement, i.e., the largest crash rate reduction, following CRs.
The results of the CR Effectiveness model were broken out by SafeStat category group based on each carrier's SafeStat category prior to receiving its 2003 CR. Table 3-3 shows, for each SafeStat category group, the number of carriers in the group that received CRs in 2003, the pre-CR average crash rate, and the adjusted percent change in the average crash rate after receiving the CRs. Table 3-3 also shows, for each SafeStat category group, the estimated number of crashes avoided as a result of the CRs.
| SafeStat Category Group | Number of Carriers with CRs in 2003 | Pre-CR Average Crash Rate* | Percent Change in Average Crash Rate | Estimated Number of Crashes Avoided in 2003-2004 |
|---|---|---|---|---|
| A-B | 3,029 | 7.295 | -25.0 | 1,150 |
| C | 889 | 4.459 | -20.8 | 143 |
| D-G | 2,326 | 5.792 | -15.2 | 693 |
| None | 2,343 | 3.549 | -9.4 | 290 |
| All Carriers | 8,587 | 5.290 | -17.6 | 2,276 |
| * - Crashes per 100 power units | ||||
Carriers in Categories A and B, the carriers with the highest priority for CRs, had the highest pre-CR average crash rate as well as the greatest percent reduction in their average crash rate. The post-CR average crash rate showed a decrease of 25.0 percent. The carriers in this group accounted for 1,150 crashes avoided in 2003-2004, which is over half (50.5%) of the total of 2,276 crashes avoided.
Carriers in Category C showed a decrease of 20.8 percent in their average crash rate. The pre-CR average crash rate for this group was much lower than for the carriers in Categories A and B, probably because none of the carriers in Category C were deficient in the Accident SEA.
Carriers in Categories D-G showed a decrease of 15.2 percent in their average crash rate. Carriers not in any SafeStat category showed a decrease of 9.4 percent in their average crash rate. In summary, the percent reduction in the average crash rate was directly related to carrier safety status. That is, the higher the priority group, the greater the percent reduction in the average crash rate.
The results indicate that the carriers that SafeStat is identifying and prioritizing for compliance reviews are the carriers that show the greatest reductions in crash rates following CRs. Therefore, the SafeStat prioritization is increasing the overall effectiveness, in terms of crash rate reduction, of the compliance review program.
The pre-CR and post-CR average crash rates for the Control (i.e., non-CR) Group are actually the weighted averages of the average crash rates of the individual years, as shown by the following derivation.
The weighted average of the crash rates of two individual years is calculated by the equation:
Weighted Average Crash Rate
where Rn = the average crash rate for year n, and Wn = the weight for year n.
Rn, the average crash rate for year n, is defined as:
Cn
= ---
Pn
where Cn = the number of crashes in year n, and
Pn = the number of power units at the end of year n
In this case, Wn, the weight for year n, is defined as Pn, the number of power units at the end of year n
Therefore, the weighted average of the crash rates for years 1 and 2
Therefore, the weighted average of the average crash rates for the Control Group for 2002 and 2003
Crashes in 2002 + Crashes in 2003
= ----------------------------------------------------------------------
Power Units at the end of 2002 + Power Units at the end of 2003
= Pre-CR Average Crash Rate for the Control Group
Also, the weighted average of the average crash rates for the Control Group for 2003 and 2004
Crashes in 2003 + Crashes in 2004
= ----------------------------------------------------------------------
Power Units at the end of 2003 + Power Units at the end of 2004
= Post-CR Average Crash Rate for the Control Group
The 414,998 carriers in the Control Group had a pre-CR average crash rate of 1.880 crashes per 100 power units and a post-CR average crash rate of 2.054 crashes per 100 power units.
The percentage change in the average crash rate of the Control Group was calculated as follows:
Percentage Change in Average Crash Rate of Control Group
Post-CR Average Crash Rate - Pre-CR Average Crash Rate
= ------------------------------------------------------------------- X 100
Pre-CR Average Crash Rate
2.054 - 1.880
= ------------------- X 100
1.880
= +9.26% (i.e., an increase of 9.26 percent)
This increase in the average crash rate of the Control Group, and therefore, the general carrier population, is the sum of the effects of (1) any change in the average crash rate of the general carrier population and (2) other underlying factors in the general carrier population (e.g., changes in crash reporting). To determine how much of the increase was due to each element, the change in the average crash rate of the general carrier population was calculated.
To verify if the crash rate actually increased during the period in which the benefits from the CRs conducted in 2003 would have occurred (i.e., 2003-2004), data independent of the state-reported crash data used in the CR Effectiveness Model were used to calculate the large truck crash rates for the periods 2002-2003 and 2003-2004. The percentage change in the two crash rates was then calculated.
These crash rates were calculated using large truck crash data from the Fatality Analysis Reporting System (FARS) and the General Estimates System (GES), which are maintained by the National Highway Traffic Safety Administration (NHTSA). Counts of fatal crashes were obtained from the FARS, which contains data on a census of fatal crashes. Counts of injury crashes and property-damage-only crashes were obtained from the GES, which produces crash estimates from a national probability sample of all police-reported crashes. Crashes are included in the sample whether or not they are reported by the states to the FMCSA.
The NHTSA crash classification system differs from the National Governors' Association (NGA) standard used by the states to report crashes to the FMCSA. In both systems, a fatal crash is defined as a crash resulting in at least one fatality, although the NHTSA rule specifically requires that at least one death occur within 30 days of the crash. For non-fatal crashes, the differences are much greater.
The NGA categories of non-fatal crashes are injury and towaway:
The NHTSA categories of non-fatal crashes are injury and property-damage-only:
The NHTSA non-fatal crash categories include many more crashes of lower severity than do the NGA non-fatal crash categories. Since it is the change in crash rates that is being measured, rather than the crash rates themselves, using the FARS and GES data should provide a reasonable indication of the change in the NGA crash rate calculated using the FMCSA's MCMIS data.
Power unit data were obtained from the Federal Highway Administration (FHWA). The FHWA collects truck registration data from the 50 states and the District of Columbia. The data obtained were the numbers of large trucks registered in the U.S. in 2002, 2003, and 2004.
The change in the average crash rate of the general carrier population, as measured by the FARS and GES data, is calculated as follows:
Percent Change Post-CR Average Crash Rate - Pre-CR Average Crash Rate
in Average = ----------------------------------------------------------------------- X 100
Crash Rate Pre-CR Average Crash Rate
The pre-CR crash rate is the average crash rate for the entire pre-CR period, i.e., 2002-2003, while the post-CR crash rate is the average crash rate for the entire post-CR period, i.e., 2003-2004. The pre-CR and post-CR average crash rates are calculated as follows:
Pre-CR Crashes in 2002 + Crashes in 2003
Average = ----------------------------------------------------------------------- X 100
Crash Rate Large Trucks Reg. in 2002 + Large Trucks Reg. in 2003
Post-CR Crashes in 2003 + Crashes in 2004
Average = ----------------------------------------------------------------------- X 100
Crash Rate Large Trucks Reg. in 2003 + Large Trucks Reg. in 2004
The general carrier population had a pre-CR average crash rate of 5.439 crashes per 100 power units and a post-CR average crash rate of 5.249 crashes per 100 power units.
The percentage change in the average crash rate of the general carrier population was calculated as follows:
Percentage Change in Average Crash Rate of General Carrier Population
Post-CR Average Crash Rate - Pre-CR Average Crash Rate
= -------------------------------------------------------------- X 100
Pre-CR Average Crash Rate
5.249 - 5.439
= ---------------------- X 100
5.439
= -3.49% (i.e., a decrease of 3.49 percent)
Thus, the combined data from NHTSA and FHWA suggest that the actual change in the crash rate for large trucks was a decrease of 3.49 percent from 2002-2003 to 2003-2004. This result suggests that the apparent increase in the crash rate obtained for the Control Group in this analysis resulted from increases in the completeness of crash reporting in various states, rather than an actual change in motor carrier behavior.
Therefore, the increase in the crash rate of the Control Group caused by changes in other underlying factors (e.g., changes in crash reporting) in the general carrier population was:
Percentage Change in Average Crash Rate of Control Group due to Other Underlying
Factors in General Carrier Population
= Percentage Change in Average Crash Rate of Control Group (from state-reported data)
− Percentage Change in Average Crash Rate of General Carrier Population (from FARS and GES data)
= 9.26% - (-3.49%)
= 12.75%
Therefore, the 9.26 percent increase in the average crash rate of the control group, and therefore, the general carrier population, was the sum of a 3.49 percent decrease in the crash rate of the general carrier population and a 12.75 percent increase due to other underlying factors in the general carrier population (e.g., changes in crash reporting).
To calculate the projections of the extended benefits of the compliance reviews (CRs) conducted in a given year, three parameters must be estimated:
Data for carriers with CRs in 2000, 2001, 2002, and 2003 were used to estimate these parameters. For carriers with CRs in 2002 and 2003, the original calculations of the initial benefits were used, rather than the newly generated tables.
1. Change in the Pre-CR Average Crash Rate
Every year, some of the carriers that received CRs in the year being analyzed, i.e. the CR Group, become inactive. Therefore, a smaller set of carriers is used to project the benefits that occur in the first extended benefit period than was used to estimate the initial benefits. Still smaller sets of carriers are used to project the benefits in the second and third extended benefit periods. The changing composition of the active carriers in the CR Group causes changes in the pre-CR average crash rate. This is the rate from which the change in crash rate is calculated.
Table C-1 shows the percent changes in the pre-CR average crash rates from the previous year for carriers with CRs in 2000, 2001, and 2002. Note that EB1 is extended benefit period 1, EB2 is extended benefit period 2, etc.
| Year of CRs | Percent Change in Crash Rate from Previous Year | ||
|---|---|---|---|
| Extended Benefit Period | |||
| EB1 | EB2 | EB3 | |
| 2000 | -1.39 | -1.65 | -0.52 |
| 2001 | -02.30 | -0.80 | ----- |
| 2002 | -0.30 | ----- | ----- |
| Average | -1.33 | -1.23 | -0.52 |
| 2003 Estimate | -1.35 | -1.25 | -0.50 |
In the absence of a clear pattern in the limited amount of available data, the estimates were obtained by rounding the benefit period averages to the nearest .05.
2. Change in the Number of Post-CR Power Units
In the years following their CRs, the number of power units operated by active carriers in the CR Group changes each year because (1) some carriers become inactive, and (2) there are changes in the numbers of power units operated by the carriers that are still active.
Table C-2 shows the percent change in the number of post-CR power units from the previous year for carriers with CRs in 2000, 2001, and 2002.
| Year of CRs | Percent Change in Post-CR Power Units from Previous Year | ||
|---|---|---|---|
| Extended Benefit Period | |||
| EB1 | EB2 | EB3 | |
| 2000 | -2.46 | -2.22 | -0.52 |
| 2001 | -0.42 | -0.68 | ----- |
| 2002 | -3.04 | ----- | ----- |
| Average | -1.97 | -1.45 | -0.52 |
| 2003 Estimate | -1.95 | -1.45 | -0.50 |
Again, in the absence of a clear pattern in the limited amount of available data, the estimates were obtained by rounding the benefit period averages to the nearest .05.
3. Adjusted Percent Change in Pre-CR Average Crash Rate due to Compliance Reviews
The third and most important parameter to estimate is the change in the pre-CR average crash rate due to the CRs performed on the carriers. This change is obtained by subtracting the crash rate change in the Control Group from the crash rate change in the CR Group. This adjusted change was not calculated when the projections of the extended benefits of the CRs conducted in 2002 were made.
Table C-3 shows the adjusted percent changes in the pre-CR average crash rates of carriers in the initial and extended benefit periods. The table includes carriers with CRs in 2000, 2001, 2002, and 2003. Note that IB is the initial benefit period.
| Year of CRs | Percent Change in Post-CR Power Units from Previous Year | |||
|---|---|---|---|---|
| Extended Benefit Period | ||||
| IB | EB1 | EB2 | EB3 | |
| 2000 | -8.46 | -9.09 | -11.92 | -16.75 |
| 2001 | -7.39 | -9.84 | -14.98 | ----- |
| 2002 | -12.59 | -10.71 | ----- | ----- |
| 2003 | -17.61 | ----- | ----- | ----- |
| Average | -11.51 | -9.88 | -13.45 | -16.75 |
| 2003 Estimate | -15.0 | -18.0 | -21.0 | |
For carriers with CRs in 2000 and 2001, the amount of crash rate reduction increased for each benefit period. For carriers with CRs in 2002, however, the crash rate reduction in the first extended benefit period was less than the crash rate reduction in the initial benefit (i.e., post-CR) period. Therefore, the estimates of future crash rate reduction for the carriers with CRs in 2003 must represent compromises of the indications.
First, for carriers with CRs in 2002, dividing the crash rate reduction in the first extended benefit period (10.71%) by the crash rate reduction in the initial benefit period (12.59%) produced a ratio of 85.1 percent. Applying this factor to the crash rate reduction in the initial benefit period for carriers with CRs in 2003 produced a crash rate reduction estimate of 15.0 percent:
Crash Rate Reduction in Extended Benefit Period 1 = Crash Rate Reduction in Initial Benefit Period X 85.1% = 17.61 X .851 = 14.99 percent ˜ 15.0 percentThis estimate is shown in the bottom row of Table C-3.
A lower crash rate reduction estimate for the first extended benefit period (less than 15 percent) would have been more in line with the results for the three previous years of CRs. Given the high level of crash reduction in the initial benefit period (17.61 percent), however, a lower crash rate reduction estimate (for the first extended benefit period) would have resulted in a larger decrease in the number of crashes avoided from the initial benefit period to the first extended benefit period. Such a large swing in the estimates would have run counter to the desire to "smooth out" the results.
To make the crash rate reduction estimates for the second and third extended benefit periods, the historical averages were examined. As shown in Table C-3, the average changes in the pre-CR average crash rate were 9.88, 13.45, and 16.75 percent for the first, second, and third extended benefit periods, respectively. Therefore, the average crash rate reduction was 3.57 (13.45 - 9.88) percent greater in the second extended benefit period than in the first extended benefit period, and 3.30 (16.75 - 13.45) percent greater in the third extended benefit period than in the second extended benefit period.
Applying these increases to the estimate of 15.0 percent for the first extended benefit period would have produced estimates of 18.6 (i.e., rounded up from 18.57) percent for the second extended benefit period and 21.9 (i.e., rounded up from 21.87) percent for second and third extended benefit period. In an effort to be conservative, the estimates were lowered to 18.0 and 21.0 percent. Since the crash rate reduction in the initial benefit period, 17.61 percent, was the highest recorded in any period so far, adding large increases to such a high starting point would have run the risk of overestimating the numbers of crashes avoided in the extended benefit periods.
4. Projections of Crashes Avoided
Using the parameters estimated in Sections 1, 2, and 3, projections of crashes avoided in the three extended benefit periods were made. In addition, the model was implemented to produce an estimate of the number of crashes avoided in the first extended benefit period by the carriers with CRs in 2002. Table C-4 shows this estimate and the projections.
| Compliance Reviews Conducted in | Crashes Avoided | ||||
|---|---|---|---|---|---|
| 2002-2003 (Estimated) | 2003-2004 (Estimated) | 2004-2005 (Projected) | 2005-2006 (Projected) | 2006-2007 (Projected) | |
| 2002 | 1,426 | 1,171 | 1,475 | 1,526 | |
| 2003 | 2,276 | 1,876 | 2,191 | 2,531 | |
5. Summary
Two conclusions regarding the projections of the extended benefits from the CRs conducted in 2003 are apparent from the data:
If one accepts these conclusions, then the only remaining questions are what should be the magnitudes of the decrease and the subsequent two increases in crash reduction. The desire to be conservative in the projections of crashes avoided must be balanced against the desire to avoid wild swings in the projections.
Footnotes:
Executive Summary:
1 A report documenting these results can be found at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=22#fmcsaA.
2This report is available at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=22#fmcsaA
3 SafeStat (Safety Status Measurement System) is an automated, data-driven analysis system that is designed to incorporate on-road safety performance information and enforcement history with on-site compliance review information in order to measure the relative safety fitness of interstate motor carriers. A thorough description of SafeStat methodology can be found in: John A. Volpe National Transportation Systems Center, Motor Carrier Safety Assessment Division, DTS-47, SafeStat, Motor Carrier Safety Status Measurement System, Methodology: Version 8.6, January 2004. This document is available at ai.fmcsa.dot.gov/SafeStat/safestat.asp?file=method.pdf.
Section 1:
1 This report is available at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=22#fmcsaA.
2 A report documenting these results can be found at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=22#fmcsaA.
Section 2:
1SafeStat (Safety Status Measurement System) is an automated, data-driven analysis system that is designed to incorporate on-road safety performance information and enforcement history with on-site compliance review information in order to measure the relative safety fitness of interstate motor carriers. A thorough description of SafeStat methodology can be found in: John A. Volpe National Transportation Systems Center, Motor Carrier Safety Assessment Division, DTS-47, SafeStat, Motor Carrier Safety Status Measurement System, Methodology: Version 8.6, January 2004. This document is available at ai.fmcsa.dot.gov/SafeStat/safestat.asp?file=method.pdf.
2 The pre-CR average crash rate is actually the weighted average of the average crash rates for 2002 and 2003. The post-CR average crash rate is actually the weighted average of the average crash rates for 2003 and 2004. A detailed derivation of these formulas can be found in Appendix A.
3 A fatal crash results in at least one fatality. An injury crash results in no fatalities, but bodily injury to at least one person who, as a result of the injury, immediately receives medical treatment away from the scene of the crash. A towaway crash results in no fatalities or injuries requiring transport for immediate medical attention, but in one or more motor vehicles incurring disabling damage as a result of the crash, requiring the vehicle(s) to be transported away from the scene by a tow truck or other motor vehicle.
4A report containing a description of the calculation of the benefits for the other periods can be found at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=22#fmcsaA.
Section 3:
1 SafeStat (Safety Status Measurement System) is an automated, data-driven analysis system that is designed to incorporate on-road safety performance information and enforcement history with on-site compliance review information in order to measure the relative safety fitness of interstate motor carriers. A thorough description of SafeStat methodology can be found in: John A. Volpe National Transportation Systems Center, Motor Carrier Safety Assessment Division, DTS-47, SafeStat, Motor Carrier Safety Status Measurement System, Methodology: Version 8.6, January 2004. This document is available at ai.fmcsa.dot.gov/SafeStat/safestat.asp?file=method.pdf.
2A report documenting these results can be found at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=22#fmcsaA.
3A report documenting these results can be found at ai.fmcsa.dot.gov/CarrierResearchResults/Archives.asp?p=22#fmcsaA.
4Alaska, Delaware, Hawaii, Maine, Montana, New Hampshire, Rhode Island, and Vermont