CSA Scores Unreliable, U.S. Gov. Agency Review Agrees
ARLINGTON, VA — The Federal Motor Carrier Safety Administration’s (FMCSA) CSA safety enforcement program cannot accurately assess the safety of small carriers, the Government Accountability Office (GAO) found in its recent review.
"The GAO's review of FMCSA's Compliance, Safety, Accountability (CSA) program was comprehensive, thoughtful and balanced," said Bill Graves, the president and CEO of the America Trucking Associations (ATA). "While ATA has long supported CSA's objectives, we can't help but agree with GAO's findings that the scores produced by the program don't present an accurate or precise assessment of the safety of many carriers."
The ATA stopped short of saying, “we told you so,” but they did find similar shortcomings with the CSA program in a white paper published in December 2013. At the time, the ATA said the safety program was “plagued with problems”.
GAO reviewed the CSA at the request of Senators concerned about the effectiveness of the system. The watchdog agency said the CSA has benefits, but because of data shortcomings it is not as strong a predictor of crash risk as it could be.
The report found that FMCSA lacks sufficient safety performance data on most carriers to reliably compare them with other carriers. The lack of data is especially problematic for small carriers, who make up 95 percent of the industry.
For CSA to be effective in identifying carriers more likely to have a crash, the violation data used by the system should have a relationship to crash risk, GAO said. However, for most violations, FMCSA has not demonstrated the relationship between violation groups and motor carrier crash risk due, in part, to the lack of data. According to GAO's analysis, of those with sufficient data, only 13 violations consistently had some association with crash risk and only two "had sufficient data to consistently establish a substantial and statistically reliable relationship with crash risk across all (of their) tests."
The agency should revise the Safety Measurement System that is a core CSA component, and it should take the limitations of the system into account when it installs the pending safety fitness regime, GAO said.
ATA’s Executive VP Dave Osiecki, commented: “Since scores are so often unreliable, third parties are prone to making erroneous judgments based on inaccurate data, an inequity that can only be solved in the near term by removing the scores from public view.”
But GAO did not look at the effects of displaying or from public use of the CSA scores, according to the report.
The report also calls into question FMCSA's intention on using CSA data as a basis for its Safety Fitness Determination rulemaking, scheduled for later this year.
"Basing a carrier's safety fitness determination on limited performance data may misrepresent the safety status of carriers," the report stated.
"It would clearly be improper for FMCSA to proceed with its plan to base carrier safety fitness determinations on data from the system, until the problems identified by GAO have been rectified," Osiecki said.