Energy
Wednesday, July 20th, 2022 10:23 am EDT
Quite surprisingly, the National Highway Traffic Safety Administration (NHTSA) recently started collecting somewhat broad data on automobile accidents involving SAE Level 2 Advanced Driver Assistance Systems (ADAS) — driver-assist technology like cruise control, lane keeping, and automatic lane changing. The first report on the collected data was published last month. Immediately, a rather shocking figure was pulled out of it: “Tesla vehicles made up nearly 70 percent of the 392 crashes involving advanced driver-assistance systems reported since last July, and a majority of the fatalities and serious injuries — some of which date back further than a year.”
Immediately, the figure made no sense to me. Yet, instead of seeing any real critiques of the conclusion, I saw it referenced from time to time to disparage Tesla Autopilot and FSD Beta. I have Tesla Autopilot and FSD Beta and am well aware that they are not perfect but that they are about as good as it gets for current ADAS. The idea that so many more crashes are occurring due to Tesla ADAS despite there being so many more inferior non-Tesla ADAS systems being on the road defies basic logic. Yes, you could assume that Tesla drivers are expecting more of their ADAS and being more reckless, leading to more crashes, but 70% of all ADAS-related crashes in the USA? Really? Of course not.
No, there’s something else at play with these data, and NHTSA actually spells it out in its various disclaimers — it just seems that no one cared to pay attention to the disclaimers or give them much weight. Most importantly and most simply: you only have the data you have. The assumption that NHTSA has collected and logged all crashes involving ADAS is way off the mark. But let’s just jump over to the agency’s own language — aptly included at the beginning of its first report on this topic.
NHTSA Data from 1st Report on ADAS-Related Crashes
Here’s the first line of the report: “In June 2021, the National Highway Traffic Safety Administration (NHTSA) issued a Standing General Order (the General Order) requiring identified manufacturers and operators (reporting entities) to report to the agency certain crashes involving vehicles equipped with SAE Level 2 Advanced Driver Assistance Systems (ADAS).” In other words, the collected crash data are data self-reported by the automakers. One could presume nefarious motives by some automakers to hide such crash data and not report their numbers to the NHTSA, but I think the problem is much simpler and less nefarious than that. It seems that the NHTSA does as well.
First, the total number of ADAS-related crashes in nearly one year of data collection from automakers: “As of May 15, 2022, NHTSA has received incident reports for 392 Level 2 ADAS-equipped vehicle crashes.” That’s right — just 392 crashes, 70% of which were reported by Tesla. That’s just 119 crashes reported by other automakers. If you think there were only 119 crashes involving non-Tesla ADAS in almost one full year, I’ve got a bridge I’d like to sell you.
The NHTSA then points out that it never really got data on such crashes until the General Order noted above (which is actually sort of surprising in itself). Here’s exactly what the NHTSA wrote, but the most interesting and relevant points aren’t until the next section (“Data and Limitations”): “Prior to the implementation of the General Order, NHTSA’s sources of timely crash notifications were limited (e.g., media, vehicle owner questionnaire input, voluntary direct company outreach, and Early Warning Reporting) and generally inconsistent across manufacturers, including developers.
“The General Order allows NHTSA to obtain timely and transparent notification of real-world crashes associated with Level 2 ADAS-equipped vehicles from manufacturers and operators. With these data, NHTSA can respond to crashes that raise safety concerns about ADAS technologies through further investigation and enforcement. As an example, the General Order has provided the agency information on specific crashes of ADAS-equipped vehicles. NHTSA has followed up with several manufacturers to acquire more data on these crashes, and this information has informed an ongoing investigation the agency opened on a driver assistance system in late 2021. The information also provides transparency into the safety of vehicles equipped with this technology that are currently operating on public roads. It also provides crucial data necessary for research and for the development of policies to enhance the safety of these technologies. Safety is crucial to the deployment and public acceptance of new technologies and the General Order is an important measure in evaluating the safety of ADAS and ADS technology.”
That all sounds fine and dandy, as long as you are going to truly get comprehensive data from automakers. The problem is laid out somewhat subtly but clearly enough in the next section, titled “Data and Limitations” in the NHTSA report.
NHTSA Data Limitations on ADAS-Related Crashes
The first line of this section tells the reader to really, truly, seriously consider these limitations before making any conclusions: “Understanding the terms and requirements of the General Order, the data it aims to obtain, and the limitations of those data are crucial for accurate interpretation and analysis. The following should be considered when reviewing these data and other information.” There are then 5 sub-sections to this section:
- Access to Crash Data May Affect Crash Reporting
- Incident Report Data May Be Incomplete or Unverified
- Redacted Confidential Business Information and Personally Identifiable Information
- The Same Crash May Have Multiple Reports
- Summary Incident Report Data Are Not Normalized
You can get the point just from those subheadings, but the short and sweet of it is that automakers often aren’t equipped to collected data on ADAS-related crashes, and thus aren’t able to supply NHTSA with the data it’s requesting. Without most of the industry data, naturally, one can’t make conclusions comparing the different ADAS systems or automakers.
Here are a few statements from NHTSA from the first subsection of this section (emphasis added):
- “Crash data recording and telemetry capabilities may vary widely by manufacturer and driving automation system.”
- “Many Level 2 ADAS-equipped vehicles may be limited in their capabilities to record data related to driving automation system engagement and crash circumstances.”
- “The vehicle’s ability to remotely transmit this data to the manufacturer for notification purposes can also widely vary.”
- “Furthermore, Level 2 ADAS-equipped vehicles are generally privately owned; as a result, when a reportable crash does occur, manufacturers may not know of it unless contacted by the vehicle owner.”
- “Manufacturers of Level 2 ADAS-equipped vehicles with limited data recording and telemetry capabilities may only receive consumer reports of driving automation system involvement in a crash outcome, and there may be a time delay before the manufacturer is notified, if the manufacturer is notified at all.”
- “Furthermore, some crashes of Level 2 ADAS-equipped vehicles with limited telematic capabilities may not be included in the General Order if the consumer did not state that the automation system was engaged within 30 seconds of the crash or if there is no other available information indicating Level 2 ADAS engagement due to limited data available from the crashed vehicle.”
- “By contrast, some manufacturers have access to a much greater amount of crash data almost immediately after a crash because of their advanced data recording and telemetry.” (Read: Tesla)
Most likely, most of that is the NHTSA rewriting what automakers told the NHTSA as to why it didn’t really have any data to hand over. Here’s my hypothetical example of what automaker lawyers or executives told the NHTSA: “Well, we don’t have real on-the-road data on this because our vehicles are not equipped to collect such data, and certainly not equipped to automatically send it back to us. The only way we know if an advanced driver-assist system (ADAS) was in use when a crash occurred is if a vehicle owner notifies us to tell us this.”
It’s great that the NHTSA is now working to collect data on this matter and investigate the safety of different systems, but at the moment, the agency simply doesn’t have the data. Automakers don’t have the data and thus can’t send it over.
Perhaps, with cars getting smarter and smarter, and with automakers including more and more sophisticated ADAS, the data will start to flow and the NHTSA will have something useful to do in comparing and improving systems. On the other hand, with this investigation and reporting system now in place, automakers may also be reticent to start improving their in-house data collection — and risk regulatory or legal issues.
There’s plenty more in the NHTSA report. Check it out to explore further, but at the least, don’t be tempted to jump to conclusions about “US ADAS crash data” that is drastically incomplete.
Appreciate CleanTechnica’s originality and cleantech news coverage? Consider becoming a CleanTechnica Member, Supporter, Technician, or Ambassador — or a patron on Patreon.
[embedded content]
Advertisement
This post has been syndicated from a third-party source. View the original article here.
