backtop


Print

While recent report dwells on sensationalist hypotheticals regarding pedestrian collisions, the facts speak to a different truth

So far only three states -- California, Florida, and Nevada -- have passed laws allowing self-driving cars to freely travel their roadways (the District of Columbia also has a law allowing self-driving cars).  Michigan -- America's "automotive capital" -- also has laws allowing more limited testing.  Given the proximity to Silicon Valley, the majority of self-driving vehicle miles have been logged in California and Nevada.  While the sample set for these vehicles' performance is still relatively small, a hot button question is how these vehicles are stacking up safety-wise.

I. Self-Driving Vehicles -- A Threat to Mankind?

An Associated Press (AP) report published this week revealed that since Sept. 2014 (the past nine months/three-quarters-of-a-year) four self-driving cars have been involved in crashes in the state of California.  Google Inc. (GOOG) -- a pioneer in the field -- owned vehicles involved in three of the crashes.  The fourth involved a vehicle owned by Delphi Automotive plc (DLPH), a top supplier to traditional automakers.

Google currently maintains a fleet of 23 self-driving vehicles in the state of California, according to statistics from the state's Department of Motor Vehicles.  The base vehicles for its fleet were purchased from Toyota Motor Corp.'s (TYO:7203) Lexus luxury imprint (earlier Google used Toyota Prius hybrids fro the project).

The idea of self-driving cars crashing is certainly a scary one as it evokes widely held fears of out-of-control AI.  Combined with the very real growth that Google's fleet has seen in recent years, it's only human to be a bit concerned about these new robotic motorists.  It's important, however, to exercise discipline in examining these figures.  While it's important to scrutinize the safety performance, it's equally important to avoid falling into a sort of fantastic luddite hyperbole when it comes to self-driving cars.

Google's self-driving cars
Is Google's fleet of self-driving cars a threat to motorists and pedestrians? [Image Source: AP]

With that in mind let us examine Google's response and then draw some quick conclusions about what it means.

In an official blog post response carried by Medium,  Google engineer Chris Urmson reports that the company's self-driving cars have seen eleven crashes in California over the past six years.  He writes (emphasis his):

Over the 6 years since we started the project, we’ve been involved in 11 minor accidents (light damage, no injuries) during those 1.7 million miles of autonomous and manual driving with our safety drivers behind the wheel, and not once was the self-driving car the cause of the accident.

(Note: Urmson says the vehicle algorithms drove for roughly 1 million of the 1.7 million miles, while human assistants took over for roughly 700,000 of them.)

The final statement is sure to provoke debate, as blame in any vehicular accident is always a debatable issue.  Were Google's cars truly the innocent victims of clumsy human motorists as it suggests?  To a degree likely yes, but the truth is likely somewhat more complex that Google's Urmson suggests.  Even self-driving cars likely make minor mistakes or misjudgements, which some cases may put them at risk of getting in accident, even if the vehicle was in the right in terms of road laws.

Open roads
Let's not guess at the dangers -- let's get the facts and draw real conclusions. [Image Source: Bing]

Blame is a budensome beast, so let's not preoccupy ourselves with that.  The more important hard statistic here is the number of accidents per miles travelled.  Also useful is the information on the type of miles (mostly city streets) and how often the AI was behind the wheel.

II. Man vs. Machine (Driving Edition)

Averaging the crash rates. we get that Google self-driving cars get in roughly 6.5 crashes per million miles traveled.

By itself that statistic isn't overly meaningful.  Thus we first need to find crash data on human drivers for the sake of producing meaningful comparisons.  Finding in-depth data on none-fatal pedestrian traffic accidents is a bit difficult.  Indeed, from what I could ascertain, massive in-depth surveys relating to the details and demographics of accidents is conducted infrequently -- perhaps only once or two times per decade.

I was able to locate one such survey, albeit a slightly dated one -- a 1990 report [PDF] authored by Dawn L. Massie and Kenneth L. Campbell of the University of Michigan Transportation Research Insitute (UMTRI).  The report provides the kind of in-depth statistics that we can use to dig into accident rates among human drivers and extrapolate based on the more minimal annual information collected by the U.S. federal government.

Car crash
Annual car crashes have remained relatively steady at roughly 5-6 million per year over the past 25 years.  Most accidents don't involve injury or death. [Image Source: Streets Blog]

In 1990 there were approximately 5.6 million involvements in crashes.  In 2013 there were approximately 5.6 million crashes, according to a report [PDF] by the U.S. Department of Transportation (DOT).  So while the outcomes, causes, and demographics may have shifted over the two-and-a-half decades since the 1990 report, we can assume that the general number of crasheshave stayed relatively similar.

What has shifted is the number of miles travelled.  In 1990 drivers travelled approximated 1.6 trillion miles.  In 2013 drivers travelled approximately ~3+ trillion miles [source].  So travel rates have roughly doubled, which ultimately indicates a 50 percent overall reduction in traffic accident rates.

USA vehicle miles travelled
Vehicle miles travelled has roughly doubled over the past 25 years. [Image Source: TravelAdvisor]

With that in mind, let's first examine the rate of all accidents (including ones without injury or death), extrapolating to the present tense.

The 1990 report stated that on average, drivers were involved in 6.08 crashes per million miles.  If you looked at just teen drivers -- the highest risk group -- they experience more than 20 crashes per million miles.

So from a simplistic perspective Google's self-driving cars are doing pretty well.  They're close to the average in terms of crash rates for 1990, and are much safer than the average teenage driver, statistically speaking.  In the present context, they're likely about twice as likely to get in a crash as a current driver, given the decline in rates of accidents between 1990 and present.

III. Highway vs. City Street and the Injury Angle

But are the cars really getting in nearly twice the number of accidents as the average driver in 2013?  The answer -- and twist -- lies in where human motorists are getting in accidents and where Google vehicles are most frequently driving.

According to the U.S. DOT's Federal Highway Administration (FHA), in 2013 more than 9 out of 10 vehicle miles travelled was on a highway.  Assuming Google's assertion about 55 percent of accidents occurring off the highway holds true, we can estimate rough averages of 12.5 and 0.9 accidents (on average) per million miles travelled on the surface streets and highways, respectively.

Google self-driving Prius
Google's self-driving cars project has remarkably seen an accident in which either driver was injured.

Urmson writes that "most" of the driving is on city streets.  Assuming that means >90 percent of miles travelled, that could indicate that Google's accident rate of 6.5 crashes per million miles traveled is roughly half as many crashes as the average driver experiences, when considering that the majority of miles were travelled off the highway -- a setting that sees more than a 10-fold increase in accident rates per million miles travelled.

So far things are looking pretty good for Google.  Let's next examine accidents with fatalities or injuries.

There were roughly 0.03 fatalities per million miles driven, so on average a fleet of cars would have to travel 33 million miles (roughly) to see a fatal crash.  Google's fleet hasn't seen any fatal crashes, but statistically speaking, it is quite unlikely to have, given the number of miles travelled.

Lies and self driving cars
The statistics can be manipulated to suggest self driving cars are too dangerous, but in reality the full picture indicates they're better drivers than humans. [Image Source: Jason Mick/DailyTech LLC]

On average, though, there were 2.04 injuries per million vehicle miles travelled (VMT).  So statistically speaking, we'd expect roughly 3.5 injuries from Google's fleet at this point, from a high level/paper napkin style analysis.  The 2013 DOT report indicated a 0.78 rate of accidents with injury per million miles traveled.  So even at today's reduced accident rates, you'd expect Google's self-driving cars to have been involved in at least one accident with injury by now.

Google has seen no injuries, which means either its self-driving cars are very lucky -- or they're better than humans at avoid bigger mistakes (the kind that lead to a more serious injury-causing collision, fault aside).

IV. Executive Summary

So let's recap.  Basic stastics suggest:
  • Accident rates have fallen roughly in half for human drivers in the past 25 years (likely thanks to modern safety features).
  • In the past 25 years the population has grown roughly 25 percent [source], while vehicle miles travelled by all U.S. drivers have doubled.
  • Total accidents have held roughly steady over the past 25 years given the greater number of miles travelled both for all drivers and on a per capita basis.
Google self-driving cars

In terms of self-driving cars, the numbers indicate:
  • Google self-driven cars are getting in roughly 50 percent less crashes in off-highway driving than average drivers.
  • A teen is roughly seven times more likely to crash in off-highway driving than a Google self-driving car.
  • Although the statistical sample of miles travelled is small, Google cars appear to be well below the average rate of serious accidents (accidents in which at least one driver is injured).
Add in that the AP report states...

Regulators in Nevada, Michigan and Florida told The Associated Press that they weren't aware of any accidents.

...and it's starting to appear as if self-driving cars are substantially safer than humans.  That's important when you consider the scrutiny self-driving vehicles are currently receiving from federal regulators.  It's important not to whitewash autonomous driving failures, but it's equally important not to let sensationalist media commentary drive (pun not intended) a luddite regulatory agenda.

V. Sensationalist Hypotheticals vs. Reality

And yet the AP report quotes University of South Carolina law professor Bryant Walker Smith as saying:

Assuming that you are not dead, you are in a much better positon than if you had been hit by an ordinary, human-driven vehicle.

The report does clarify in the next paragraph that the statement is talking about a pedestrian victim's legal recourse, not their physical condition, although perhaps that should have been stated first to avoid the ambiguity.  The premise is that a company like Google has more financial resources to defend itself against lawsuits, than the average motorist who strikes a pedestrian.

car crash w/ pedestrian
While there has been no known incident to date of a pedestrian getting hit by a self-driving car, the AP report dwells on that possibility. [Image Source: Barrera Law Firm]

But the whole discussion is hypothetical in the first place as self-driving vehicles haven't hit a pedestrian (or at least the report makes no reference to any such known incident).  Given the tone, you could say the talk about pedestrian injuries/fatalities is treated as a hypothetical bordering on hyperbole in the piece.

Further, with all due respect Professor Smith's claims are flawed on a couple of grounds.  
First, self-driving car makers can face class action lawsuits from pedestrians, which would potentially give them better chances in court versus standard individual litigation against human drivers.  Second, those "deep pockets" of automakers (and companies like Google) will only work to stifle litigation if the number of cases is small.

Lastly, the report does mention in passing that the vehicles carry much higher insurance premiums -- $5M USD per car, in Google's case.  Thus there'd be more insurance money up front to reach a payout to an injured pedestrian if accident rates are similar or less than everyday drivers.

The report does emphasize one positive -- under Californian state law self-driving cars are required to carry black boxes that can preserve 30 seconds of data prior to the present, data which is permanently saved the second a crash occurs.  As Santa Clara University law professor and insurance expert Robert W. Peterson points out to the AP, that data could actually give pedestrians the edge in court, in addition to dramatically reducing injury incidents.  He states:

[If a self-driven vehicle] fails to behave in a way which a reasonable consumer would expect it to behave, that is a defect.

Oh and it's also important to note that the Delphi vehicle crashed when a human was driving it.  So that "4 crashes" is really "3" in the context of the majority of the article.  The fact that the AP report left that fact off till the end of the piece is troubling.  Further, the true number may be even lower as the AP did not state whether the Google vehicles crashed under AI control or under human control.

VI. Closing Thoughts

Reports such as the recent one by the associated press are likely inevitable as most states with self-driving cars are required to report crash statistics.  And when it comes to crash statistics it's easy to sensationalize and sensationalism can be a lucrative approach.

deflated airbags
The AP's sensationalist hypotheticals are deflated by factual analysis. [Image Source: Auto Trader NY]

To that end the following can be said of the AP piece:
  • It devotes nearly two-thirds of its text to a hyperbolic hypothetical that hasn't happened -- a self-driving car striking and injuring a pedestrian.
  • It devotes no substantial space to comparing self-driving vehicle accident rates with human ones.  (If it had, the sensational angle would quickly die or diminish.)
  • The report to some degree confounds AI self-driving with human driving in a car with self-driving capabilities.  The headline is technically accurate, but misleading.
  • After driving towards a seemingly sensationalist conclusion, the report devotes much of the remaining space to backtracking on its early suggestions (namely, to unrolling its suggestion that passengers are at more financial risk from Google self-driving cars).
I would thus conclude the AP piece is deeply flawed.  And while Google's response blog does offer some seemingly ambiguous statements that are also problematic (e.g. it does not appear to state how many times the self-driving vehicles were struck while manual driving), in the end statistics seem to be on the side of Google and company.  Self-driving cars aren't failproof, but they seem to be safer than human drivers, thus far.

I'm rather surprised to see this kind of spin coming from AP, as in my experience usually they do an impeccable job thoroughly report on the facts.  If anything, typically I find that their coverage might be a bit too dry or rigorous for most readers.  To see them devote so much to an inflammatory "what if" is rather out of character for their technology coverage.

It doesn't take a rocket scientist to figure that out from the numbers and information freely available online, but it's important to document that conclusion given that AP failed to do so.

Ethical Disclosure:
I have no financial relation with Google or any other automaker, I own no automotive stock (or Google stock), and I do not work for Google or any other automotive affiliate.  My sole interest in this analysis is as a futurist interested in statistics and the true story.

Addendum:
If you find any flaws in my estimates, please email me and let me know where.  Many of the statistics I presented are direct from the source, but some calculations were involved to extrapolate other useful metrics.  Refer to the source links if you want to double check my numbers.

Sources: AP [Yahoo News syndicated], Google [on Medium], via USA Today





"Google fired a shot heard 'round the world, and now a second American company has answered the call to defend the rights of the Chinese people." -- Rep. Christopher H. Smith (R-N.J.)







Latest Blog Posts






botimage
Copyright 2017 DailyTech LLC. - RSS Feed | Advertise | About Us | Ethics | FAQ | Terms, Conditions & Privacy Information | Kristopher Kubicki