The recent woes of robotaxi company Cruise bring to light many interesting problems currently plaguing the autonomous vehicle industry, most importantly issues around transparency and public trust.
How are we supposed to trust these companies with our lives — and indeed, you are putting your life in the hands of a company when you ride in their driverless cars — when they don’t see fit to disclose all of the information we need to best understand the technology?
Last month, the California Department of Motor Vehicles suspended Cruise’s license to operate robotaxis after a Cruise vehicle struck a woman who was flung into its path by a hit-and-run driver. The driverless Cruise vehicle ran over her, stopped briefly, then proceeded to drag her for 20 feet while pulling over to the curb, causing her severe injuries.
How are we supposed to trust these companies with our lives?
In response, Cruise grounded its fleet nationwide while it investigates what went wrong. The company’s board hired an outside law firm and technology consultants to review its response to the incident.
One of the questions seems to be whether Cruise withheld key evidence from the DMV. The department claims the company showed video of its car braking hard and stopping but failed to show footage of it dragging the victim as it pulled over to the curb. In its statement on the suspension, the DMV alleges that Cruise “misrepresented” information related to the safety of its vehicles. But Cruise disputes the claim, arguing that it showed the entire video to the DMV.
To be sure, Cruise is in a tough spot. Its vehicle was very clearly in the wrong place at the wrong time. A human driver (who still hasn’t been apprehended, despite an abundance of video evidence of the incident) committed a crime for which the autonomous vehicle has paradoxically taken most of the blame. But tragically, the incident is now leading to a fuller public understanding of how these vehicles operate. And that shouldn’t be the case.
Last week, The New York Times published a story about Cruise’s reaction to the suspension and how the company’s aggressive, hard-charging mission — with assistance from parent company General Motors — may have contributed to the present situation. Toward the end of the story, reporters Tripp Mickle, Cade Metz, and Yiwen Lu toss out an interesting data point: Cruise employees “intervene” in the company’s vehicles every 2.5 to five miles.
In a fascinating twist, Cruise CEO Kyle Vogt responded to the Times story — not in the Times itself but in a comment to a post on Hacker News about the Times story a couple of days after it came out.
“Cruise AVs [autonomous vehicles] are being remotely assisted (RA) 2-4% of the time on average, in complex urban environments,” Vogt wrote. “This is low enough already that there isn’t a huge cost benefit to optimizing much further, especially given how useful it is to have humans review things in certain situations.”
Setting aside the extremely 2023-ness of how this comment came to light, Vogt’s offhand confirmation of the Times’ reporting, while adding additional context about remote assistance, is extremely interesting for a number of reasons. First of all, it’s more information than we typically get from Cruise, which has a reputation of being very tight-lipped about its operations.
“Cruise AVs [autonomous vehicles] are being remotely assisted (RA) 2-4% of the time on average”
All autonomous vehicle companies operating in California are required to report the number of miles driven, the number of vehicles, and the number of “disengagements,” which almost every expert agrees is an entirely worthless number. But beyond that, AV companies are allowed to operate almost entirely in secrecy. And as we can say from these past few weeks, that is a major problem.
“Companies tend to only publicly report numbers that regulators force them to report, or that they think make them look especially good,” said Philip Koopman, a Carnegie Mellon University professor who has conducted research on autonomous vehicle safety for decades. “That is true of all car companies, not just Cruise.”
California has the most stringent disclosure rules around AV operation of all the states. Most of the companies that still have cars on the road are testing there. It’s home to Silicon Valley and, therefore, most of the engineering talent. But other states that have a lot of driverless cars, like Arizona and Texas, have virtually no disclosure rules. And that means that the only updates the public gets are through a company’s own marketing material. That is not a recipe for building public trust.
Only when something goes wrong do these types of facts and figures come to light. It took Uber killing a pedestrian in Tempe, Arizona, in 2018 for us to learn more about the company’s lax approach to safety or the fact that it turned off the factory-installed automatic emergency braking mechanism in its vehicle that could have saved Elaine Herzberg’s life to avoid conflicts with the autonomous driving system.
“Companies tend to only publicly report numbers that regulators force them to report, or that they think make them look especially good.”
Now, another person is severely injured, and we are learning new information about the level of human involvement in the decision-making of Cruise’s vehicles. Don’t get me wrong: I’m glad that Cruise is opening up about the level of human help its vehicles still require. I just wish it were under better circumstances.
Experts I spoke to agree that Vogt’s disclosure about remote assistance is interesting, less so in the context of the incident involving the hit-and-run victim and more so for the ability of robotaxi companies to operate a service that makes financial sense.
“The frequency of remote assistance being reported is not surprising and one of the many hidden challenges to the business model of automated ride hailing,” said Bryan Reimer, a research scientist at MIT’s AgeLab and associate director of the New England University Transportation Center. “Until remote assistance is driven out of the equation we are talking more about a highly automated system with periodic human intervention vs. the dream of autonomous ride hailing.”
Only when something goes wrong do these types of facts and figures come to light
For years, driverless car operators have been selling this dream of human-free driving. But the reality is that the cars still need a human eye — and occasionally, a human hand. And we still don’t know a lot about these remote monitors. How many cars does each remote operator watch at a time? They may not be controlling the cars from afar — also known as teleoperation, or by some as “joy sticking” — but they are helping the cars make decisions in some capacity.
Those people need to get paid. And if the point of driverless cars is removing humans from the equation to make them cheaper to operate, then the financial impact of keeping all of these people on the payroll starts to add up.
“Instead of a human driving the vehicle, one is now potentially assigned to each ‘driverless’ vehicle on the road,” said Raj Rajkumar, professor of electrical engineering at Carnegie Mellon University. “And that remote operator is likely getting paid more than a normal cab driver – causing problems to the core business model at present.”
This is relevant information that cities may want to have as they weigh decisions on whether to let driverless cars flood their streets. What are the tradeoffs? If robotaxis take off and human ridehail drivers start to find themselves out of jobs, what are the financial impacts? Are we just robbing Peter to pay Paul?
Again, we would never know these numbers had Cruise’s Vogt not felt compelled to respond to the Times’ reporting. There are no rules requiring Cruise to disclose these numbers — or Waymo, Zoox, Aurora, Motional, or any other autonomous vehicle company. Some states have a few rules, but most do not have any. And legislation that remains stalled in Congress would do little to change this fact, even if, by some miracle, it became unstalled and found its way to the president’s desk. The legislation, which has largely been promoted by the tech industry, would remove barriers to putting more driverless cars on the road — not impose any new disclosure rules.
How many cars does each remote operator watch at a time?
The result is that driverless cars, a technology that has the potential to change a lot about how we get around, are losing trust. Consumer confidence is down for the second year in a row, according to an annual survey conducted by JD Power. And this is at a time when the technology remains out of reach to the vast majority of people in the country. With only a small number of deployments in California, Texas, and Arizona, most people are still not exactly impressed. And the companies aren’t doing enough to address this.
Waymo has arguably done more than Cruise, reporting crash stats and publishing in-depth analyses on how its cars measure up to human ones. The purpose is to provide some statistical foundation to its core argument that robot-driven cars can operate more safely and predictably than human-driven ones.
But as Trucks VC general partner Reilly Brennan likes to argue, autonomous vehicle companies are all in the same boat — like it or not. “Each company’s success is individual, but their failures are often clumped together as if all operators were one company,” he wrote recently, dubbing the sentiment WALTER, for “win alone, lose together.”
Cruise’s stumbles become Waymo’s problems, too. Until some entity steps in to establish a clear legal and regulatory foundation, this will likely remain an enormous hurdle for all of the companies that dream of robots ruling the road.
