Self-driving car safety
A few days before this was written, an Uber self-driving prototype struck and killed pedestrian Elaine Herzberg in Tempe, Arizona — the first death involving a fully autonomous vehicle on a public road. Industry boosters had been promising imminent commercial deployment for years.
Some may have heard that an Uber self-driving car hit an Arizona pedestrian, killing them. Although N = 1 is too small for firm conclusions, since self-driving cars have driven under ten million miles on public roads, we should expect zero deaths if they were as safe or safer than humans. (US fatality rate is currently ~1 per 100 million miles.) IMO, this is an important data point for overall AI development during the next few decades; there’s a big gap between “invention”, and consistently matching human performance in real-world conditions. I wrote about this to a friend last July: “The DARPA Grand Challenge showed that self-driving cars was ‘solved’, in the sense that a car existed that could do things reasonably described as self-driving, but it’s twelve years later and we still don’t have a date for commercial deployment (even though it’s being worked on basically as hard as current tech companies can). I’d expect there to be a period where systems exist that can be called ‘AGI’, but where they present minimal danger on account of being very buggy and/or very slow. This gives us additional lead time but may divert attention from safety research.”
They replied that they thought regulation was an obstacle for AI cars, to which I responded (several months later): “I strongly disagree with this view. I know of no evidence that any significant political force opposes introducing self-driving cars, and ample evidence that many governments have tried to actively cooperate with their development, insofar as they practicably could. There is only one example I know of where regulators ‘interfered’ with self-driving cars, namely George Hotz and the comma one, and the ‘interference’ consisted of sending them a letter asking the following questions (paraphrased by a Hacker News commenter): What’s in the installation + operating manual? — Items 1–3 Is it safe? Did you do testing? How? — Items 4–6 What vehicles does it work on? What happens if you screw up during installation? — Items 7–8 What happens if an owner screws up and puts in an unsupported car? — Item 9 Did you think about whether this keeps the car compliant with other safety regulations? In particular, did you think about the legality of blocking/removing the mirror? Did you do testing? — Items 10–12 When are you selling/shipping it? — Items 13–14 Anything else you want us to know? — Item 15 (the original list is at this Hacker News thread if you want to review it, although I think the paraphrase is fair) The overwhelming belief among well-informed commenters was that Hotz shut down his company immediately after receiving this letter because he did not do any basic safety testing, and was in fact nowhere near production ready (despite claiming this to the public). The technical reality is that present-day machine learning systems are notoriously hard to make reliable, which is why Google of all places resisted putting ML into their search rankings for over a decade. Just matching human performance in driving requires only one fatal accident per hundred million miles driven, or something like ten-nines reliability, and that’s really hard and requires many years of engineering even after something ‘works’ in a demo.”
I think the recent death is more evidence for this, showing that regulators have been cooperating even to the extent of allowing self-driving systems on public roads that were likely more dangerous than humans. (I’d also note that, from a tech perspective, all these comparisons are biased, since it’s really human vs. human plus AI rather than human vs. AI. All AI cars regularly “disengage” and let a human take over, while this is not an option for a human.)