Getting a “lead” in AI research
I’ve heard several people, most famously Robin Hanson, speculate that the impact of any particular research group on AI’s trajectory will be minimal, because no group can get a significant “lead”. That is, it’s much easier to copy something than to invent it, so any important discovery will rapidly be copied. Since anything that one group invents will quickly become available to all, it doesn’t matter much if, eg., Google commits to not using AI for military purposes; the military will get it soon anyway. And the data seems to confirm this: if you hear about an advance in AI, then search Google Scholar for replications and similar work, you’ll probably find many examples.
However, my experience with Apprente (now McDonald’s Technology Labs) has made me very skeptical of this view. Although light-years from AGI, I think we did advance conversational agent technology in a commercially meaningful way. Our company came out of stealth mode over a year ago, and our acquisition was featured in Bloomberg, TechCrunch, Wired, and many other publications. And although we didn’t publish our source code, we did have several patents on our technology, and even published a paper at NIPS . But to the best of my knowledge, no one has replicated our system, or is even trying; no one has ever noticed or read the patents; and the NIPS paper got two citations, one of which was a literature review.
I think this mismatch between model and data largely boils down to different incentives. Becoming an American academic requires years of competing for academic prestige, and you get that prestige by broadcasting your work to a large audience, especially the sort of people who would replicate it, or build a better version themselves. This creates incentives like: publishing, as often as possible, in prestigious journals in your field; selecting problems that are well-known, and whose solutions are known to be hard, so that an advance will be clearly impressive; choosing performance metrics which are well-established and unambiguous, to make improvements easy to see; focusing on a problem’s math and pseudo-code, rather than a robust and maintainable implementation, to better fit the peer-review process; choosing techniques and methods that, although new in some way, will be familiar to peers and fast to explain to them.
The incentive for maximizing press coverage is less clear — some parts of, eg., academic math seem to discourage that — but DeepMind and OpenAI also do a lot of it, perhaps to attract job candidates or get more funding from Google/Microsoft. In any case, for an engineering project with a direct goal, whether commercial, military, personal, or altruistic, these incentives mostly go away. You don’t have to go all the way to “airtight secrecy” to slow down replication; just removing the active incentives to make projects that quickly replicate will have a very large effect. (In other fields, I’ve seen it take decades between someone discovering an important fact, and it being widely shared. In medicine, the usual figure is “17 years”, which is… well, it’s the right order of magnitude.)
But suppose you did go all the way to “airtight secrecy”. Consider the classic scenario of, eg., a military or intelligence-centered Chinese AI project, with the goal of getting a technological advantage over the West. For a group that wants to replicate the state-of-the-art, or even just keep tabs on the field, there are huge barriers even compared to our own project: the secrecy can be much tighter, with formal information controls, no website, no LinkedIn profiles, no publicly visible office in the middle of Silicon Valley, etc.; if anything does leak, it will be written in Chinese, so very few Westerners will be able to read it; previously, I’ve found that Google and Bing tend not to index Chinese-hosted documents, so a leak might not be visible even if you searched the right terms; most of the researchers and engineers may be relative unknowns, without the long academic records and publication histories that give you a sense of what eg. Harvard professors are working on.
At a guess, I’d expect the US government to really start noticing this only when it was actively being used for military applications, to blow up tanks, build robot soldiers, make drones more maneuverable, etc. The broader AI community might only find out about it years after that, perhaps when some other group happens to independently replicate important parts in a more public way.