The case for a short-term AI bubble
An unfinished draft from June 2024. Several sections have not been filled in beyond a bullet-point sketch, and a few placeholders for specific numbers (“$X trillion”, “$Y trillion”) remain unresolved; these have been preserved as written. This post argues that a short-term bubble in AI has, say, a 60% chance of popping over the next five years. To be clear, I’m not making claims about the long term, or even the medium term (past 2030). I’m also not very confident; uncertainty seems appropriate. But right now, a bubble popping looks reasonably likely, while my guess is that many in AI see it as an unlikely tail risk (or don’t think about it at all, maybe, for unreflective Twitter pundits). The Dot-Com Bubble: A Case Study this seems like an appropriate analogy for several reasons; the Internet was really a huge deal, but most of the benefit didn’t come until later the value of the NASDAQ in 1999 was $X trillion the value of tech companies in 2024 is $Y trillion. The value people expected was real! It just took longer to arrive than investors thought at the time “most people overestimate change in the short term and underestimate it in the long term” AI Investment Is Massive As of June 2024, Nvidia’s valuation is around $3 trillion. That’s the second-largest company in the world, but one could justify it if one thought AI would become much of the economy, and Nvidia would get a big chunk of that. More importantly, though, Nvidia’s Q1 datacenter revenue (almost all AI) is $23 billion, or around $100 billion annualized. That’s a crazy amount of investment. It’s around 0.1% of world GDP; but most of world GDP goes to immediate consumption (food, cars, electricity, and so on), and can’t be redirected easily. It’s about X% of all US investment. It’s about twice NASA’s budget at the height of Apollo, or about four Manhattan Projects per year (inflation-adjusted), or around ten times all spending on US interstate highways. Notably, unlike investing in startups, this investment must deliver returns now . Nvidia equity, or AI startup equity, can have high valuations because of high expected returns in 2035 or later. But AI chips can’t, because they become obsolete and depreciate quickly; no one wants a five-year-old chip, never mind a fifteen-year-old chip. So for this investment to be profitable, AI must deliver enough value by, say, 2027 to pay for hundreds of billions in accelerator spending (plus other costs like electricity, servers, cooling, technician and DevOps labor, the labor of training models, etc). The near-term question is not, “can AI ever deliver $500 billion in value?”, but “can ten million H100s ever deliver $500 billion in value?”, in the few years before they get replaced by Blackwell chips. Diminishing Marginal Returns Space writers sometimes say that an asteroid is worth some absurd amount, like $50 trillion, if you mined it and brought it to Earth. The easiest way to get those numbers is to just take the market cap of each valuable metal, per kilo, and multiply that by how much there is in the asteroid. However, if you brought the asteroid to Earth, you couldn’t get $50 trillion for it, because selling tons and tons of (say) platinum would crash the price. The first bits of platinum you find, the ones people have already extracted, get used for the highest value applications, like catalytic converters. A new chunk of platinum can’t be used for catalytic converters, because that role is taken by existing platinum; it’ll have to be used for something less important. If there were billions of tons of platinum laying around, people would use it as filler for sandbags. Likewise, it’s true that: the total value of the software industry is $X trillion a year even in the short term, AI could easily double or triple the amount of code written but that wouldn’t be $X*2 trillion of value, because the second billion lines of code is worth much less than the first billion. All of the easy spots to create economic value with code, like replacing travel agents and manual payrolls, are already taken. Amdahl’s Law famous law in computer science; in a serial computer, if you parallelize X% of it, you can never make it faster than 1/(1−X), because it’s still bound by the serial part eg. AI helps a lot with education, but a lot of education is physically watching kids, which current AI doesn’t help you with even an arbitrarily good education LLM, like Neal Stephenson’s Young Lady’s Illustrated Primer, won’t replace a lot of education, because it won’t prevent eight-year-olds from making physical messes Integration Is Slow The idea of using conversational AI to replace customer service phone trees (IVR systems), which both customers and businesses hate, is obvious. It’s so obvious that I was working on a startup for that (Apprente) seven years ago, long before GPT-4 or even GPT-2. With 2017 technology, we decided that replacing customer service was overly ambitious, so we picked the easier problem of AIs taking orders at restaurant drive-thrus. We made a good beta product, and successfully got acquired by McDonald’s in 2019. However, the system was never deployed to the vast majority of restaurants, and was recently cancelled completely. There was nothing wrong with our basic idea, but the vast majority of our time was spent dealing with things like: integration into the point-of-sale systems at McDonald’s restaurants dealing with the McDonald’s menu, which had thousands of items, and was partly not even computerized (each store manager gave staff verbal instructions) issues with the physical speakers and microphones at McDonald’s windows unrelated business disputes between McDonald’s headquarters and franchisees random surprises like McDonald’s celebrity promotions, which were both a large fraction of all orders, and were announced with very little lead time If AI were truly general, and could replace all human labor, then you could simply start a fully AI-powered restaurant to compete with McDonald’s and steal its customers. But if AI can, say, increase McDonald’s revenue by 10%, that will take years to happen because big corporations (and especially governments) move slowly. The Economy Is 95% Physical Something like 95% of the world economy is about producing physical products and services. Ultimately, this is because people are made out of meat, and most human needs are physical (food, shelter, transportation, healthcare, child care, elder care). This is obvious for agriculture and manufactured goods, but almost all “services” still have a physical component. A very good LLM can replace calling a doctor, but it can’t replace a physical stay at a hospital; it can replace a textbook, but it can’t replace supervising unruly eight-year-olds; it can replace a recipe book, but it can’t cook and deliver a meal, and so on. Indeed, since skilled American labor is very expensive, any service that can easily be “virtualized” was probably outsourced to cheaper countries years ago. Of course, that doesn’t mean AI can’t help with physical tasks. Three obvious options are building new robots, turning existing machines into robots (as with self-driving cars), and making human workers faster with really good AI-powered HUDs. But current LLMs aren’t set up to do this at all. Multimodality is one step forward, and the low latency of GPT-4o is another step, but what you really want is real-time high resolution video as an input and an HUD as an output (think of what’d be most useful for, say, a surgeon or construction worker). I’m pretty confident a product like that will exist before 2035, but it doesn’t exist now , and it’s not on the immediate horizon for Q3 2024. Even GPT-4’s “agent” functionality, announced over a year ago and limited to calling existing APIs, still hasn’t been deployed AFAIK. Much of GPT-4o also still hasn’t been deployed. Not All AIs To make economic sense, ten million H100s have to be able to generate something like $500 billion in revenue , not just $500 billion in value. This will probably be tricky, because in a free market, the price drops to the marginal cost of production; the marginal cost of software is zero, and the marginal cost of unused on-device GPU cycles is epsilon. Any value that can be provided by, say, Llama-3-8B, or things like better text classifiers, can’t defray the cost of the H100s, because no one will pay more than epsilon for them. $500 billion must come from, not just “AI”, but the subset of AI tasks that can only be done with a cutting-edge model, or one that is so large that it needs big H100 servers for inference. Value Uncertainty In addition, a lot of what AI does is very hard to value accurately (as is most information). For example, IDEs are an existing tool that makes software development faster. It’s not crazy to guess that they increase programmer productivity by 10%. But JetBrains is an $X billion company, not a $Y billion company, because 10% * the value of all software is the total value, and most of that is consumer surplus, not revenue (in fact it’s even less, because of the earlier point about diminishing marginal returns). In an ideal world, if my productivity increased by 10%, my salary would go up by 10%, and I’d be happy to spend 9% of my paycheck on that productivity tool if that was the market price. In reality, I might spend $20 a month, because my employer’s estimate of my value is very fuzzy, labor markets are inefficient, and it’s hard even for me to judge exactly how much benefit I get. Electricity, Eee-Lec-Tri-City scaling is going to run into hard constraints on datacenter electricity very soon 25,000 H100s draw around 25 MW of power