Toy models of the intelligence explosion
(MIRI’s Intelligence Explosion Microeconomics addresses some of these questions, but doesn’t appear to consider the types of models I’m thinking about, and at any rate is somewhat of a rambling mess.) In previous research, Eliezer has talked about the “nuclear fission model” of strong AI. In this framework, once a certain critical threshold is reached, an improvement to intelligence causes more than one additional improvement, each of which in turn causes more than one additional improvement, and so on, creating very rapid growth until hard limitations of the underlying substrate are reached. This is analogous to the nuclear fission chain reaction, where every neutron causes the emission of more than one additional neutron. Many fission neutrons move at relativistic speeds, so the nuclear chain reaction can happen very fast indeed. However, it seems like this analogy is not fully accurate. The speed of neutrons, and therefore the reaction doubling time, is determined by the underlying physics of hadrons, which are independent of human actions and are constant in every environment. But the speed of computer self-improvement isn’t. A very powerful (but non-saturated) system, running on Jupiter Brain hardware, could indeed have very short doubling times on the order of seconds or minutes. However, (eg.) a single uploaded human brain running in real time wouldn’t; we know from neuroscience that the brain is very hard to modify in a stable, positive way. It could be done, almost certainly, but the doubling time would likely be on the order of millennia. At the extreme end, one could say that we’ve already built a recursively self-improving intelligence, since we could probably figure out a general enough evolutionary algorithm to simulate the entire history of life; it’s just that the doubling time would be impractically large (maybe on the order of ~10^30 years, with current hardware). Hence, the overall doubling time of the capabilities of AI is determined by some combination of a) the hardware doubling time (Moore’s Law), b) the rate at which humans can write better software, c) the rate at which the AI itself can write better software, and d) limitations caused by low-hanging fruit depletion, since there’s a limited number of easy problems before one must tackle the hard problems (ultimately bottoming out in physical limits). In Eliezer’s model, we might assume that c) is only a function of c) and d); if we assume the doubling speed increases linearly relative to existing software capabilities and we ignore d) for the time being, as a toy model we get the differential equation: dy/dx = cy/d = cy^2 (linear constant c) which has the solution: y(x) = 1 / (c1 − cx) This is a very fast hyperbolic growth curve (going to infinity). If we start with arbitrary initial conditions of x = 0 and y = 1 and a linearity constant of 1 (for an initial doubling time of, say, one week), we get y(x) = 1 / (1 − x), which is an alarmingly fast curve. The first order-of-magnitude increase takes six days; the second takes fifteen hours; the third ninety minutes; the fourth nine minutes; the fifth fifty seconds, and so on. However, if we assume that c) is a function of all of a), b), and c), we get a different model. Still ignoring physical limitations, suppose that the doubling time d is two years for improvements to software and hardware (the general consensus is that Moore’s Law will slow down somewhat in the short/medium term), multiplied by a factor of 1/(1 + cy) for the rate at which computers can improve themselves. Since this is currently negligible, we might want to start out with x = 0, y = 1, and a very low c of (say) 10^−20. This gets us the differential equation: dy/dx = y/d = y/(2/(1 + 10^−20*y)) = y * (1 + 10^−20*y) / 2 = (y + 10^−20*y^2) / 2 which has the solution: y(x) = 10^20 * e^(10^20*c1 + x/2) / (1 − e^(10^20*c1 + x/2)) or, plugging in the point x = 0, y = 1 and rounding a bit: y(x) = 10^20 * e^(x/2 − 46) / (1 − e^(x/2 − 46)) and taking the log base 10 to get number of orders of magnitude: y(x) = (x + 2*ln(10^20 / (e^46 − e^(x/2)))) / ln(100) This model is still hyperbolic (goes to infinity after 92 years), as it again ignores physical constraints. However, the increase is more gradual. The times to get each new order of magnitude of capability (after decades of a steady exponential run-up) are: 4.5873 years 4.4434 years 3.4095 years 1.1957 years 2.047 months 6.57 days 15.77 hours (etc.) Hence, there is still ultimately an explosion, but it does not start all at once; there is no sudden “mode shift” from a slow regime to a fast regime.