Thoughts on General Biotics
General Biotics was an early-stage startup proposing to develop livestock probiotics — engineered microbial colonies fed to farm animals to alter their growth, health, or behavior. Anna Salamon (co-founder of the Center for Applied Rationality) had asked Alyssa for an evaluation, citing a then-recent case study by Alang and Kelly in which a fecal microbiota transplant (FMT) appeared to transfer obesity from a teenage donor to a previously normal-weight recipient. The reply pushes back on the inferential leap from one suggestive case study to a working commercial pipeline.
The thoughts below aren’t super confident, since they don’t come out of detailed research, but it’s worth writing up the general picture so far.
Back in the day, I used to dabble in hobbyist electronics. If you start doing electronics, pretty much the first thing that happens is: you design a nice circuit, you put all the transistors and resistors in place, you flip the power switch, it doesn’t work, and you have no idea why. You come up with a theory for why it doesn’t work. Your first theory is wrong. And so is the second. And the third. And the tenth. And after an hour or two of banging your head against the wall and poking at it with a multimeter, you finally discover the problem, probably something it had never occurred to your System 1 to think about. And then it still doesn’t work, so you start all over again…
( This article gives some of the feel of it. In particular this part: “Make no assumptions!!! You may have heard of soldiers who were commanded to ‘take no enemies,’ well this is the troubleshooting equivalent. When we get stuck and cannot proceed, this is virtually always the reason. When you reach this point, you must recheck your work and that is where incorrect assumptions preclude progress. One simply cannot say, ‘I checked that detail yesterday so that cannot be the problem.’ That could be a bad assumption – RECHECK AGAIN! Remember that humans make mistakes, but electrons never make mistakes…”)
A modern computer is a carefully controlled environment — a ten-line Python program isn’t going to fail because the motherboard had DDR3-1333 RAM instead of DDR3-1066 RAM. But physical stuff isn’t; you can’t do “write once, run everywhere”, unless you carefully account for every factor that might differ between situations. Even lower-level programming is like this, actually; eg. OpenCL requires you to know a lot about the hardware you’re designing for, because GPGPU is too new for user-friendly interface layers. So you get problems like what Scott describes in Social Psychology Is a Flamethrower : “Social psychology experiments in the laboratory tend to throw up spectacular mind-boggling effects. Many of these fail to replicate and are later discredited. The ones that do replicate are not always generalizable – sometimes an even slightly different situation will remove the effect or create exactly the opposite effect. The effects that remain robust in the laboratory may be too short-lasting or too specific to have any importance in real life. And the ones that do matter in real life may respond unpredictably or even paradoxically to attempts to control them.”
(My personal favorite example of this kind of thing is a published case study that showed a trans woman had taken some random endocrine drug, and miraculously acquired female secondary sex traits. This was brought up on a forum, and after a little while the study subject came by personally to explain that the data were entirely fabricated.)
So I’m sure there’s a ton of interesting stuff going on in the microbiome, but things like the Alang & Kelly case study are just really, really far away from the level of detail and precision and understanding you need for the commercial application stage. Eg. just picking the most obvious thing in that particular case study, at the time of FMT, both the donor and donee had a BMI of 26. Afterwards, the donee’s BMI increased to 34.5, but the original donor’s BMI had only increased to about 30, and nobody knew what caused the difference. And even the increase to 30 isn’t clear, because the donor was a teenager whose weight gain could have been partly from height growth. And the FMT was done rectally via colonoscope, not orally. And the donor was her daughter, which would give you lots of shared genetics and environment and other stuff you wouldn’t have in a commercial setting. And the donee had previously had both a C. difficile infection and an H. pylori infection, and was treated for both with vancomycin and several other antibiotics, all of which had their own effects (as the authors noted), and…
It’s the same thing when thinking about artificial microbiome selection. In (eg.) race horses, we have a pretty detailed understanding of how inheritance works: we can get exact genetic sequences of both parents, and from that we can predict fairly precisely how likely it is that a child will have any particular gene, and we’ve even gotten pretty good at predicting complex phenotypic traits like height, and all of that is backed up by hundreds of years of experience, and this lets us create super horses that gallop at 17 meters a second. But one can’t simply assume that everything in the race horse world (or whatever example one picks) automatically transfers over to microbial colonies. If you breed two horses, and wait one generation, you know pretty exactly what’s going to happen, at least as a statistical distribution. If you get a colony of a thousand microbial species, put it in the lab, and wait one generation, what will happen is pretty near impossible to model. Some species will grow, some will probably die entirely, a bunch of bacteria will mutate, different ones will mutate at different rates, many of them will swap DNA, but you don’t have any idea which species are sending DNA and which are receiving DNA and what each piece of DNA is doing, and so on ad infinitum. It’s a giant mess. Mammalian DNA is a very high-fidelity information storage mechanism, so if you make a change, it’s stable enough that you know it’ll be there in the morning. If there’s anything stable about a bacterial colony, I’d be surprised if anyone knew what it was.
If I were doing this myself, my first idea would be to go through the literature, find the clearest, most spectacular, most obvious, simplest, most well-understood, most well-studied experiment of feeding animals bacteria having any effect at all that one could cheaply replicate, and go out and replicate it. And then replicate it again, preferably with a different team, so you’re absolutely sure the effect is real. Once that is done, you could slowly alter the experiment parameters towards something commercially realistic, until you’re doing everything exactly identically to how it would be done commercially and still getting a significant effect. That way, when you miss a step — eg., this cryoprotectant works on species #1–89 but not species #147–290, you need to use a different one — you can backtrack to something you know works and try again. You aren’t stuck with a non-working system that could have gone wrong in any of a hundred different places.
Or, to take another approach, we already have tons of data on the effects of millions of different chemicals in humans. Some of these have significant evidence for weight gain, eg. the psychiatric drug clozapine. Since chemicals are so much simpler than bacteria, you could compile a list of eg. the thousand drugs most likely to cause weight gain in humans, get a bunch of farm animals bred and raised under commercial conditions (eg. homogeneous strictly-controlled food supply, so appetite effects are irrelevant), and throw molecules at them Edison-style until something happened to stick. This kind of approach has worked before in humans (eg. the military happened to notice that chemical weapons could counteract lymphoma), so it’s at least plausible that something could be found if you tried enough things.