Why San Francisco housing data lies
Corey Smith ran the San Francisco Housing Action Coalition, a YIMBY-aligned nonprofit pushing for more housing construction in a city where the median home price had passed $1.1 million. He had asked Alyssa for help sourcing data on housing affordability for SFHAC’s research. The note below explains where the relevant Census microdata lives (IPUMS), and then makes a more uncomfortable point: even with the best dataset, the headline numbers will mislead you, because in San Francisco the price someone actually pays for housing is decoupled from the housing itself by Prop 13, rent control, mortgage type, and the byzantine rules around below-market-rate units.
For #1 and #3, it looks like the data you want is from the Census Bureau, via a research portal called IPUMS . To get the data, you have to register, explain who you are and explain what you’re doing with it, but this process looks relatively painless. I’d just do it myself, but you’d probably look more credible as an employee of an established nonprofit.
It’s very important to note that how affordable a house is is not an inherent property of the house, like the house’s location or construction date or square footage. The exact same house can cost wildly different amounts, depending on circumstances. There are five main categories: People who own their house free and clear. They pay nothing except property taxes (and frankly, if they’re in San Francisco, they’re probably millionaires now). However, even here it gets complicated — because of Prop 13, the property taxes paid for identical houses might vary by a factor of 5 or more. If a new household moves in, their property taxes for the same house might jump anywhere from 10% to 500%. I think the IPUMS data might include some info on property taxes, but the variable descriptions are unclear.
People who own their house, but pay a mortgage. IPUMS includes data on mortgage payments. However, I don’t think they include the type of mortgage, which is the factor most relevant to affordability. A fixed-rate, fixed-payment mortgage actually gets easier to pay every year (adjusted for inflation), but there’s also adjustable-rate mortgages, balloon mortgages, and other exotic animals that can make people’s payments suddenly jump.
People who rent an apartment, but the apartment is rent-controlled. IPUMS should include data on what they’re currently paying in rent. In normal circumstances, this number will always go down every year, adjusted for inflation.
However , if the household gets evicted (or leaves voluntarily), the rent they’re paying may jump, anywhere from 10% to 500%. Similarly, the rent for the new household who moves into the unit will also be higher, anywhere from 10% to 500%.
People who rent an apartment, and the apartment is market-rate. In theory, this should be the simplest case: the rent they pay (in IPUMS) should be the same as the market rent. But, even for large market-rate buildings, survey data indicates that the average tenant pays significantly less than a new tenant would, implying that some landlords do a sort of “voluntary rent control” where they don’t hike prices for existing tenants too rapidly.
People who rent a below-market-rate, income-restricted apartment. Generally, these are categorized by income requirements at time of move-in : eg., between 80% of AMI and 120% of AMI.
But , these income requirements might or might not resemble how much people in the building actually make. According to the MOH manual, residents are allowed to stay in a BMR apartment as long as their income doesn’t rise above 200% of the maximum limit at time of move-in, which is 240% of AMI for a building listed as 120% of AMI. I’m not sure anyone collects data on how much residents of these apartments actually make; it almost certainly isn’t in IPUMS because the sample size is tiny, and the MOH might not keep track, or might not release the info due to privacy concerns.
Some made-up examples of how all this can create misleading data: Alice is a public school teacher with a modest income. She bought her house when she moved to SF to start teaching in 1985, and the mortgage is paid off. Alice’s house is now worth two million dollars. On paper, it looks like Alice’s house is very unaffordable, but Alice doesn’t mind. In fact, she’s thrilled. Prop 13 caps her property taxes, and she can cash out by selling the house when she retires and buying a huge mansion in Florida.
Bob makes 60% of AMI as a restaurant chef. His rent is affordable, since he was lucky enough to get a BMR apartment. However, Bob has been taking some classes at night school, and the next month, Bob gets a great job as an electrical engineer, making 120% of AMI. Unfortunately, this is above the income limit for Bob’s BMR apartment, and 120% of AMI isn’t enough to afford a new market-rate apartment in SF. Bob and his family are forced to move to Richmond. This nasty scenario is known as a “ welfare cliff ”.
Carol is a billionaire, who made her fortune in the dot-com boom of 1999. She still lives in her old, cheap apartment, which is rent-controlled and costs $1,200 a month. On paper, a rent this low would be affordable to a household making (roughly) 65% of AMI. However, this is totally meaningless, because the apartment isn’t occupied by a working-class person; it’s occupied by Carol, who is filthy rich and doesn’t care how much the rent is. If Carol moves out, that doesn’t help either, because the owner can now reset the rent to a market rate of $5,500 a month.
Dave is a computer programmer, who lives in a rent-controlled apartment for $1,500 a month. Dave’s building has two units in it. However, Dave’s sneaky landlord did some construction work to remove the other unit, and since Dave now lives in a single-family house, he is excluded from rent control under the Costa-Hawkins Act. Dave’s landlord raises the rent to $7,000 a month.