The New York Times’s Statistical Contraception Failure
The New York Times published a deeply misleading infographic on contraceptive failure rates this weekend.
The NYT graphic plots contraceptive failure rates for 15 different methods of contraception over 10 years, apparently showing that, although a typical woman relying on condoms for contraception may only have an 18 percent chance of pregnancy over one year, an unplanned pregnancy becomes a virtual certainty by the time she’s been sexually active for 10 years, with the chance of pregnancy nearly quintupling to 86 percent.
Below those red-alert curves, the NYT also tracked (along the dashed grey lines) the risk of pregnancy for atypical women and their partners, those who manage “perfect use” of their preferred form of contraception.
The NYT generated these graphs by looking only at failure rates for contraception over one year, and then doing some arithmetic to model the chance of failure over the long term. It’s the same math used to figure out how often you might need to flip a fair coin to eventually wind up with “heads” but it doesn’t work so well with this kind of data.
The chance getting “heads” on the first flip is 50-50. If you want to know what the chance is that you’ll get a heads within two flips, or three, the easiest way to compute the answer is to check the odds that you’d flip tails every time, and then subtract from 100 percent. So, the odds of getting two tails in a row is .5 multiplied by .5, or .5^2. This results in a 25 percent chance of only tails, and a 75 percent chance of getting at least one “Heads” within two flips. In three trials, the chance of triple tails is .5^3, i.e. a 12.5 percent chance of getting no heads at all, and, thus, an 87.5 percent chance of getting heads at least once.
This is the math that produces the smoothly rising curves on display at the NYT. It’s the right kind of statistics to use if you’re talking about some fixed risk, like the chance of winning a coin flip. But the chance of unintended pregnancy isn’t likely to remain constant over a 10-year period.
For one thing, the women who get pregnant in their first year of using a particular form of contraception may be different than the rest of the women in their cohort. A significant proportion of the women who wind up pregnant may be more fertile than average (or are having sex with a guy with excellent sperm). Irregular cycles may make it harder for them to use NFP charting successfully. They may work schedules that make it much harder to stick to an “every day, same time” form of contraception like the Pill. The may be college students having a lot of tipsy sex, and not remembering to manage condoms very well.
Once those women get pregnant in the first year, they’re out of the analysis, as far as the NYT is concerned. But the charts continue to use the risk probabilities set by the most unsuccessful group of contraceptors as the baseline risk for everyone who remains. There’s no room in the model for selection effects.
But even if we ignore this confounder, and pretend that women who get pregnant in their first year of using contraception are behaviorally and hormonally identical to those who don’t, it would still be a bad idea to extrapolate the numbers out over a 10 year period. Women in their first year of contraceptive use aren’t going to behave the same way as women who have been sexually active and contracepting for five or 10 years.
More of the women will be in exclusive partnerships in the last five years than were at the outset, and they may have settled into more of a contraceptive routine that is easier for them and their partner to stick to. They may also just be having a lot less sex, so the risk would drop accordingly. Additionally, due to declining fertility, women in their thirties experience a lower risk of contraception failure than women in their twenties.
To paint an accurate portrait of long-term contraceptive failure rates, it would make more sense to use actual five or 10 year failure rates, instead of these flawed models. Unfortunately, in many cases, that data does not exist. A Guttmacher analysis of the 1995 National Survey of Family Growth found that women tend to switch or discontinue methods of contraception frequently.
For example, 90 percent of women quit using sponges as contraception within two years of relying on them (and presumably not due to an Elaine Benes-like shortage). At that point, the NYT‘s estimate that the sponge has a 75 percent failure rate after five years starts to look irrelevant, as well as misleading.
Presenting flawed data is dangerous, since it can mislead readers and alter their behavior. For instance, a couple looking over the NYT graphs on Sunday might be more likely to play a bit more fast and loose with their contraceptive plans, since their work seemed doomed to fail anyway. Psychologists call this the “What the Hell” effect—it’s the same impulse that prompts us to eat four more cookies once you’ve already broken your diet by eating one.
So, if there’s a small baby boom in nine months, it may be the NYT number crunchers, not the contraceptives, that failed their users.