When it comes to America’s high incarceration rate—now about five times what it was in 1970—there’s the Standard Story, and then there’s the truth.

The Standard Story is the one that has been propagated for years in mainstream-media outlets and by activists. It holds that the War on Drugs is virtually the sole culprit—that incarceration rose merely because America decided to start imprisoning nonviolent, low-level drug offenders for absurd amounts of time. It posits the simple solution of reducing or eliminating the sentences for these victimless crimes.

The truth, by contrast, is that about half of prisoners were convicted of violent offenses, and that some of the others committed violence but pleaded guilty to lesser offenses. Even the fifth of prisoners who are locked up for drugs tend to be mid-level dealers, not users or low-level distributors. And, while decades-long sentences make the news, most prisoners who committed crimes not involving the most serious violence are out within a year or two. In other words, while incarceration has undoubtedly soared—even relative to crime, which has dropped substantially since the early 1990s—our propensity to throw people in prison has simply not reached the heights of ridiculousness that many assume.

There is still “low-hanging fruit” to be had by releasing some drug offenders or subtly redefining crimes (such as changing the dollar-value threshold separating misdemeanor from felony theft), but this will not get America anywhere near the incarceration rate it had decades ago or the rates that prevail elsewhere in the developed world. Bigger reductions would require speeding the release—or declining to imprison—people who committed crimes that left very real victims, which is not so obviously a desirable outcome.

Until recently, few were discussing this reality aside from a handful of conservative commentators such as the Manhattan Institute’s Heather Mac Donald. These people typically argued that those in prison mostly deserve to be there, and that dramatic reductions to the incarceration rate run an intolerably high risk of increasing crime. But in the last several years a number of reform-minded scholars and pundits have tried to make a public case for such reductions even in full view of the facts.

The latest entry in this literature is John Pfaff’s Locked In: The True Causes of Mass Incarceration and How to Achieve Real Reform. It is an excellent overview of where America stands in regard to its prisons, and Pfaff’s proposed reforms deserve serious consideration across the political spectrum.

The role of the drug war isn’t the only issue on which Pfaff departs from the Standard Story. He also disputes the idea that the typical prisoner is spending much more time behind bars than he used to. In Pfaff’s view, the reason for our skyrocketing incarceration rate is that prosecutors have become more likely to file felony charges following an arrest, rather than that those convicted are being locked up for longer periods of time.

This is considerably more contentious among those who study imprisonment; unlike the percentage of prisoners serving time for drugs, it’s not something one can simply look up in a Justice Department report. Pfaff is at odds with the prestigious National Academy of Sciences, for instance, when he all but dismisses the role of time served. The debate involves competing data sources and intricate mathematical simulations.

But if prosecutors might not be the sole driver of mass incarceration, no one denies that they are a big one. And Pfaff expertly lays out how this happened so that we can see if it’s a process we can live with.

As is well-known, the crime explosion of the late 1960s through the early ’90s inspired lawmakers to adopt a get-tough approach, and this entailed reining in judges, for instance through mandatory-minimum laws. The concept is not inherently flawed: there are unique factors at play in each case, but in general, people who commit the same crime should receive similar punishments. The punishment should depend on the law, not the judge’s personal sense of justice or his like or dislike of the defendant. But there were two problems with these laws as they actually played out.

First, especially at the federal level, many minimums are so high that no one really thinks they’re fair and people are rarely sentenced to them. Instead, prosecutors use them as a threat to get defendants to plead guilty to lesser charges or testify against fellow criminals. (About 95 percent of cases end in plea deals rather than trials today.) In other words, they operate as a roundabout way to gut defendants’ constitutional rights: if you make the prosecution prove its case at trial and invoke your right to remain silent about criminal activities you participated in, you receive a patently unfair sentence. Incredibly, the federal prosecutors’ lobby has defended the current mandatory minimums explicitly on these grounds.

Second, and relatedly, the minimums didn’t eliminate discretion from the system: prosecutors still have plenty. A prosecutor often can decide how much time a defendant should serve and then put together a mix of charges that will require the judge to give a sentence in that ballpark. The law can offer an impressive buffet of overlapping statutes that cover the conduct a defendant is accused of.

And in addition to holding enormous discretion, prosecutors face a number of incentives that are far from ideal. District attorneys are typically elected and want to avoid going easy on a Willie Horton or a Brock Turner. The elections are county-wide, giving conservative suburban areas a lot of say as to how high crime in inner cities is handled, even though suburbanites bear little of the cost of crime or of incarceration. Prosecutors also face little resistance, because judges normally accept plea deals and most defendants rely on public defenders, which are underfunded. In 43 states defendants have to pay at least some of the costs associated with their “state-provided” lawyer.

Moreover, what we call “the justice system” is really a haphazard mashup of city, county, state, and federal agencies. Federal prisons receive a lot of attention but hold just 13 percent of prisoners. In the states, meanwhile, counties generally pay for probation and short jail stays while the state pays for the cost of imprisonment—creating an incentive for prosecutors to overuse the latter.

Reading Pfaff’s characterization of who’s serving how much time in prison these days—mainly violent offenders, mainly short sentences—one is tempted to wonder what the big deal is. Maybe we should just content ourselves with picking the “low-hanging fruit.” Yet it’s hard to accept our sky-high incarceration rate knowing it’s produced by the dysfunctional system Pfaff describes. Realigning the incentives in that system would be a worthwhile endeavor whether it cut incarceration or not.

Conservatives intuitively understood the need to rein in judges’ discretion decades ago; perhaps the same thing could be done for prosecutors today. Pfaff notes that New Jersey has given its prosecutors detailed guidelines as to the plea deals they are allowed to strike, with judges able to invalidate any deals that break the rules; they “look almost exactly like the guidelines that many states use to regulate judicial sentencing.” This is a promising idea, though the guidelines would have to be written carefully to avoid unintended results. (The New Jersey guidelines initially made it hard for urban prosecutors to give lighter sentences in “school zone” cases, for example, which was a problem because 76 percent of Newark is considered a school zone.)

Other options: cut mandatory sentences to reduce the threats prosecutors can make to extract plea deals; require prosecutors to disclose the threats they made so that judges can review them; balance out the incentives facing county prosecutors by paying counties to keep people out of state prison; appoint prosecutors instead of electing them; let cities and suburbs choose their prosecutors separately; fund public defenders adequately. Each of these moves would align incentives in a sensible way rather than seeking to cut incarceration per se.

And on a deeper level Pfaff prompts us to consider more carefully the exact tradeoffs we’re willing to make between incarceration and crime. One study, for instance, found that between 1978 and 1990, locking up an extra person for a year stopped 2.5 violent crimes and 11.4 property crimes. Thanks to diminishing marginal returns, those numbers fell to 0.3 and 2.7 respectively in the 1991–2004 period. Are the latter numbers worthwhile given the cost to taxpayers, and to offenders and their families? Is the payoff even lower today? And what if, for a given amount of money, you could reduce crime 20 percent more by hiring more cops than by incarcerating more offenders, as a different study contended?

For these reasons, Pfaff suggests we reject the assumption that reforms are worthwhile only if they don’t increase crime at all. It’s a point worth taking to heart as one considers some of Pfaff’s other reforms, the ones more directly targeted at reducing incarceration.

Risk-assessment tools are one promising development. Modern statistics allow us to calculate the chances that a given prisoner will reoffend with a reasonable degree of accuracy, based on various characteristics. There are legitimate complaints about these tools (though Pfaff takes too seriously an allegation of racial bias by the journalism outfit ProPublica), but they hold out the promise of focusing incarceration on the people who really need to be locked up lest they continue to offend. They are a dramatic improvement over the older, cruder tools like “three strikes” laws.

In a somewhat similar vein, pilot programs could experiment with releasing offenders and closely monitoring them, like the Hawaii HOPE program does for drug offenders, giving them repeated drug tests and a “swift, certain, and fair” jail stay for minor lapses.

Not all of the ideas Pfaff explores are home runs; I have trouble imagining an American state in which there’s a “cap-and-trade” system for prison capacity. But in general, these are far more serious and considered proposals for cutting incarceration than what we have seen from almost anyone else.

Pfaff’s book is targeted primarily at reformers, not skeptics. He believes the reformers misunderstand the problem and hence cannot solve it. He notes, for example, that many efforts to cut sentences for low-level offenders are coupled with increased sentences for those who commit worse crimes—which would address the problem described in the Standard Story but not the reality we actually face.

And in debunking the myth of nonviolent drug offenders haphazardly locked away for long periods of time, of course, he runs the risk of inadvertently convincing his audience there really isn’t much of a problem. He’s to be commended for taking that risk.

But, by forthrightly explaining the true nature of incarceration in America before laying out his case for reform, Pfaff poses a serious challenge for the skeptics, too. Unlike so many activists and op-ed writers, Pfaff cannot be waved away with a handful of simple statistics demonstrating that, no, our high incarceration rate isn’t the result of locking up first-time offenders caught smoking pot. He knows that, and still sees serious problems with the status quo. His ideas deserve a close look.