Is National Review “junk news”? A panel of Oxford scientists says yes. Their study, “Polarization, Partisanship and Junk News Consumption over Social Media in the US,” purports to show that on social media, conservatives are far more likely than others to share “junk news.” That conclusion has earned them glowing write-ups in left-of-center outlets like The Guardian, Salon, and The Daily Beast.

And what’s junk news? According to the study, a source is junk if it “deliberately publishes misleading, deceptive or incorrect information purporting to be real news about politics, economics or culture.”

The cry of “fake news” has become a defining weapon in our hyper-partisan age. No doubt there has been a profusion of it as the nation’s storied newsmagazines have ceded prominence to newsfeed-optimized content factories. But “fake news” is also an accusation of moral turpitude. Its scarlet letter signifies that a journalist is not merely wrong—he’s a liar.

It therefore matters quite a bit who bears the scarlet letter, since supposedly these are the people poisoning our body politic, and we’re all better off without purveyors and sharers of fake news. The incentives to apply the label with the imprimatur of “science” to an entire opposing segment of the political spectrum are thus very high.

Did Oxford fall prey to those incentives? To arrive at their finding that conservatives are more likely to share junk news, the study’s authors developed a set of criteria that websites must meet in order to be “junk,” and then examined what kinds of people shared links from those junk sites. To be junk, a website had to exhibit at least three of the following five characteristics:

• Professionalism: sites that “do not employ the standards and best practices of professional journalism.”
• Style: sites that “use emotionally driven language.”
• Credibility: sites that “rely on false information and conspiracy theories” and whose “standards of news production lack credibility.”
• Bias: sites whose reporting “is highly biased and ideologically skewed.”
• Counterfeit: sites that “mimic professional news media” by “counterfeiting fonts, branding and stylistic content strategies.”

As worded, that standard feels easy to abuse, but what results did it produce? The scientists’ criteria yielded 91 sites positively identified as junk news. Of these, 78 featured right-leaning content, 10 were apolitical, and a grand total of three—Mediaite, Occupy Democrats, and Shareblue—were left-leaning.

Maybe junk news sites mostly don’t exist on the left? The fact that the authors named three left-of-center sites could indicate that they scoured the internet and found only a few offenders. More likely is that it shows the opposite—if a relatively mainstream liberal site like Mediaite was caught in Oxford’s net, it seems implausible that outlets like Alternet, Counterpunch, Salon, Daily Kos, Truthout, and Democracy Now! slipped through.

Just last month, The Huffington Post, a beloved and enormously influential outlet on the left, had to revamp their editorial policy in recognition that it was producing unverified, ideologically charged content that occasionally crossed into outright falsehood. The problem was so bad that HuffPo’s own announcement of the policy change referenced a “tsunami of false information.” Their editors were effectively admitting, with admirable candor, that they should have been included on Oxford’s “fake news” list.

Maybe these sites were able to escape scrutiny because the “criteria” the Oxford researchers used to tell if a site is real or fake are not criteria at all. They’re flexible enough to produce any result deemed desirable in advance. A determination that a site’s reporting is “biased,” “uses emotionally driven language,” and fails to employ journalistic “best practices” earns it the junk label. The average Republican voter probably thinks that’s a fair description of the New York Times, let alone the Huffington Post.

For a prime example of a biased, emotionally charged story that deviates from journalistic best practices, we need only turn to the Guardian’s coverage of this very study. It came under the headline “Fake news sharing in US is a rightwing thing.” Most who read that will take it as license to make broad generalizations about the reading habits of conservatives. But that generalization would be dead wrong. A more rigorous study conducted by Dartmouth political scientists found that, while fake news sharing during the 2016 election was more common among Trump supporters, it was “heavily concentrated among a small subset of people”—60 percent of fake news consumption came from 10 percent of the population. The Guardian’s headline was an invitation to attribute this behavior to an entire half of the political spectrum. Under the Oxford criteria, that sounds like junk news.

However, we don’t even need to analyze the study this far, because one of its authors gave an alarming interview in which it became clear that even these minimal criteria were not fairly applied. When asked what ensnared National Review, professor Philip Howard said “I think they lost points on commentary masking as news.” If that’s the test, storied publications like The Atlantic and The Economist have been failing it for over a century.

And what about Mediaite, one of the three left-of-center sites included on Howard’s list? “That one was probably scooped up because the far-right uses links to those stories as if they themselves are news items,” said Howard. Now we’ve entered the Twilight Zone—that isn’t even one of the criteria the study claimed to use. But perhaps it’s a disarmingly honest reveal of the study’s heuristic: what sites are our enemies reading?

Eric Wemple, the Washington Post columnist who conducted the interview, concluded that Howard’s statements raise the possibility that the study “merely caught conservatives sharing conservative journalism.” It looks like he’s exactly right.

Studies like this one are dangerous because they needlessly polarize the academy, enlisting it in a political advocacy project that alienates large sections of the population. A recent Pew survey found that a majority of Republicans now say that colleges and universities have a negative effect on the country. In 2015, just 37 percent of Republicans rated the effect of universities negatively; in 2017, that shot up to 58 percent.

The academy cannot afford to see this number rise any more if it wants to retain broad public authority. Professors and students may wish to turn our institutions of higher learning into progressive fortresses, but the result will be conservatives who build battering rams. Many academics may see themselves as part of The Resistance, but if that mission bleeds into the methodology of their scholarship, their legitimacy will be permanently damaged.

Nicholas Phillips is a research associate at Heterodox Academy and president of the NYU School of Law Federalist Society. Follow him on Twitter at @czar_nicholas_