(Bloomberg Opinion) -- The U.K.’s Conservative Party temporarily changed the name of its campaign Twitter account to “factcheckUK” during party leader Boris Johnson’s debate with Labour’s Jeremy Corbyn on Tuesday. That was pretty rich. Johnson, after all, was fired from his first job, at The Times of London, for fabricating a quotation. But then, the use of a fact-checker’s mantle as a political disguise is in some ways justified by actual fact-checking practice.
The Tories changed the account name for the explicitly political purpose of “fact-checking Labour.” They ran tweets such as, “False Corbyn claim on jobs — just last week, was confirmed number of people in FULL-TIME jobs at RECORD HIGH.” They declared Johnson the winner of the debate even though polls called it a draw.
It’s clear where the Tory public relations people were coming from when they pulled their stunt: In 2017, a U.S. study found that a running fact-check of a political debate makes voters more likely to support the candidate whose statements are revealed as more accurate, and vice versa.
The trick, however, was widely criticized, including by Twitter, which threatened “corrective action” for any further attempts to mislead the public in this way. And indeed, it was an insult to the worldwide fact-checking movement that has sprung up over the last two decades: It reduced professional fact-checking to partisan punditry. But it also exposed some of the weaknesses behind even the best-intentioned efforts to separate truth from falsehood on the contentious political stage.
Back in 2013, Joseph Uscinski, a political scientist at the University of Miami, and his collaborator Ryden Butler, wrote a controversial article attacking the fact-checking movement for its lack of consistent methodology or clear criteria for establishing which statements by which politicians to check. They argued that fact-checkers often attempt to rule on the truthfulness of statements about the future (which aren’t factual by definition) or about the causality of social phenomena (which they aren’t equipped to determine). They wrote:
The subject matter of politics is often complex, ambiguous, and open to a variety of conflicting interpretations, even when empirical claims are being made. Therefore, people may genuinely disagree about the truth. The fact that a politician disagrees with a fact-checker about the facts does not make the politician a liar any more than it makes the fact-checker a liar.
The paper gave rise to a lively debate. Michelle Amazeen of Boston University wrote a rejoinder, arguing that different fact-checking outfits often arrive at the same verdicts on politicians’ statements, which should mean they aren’t exactly flailing in the dark. Ucsinski doubled down on his arguments in a counter-rejoinder, calling fact-checking “naive” and too eager to analyze unverifiable statements.
Continued fact-checking research lent support to Uscinski’s point of view. In 2018, Chloe Lim, who is working on a Ph.D. thesis about fact-checking at Stanford University, analyzed how PolitiFact and the Washington Post’s Fact Checker, the two major U.S. outlets, performed on statements made by candidates in the 2016 presidential campaign. She found that PolitiFact made 1,178 checks of such statements and Fact Checker confined itself to 325; only 77 fact-checks by the two outlets overlapped. This was consistent with Uscinski’s point about haphazard and potentially biased selection.
Lim also found that if one uses the fact-checkers’ own assessment scales instead of trying to convert them to simpler ones, as Amazeen and other researchers have done, there’s more disagreement between the outlets than previously believed. Lim wrote:
My results show that while fact-checkers perform fairly well on outright falsehoods or obvious truths, the agreement rate is much lower for statements in the more ambiguous scoring range (i.e., “Half True” or “Mostly False”). Lack of consensus among fact-checkers may arise from the challenge of verifying the accuracy of political claims. Politicians tend to be quite vague, which makes it difficult for fact-checkers to evaluate their claims in a clear and objective manner.
That fuzzy area is where fact-checkers disagree — for example, based on the same factual premises, Fact Checker judged ex-Florida Governor Jeb Bush’s statement that his state “led the nation in job creation” mostly false and PolitiFact called it half-true. That kind of effort to impose scientific objectivity on political debate and legitimate partisanship is part of what undermines trust in the independence and impartiality of fact-checkers. Recent U.S.-based research shows that people are dubious about the outlets’ evenhandedness.
It also makes professional fact-checkers easy to parody, as the U.K. Conservatives did. One of the tweets the Conservatives put out in their guise as fact-checkers stated as FACT (in all caps) that under Labour’s plans, health-care workers would end up working 2.5 million unpaid hours a week — a claim about the future that is simply impossible to verify.
None of this is to say that fact-checking deserves to be scrapped. In a 2019 meta-analysis of scholarship on the subject, Nathan Walter, a communications scholar at Northwestern University, pointed out that fact-checkers play a useful role in signaling that politicians will be held accountable for lying, and thus make them more cautious (though U.S. President Donald Trump is a powerful counterexample). In any case, setting the record straight is important even when people remain biased toward their preferred parties and candidates, and it helps make sure that fake news doesn’t crowd out honest reporting.
But the U.K. incident should be a wake-up call to the fact-checking industry. Walter’s meta-analysis shows that the more nuance a fact-check requires, and the more detailed the scale on which “facts” are judged, the less effect the practice has on people’s political beliefs. The power of fact-checkers is greatest when they dissect an outright lie or confirm a verifiable truth. Besides, the effect fact-checks have on beliefs is the lowest when campaign statements are analyzed: These are often more about rallying supporters than arguing policies, and voters tend to see them that way.
This calls for a universal methodology for selecting comments to check, and for keeping things as simple as possible. Leaving the nuance to analysts who don’t call themselves fact-checkers is probably the best approach. Otherwise, the impulse to define fuzzy claims as verifiable facts just opens the door to self-serving gimmicks like the Tories’ disingenuous “fact-check” of the leaders’ debate.
To contact the editor responsible for this story: Jonathan Landman at jlandman4@bloomberg.net
This column does not necessarily reflect the opinion of the editorial board or Bloomberg LP and its owners.
Leonid Bershidsky is Bloomberg Opinion's Europe columnist. He was the founding editor of the Russian business daily Vedomosti and founded the opinion website Slon.ru.
©2019 Bloomberg L.P.