RE:THOUGHT

Climate, politics, campaign finance, media criticism

Politi-Fallacy

with 4 comments

Quick post, but something that has been bothering me:

Fact checkers like PolitiFact have been coming under fire from all corners recently, some of it very well deserved. But many conservatives have jumped on the fact that more statements made by Republicans or conservatives are rated “false” or “pants-on-fire” as opposed to statements by Democrats or liberals. Look!, they say, clear evidence that the fact checkers are biased!

Here are a few examples:

“In July you printed a chart with two years of PolitiFact Ohio results. It showed Democrats with 42 ratings of Mostly False, False or Pants on Fire, while the Republicans had a total of 88 in those categories. Doesn’t that prove you guys are biased?”

Cleveland Plain Dealer reader

And, slightly more sophisticatedly:

“A data-driven analysis of PolitiFact Florida’s 554 rulings on statements made by individuals appears to show a clear bias against Republicans and in favor of Democrats. As the truthfulness of a statement increases, so does the percentage of Democratic claims included in PolitiFact Florida’s rating.

… This dynamic appears to be a textbook example of what statisticians call ‘selection bias.’”

(This is followed by two cherry picked examples of how and whether PolitiFact chose to review certain statements.)

— Sean Davis, Red State

Here’s the thing these two posts neglect to mention: not reporting an exact 50/50 split is only evidence of bias if the split is in reality 50/50.

Without evidence that both conservatives and liberals actually do lie at the same rate, this is a little bit like noticing that your professor is consistently grading your friend’s essays higher than yours and saying this proves he’s obviously biased against you – without showing that your essays are actually as good or better.

Lacking any sort of comprehensive evidence, it’s important to note the argument here rests in whole on an assumption that both parties are equally bad. This is something of a religious belief for inside-the-beltway media types, but it’s still simply an assumption.

It’s easy to see that this journalistic norm of “evenhandedness” is ripe for exploitation by a group that wants to lie more and get away with it. Especially when a Romney pollster point blank stated “We’re not going to let our campaign be dictated by fact checkers.” It also fits with the post-truth plan of trying to discredit all neutral arbiters of truth – if they agree with me, they’re right, otherwise they’re just biased.

My point here is simple: fact checkers should reflect reality, even if that reality is not balanced in a tidy 50/50 split of falsehoods by each major party. After all, as Bill Kovach and Tom Rosenstiel write in The Elements of Journalism, “Journalism’s first obligation is to the truth.”

It certainly would be nice for both major parties to be equally truthful and most people are inclined to believe they are.

When the truth isn’t what people are predisposed to believe that it’s all the more important for journalists to stick by their guns.

 

 
Edit: This post originally stated that the first example was written by Ted Diatun, who is actually the Cleveland Plain Dealer’s reader representative and was reproducing a comment written by reader.

Advertisements

Written by rethoughtblog

September 9, 2012 at 10:52 pm

4 Responses

Subscribe to comments with RSS.

  1. Your basic point is correct, that the raw numbers for various ratings tend to tell us little about the level of bias without some sort of baseline. But that cuts two ways, since stats compiled by journalists who fail to randomize their samples tell us nothing about who’s lying more–other than within their chosen sample, of course, and assuming that the ratings are accurate.

    As a variety of folks have written, it is irresponsible for journalists to present their work framed as though it tells us something about the truthfulness of either individuals or parties. That was the thrust of Eric Ostermeier’s widely misreported study at the University of Minnesota.

    I’d suggest an exception to your generally accurate point, however. What if there was one truth rating that differed from another on entirely subjective terms? Wouldn’t the difference between that rating and the other from which it differs subjectively actually tell us something about the bias of the persons doing the ratings?

    btw, Ted Diadiun is the public editor for the Cleveland Plain Dealer (PolitiFact Ohio). He was quoting a reader; you appear to attribute the reader’s comment to Diadiun.

    As for Sean Davis, his case is perhaps incomplete but probably correct. It’s just not that hard to find false statements by Florida Democrats. PolitiFact Florida isn’t that interested in them. The numbers are so imbalanced that instant skepticism is warranted.

    Bryan

    September 10, 2012 at 1:48 am

    • Hi Bryan,

      Thanks for your comment. That’s one thing that I had hoped to get into but decided not to for length: it just inherently very hard to get a scientifically valid, comprehensive analysis of statements to establish as a baseline of truthfulness.

      Interestingly, the conventions may have offered a pretty good opportunity since both are of similar length, with a similar number of speakers that are chosen by the parties themselves, rather than journalists. I’m not sure if anyone has done an analysis of the conventions as a whole, though.

      I’m not sure I understand what you mean about truth ratings and subjective terms. Are you talking about comparing two different fact checking groups’ approaches to subjectively ranking the same statements?

      And good point about Sean Davis’s case. I’m not too well versed with Florida media myself, so it could very well be true. My main point is that we should initially be open to both the possibility that a) the fact checking outlet is biased and b) that Republicans in Florida on average make more not-so-truthful statements.

      -Kurt Walters

      P.S. do you have a link to that Ostermeier study?

      rethoughtblog

      September 10, 2012 at 5:48 pm

      • Ostermeier’s study:
        http://blog.lib.umn.edu/cspg/smartpolitics/2011/02/selection_bias_politifact_rate.php

        With my mention of a subjective difference between two rankings I’m talking about PolitiFact in particular. PolitiFact distinguishes between “False” and “Pants on Fire” by stating that the latter (by their definition) is “ridiculous” in addition to simply false. It’s hard to find any objective criterion in their application of the rating. On the other and it’s easy to find imbalanced application of the ratings by the national PolitiFact operation (biased against Republicans). And the opposite holds true of PolitiFact Wisconsin, which is (by my count) the state operation receiving the most criticism from the left.

        I’m seeing some interesting things in applying this approach.
        http://www.politifactbias.com/2012/08/pfb-research-yes-virginia-politifact-is.html

        Bryan White

        September 11, 2012 at 3:20 am

  2. “When the truth isn’t what people are predisposed to believe that it’s all the more important for journalists to stick by their guns.”

    Does this make sense when we consider how bias inevitably creeps into our work?

    This is my pet issue: epistemology and the role of information in society. Fact checking is the mass media attempt to assert ownership over a portion of the flow of “quality” information. I just don’t think journalists are particularly trustworthy in that role. It isn’t because they wish to mislead,necessarily. More likely they just can’t separate their bias from their work and judge poorly their degree of expertise in separating truth from error.

    It’s my contention that “He said/She said” journalism has as its roots in journalists’ realization that they were not experts. Many of today’s journalists have shed much of that health humility. It’s true to some extent that journalists have increased in education. But not *that* much. 😉

    Bryan White

    September 11, 2012 at 3:41 am


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: