A Weblog Dedicated to the Discussion of the Christian Faith and 21st Century Life

A Weblog Dedicated to the Discussion of the Christian Faith and 21st Century Life
I do not seek to understand that I may believe, but I believe in order to understand. For this also I believe, –that unless I believed, I should not understand.-- St. Anselm of Canterbury (1033-1109)

Tuesday, July 10, 2012

The Reasons for Poor Political Punditry

Selected portions from Sharon Begley at the Daily Beast:
Pointing out how often pundits' predictions are not only wrong but egregiously wrong—a 36,000 Dow! euphoric Iraqis welcoming American soldiers with flowers!—is like shooting fish in a barrel, except in this case the fish refuse to die. No matter how often they miss the mark, pundits just won't shut up, and I'll lay even odds that the pundits (and pollsters) who predicted a big defeat for Tzipi Livni in the Israeli elections last week didn't slink away in shame after her party outpolled all others. The fact that being chronically, 180-degrees wrong does not disqualify pundits is in large part the media's fault: cable news, talk radio and the blogosphere need all the punditry they can rustle up, track records be damned. But while we can't shut pundits up, we can identify those more likely to have an accurate crystal ball when it comes to forecasts from the effect of the stimulus bill to the likelihood of civil unrest in China. Knowing who's likely to be right comes down to something psychologists call cognitive style, and with that in mind Philip Tetlock, a research psychologist at Stanford University, would like to introduce you to foxes and hedgehogs.

At first, Tetlock's ongoing study of 82,361 predictions by 284 pundits (most but not all of them American) came up empty. He initially looked at whether accuracy was related to having a Ph.D., being an economist or political scientist rather than a blowhard journalist, having policy experience or access to classified information, or being a realist or neocon, liberal or conservative. The answers were no on all counts. The best predictor, in a backward sort of way, was fame: the more feted by the media, the worse a pundit's accuracy. And therein lay Tetlock's first clue. The media's preferred pundits are forceful, confident and decisive, not tentative and balanced. They are, in short, hedgehogs, not foxes.

That bestiary comes from the political philosopher Isaiah Berlin, who in 1953 argued that hedgehogs "know one big thing." They apply that one thing (for instance, that ethnicity and language are primal; ergo, any country that contains many ethnic groups will break up) everywhere, express supreme confidence in their forecasts, dismiss opposing views and are drawn to top-down arguments deduced from that Big Idea. Foxes, in contrast, "know many things," as Berlin put it. They consider competing views, make bottom-up inductive arguments from an array of facts and doubt the power of Big Ideas. "The hedgehog-fox dimension did what none of the other traits did," says Tetlock, who described the study in his 2005 book "Expert Political Judgment": "distinguish more accurate forecasters from less accurate ones" in both politics (will Iraq break up?) and economics (whither unemployment?).

In short, what experts think matters far less than how they think, or their cognitive style. At one extreme, hedgehogs seek certainty and closure, dismiss information that undercuts their preconceptions and embrace evidence that reinforces them, in what is called "belief defense and bolstering." At the other extreme, foxes are cognitively flexible, modest and open to self-criticism.

 Here's how to identify fauna: foxes pepper their speech and writing with "however" and "but," recognizing uncertainty in the face of competing forces. Hedgehogs suffer from no such doubts, which (combined with their adherence to a Big Idea) makes them especially prone to overpredict change.

The media, of course, eat this up. Bold, decisive assertions make better sound bites; bombast, swagger and certainty make for better TV. As a result, the marketplace of ideas does not punish poor punditry. Few of us even remember who got what wrong. We are instead impressed by credentials, affiliation, fame and even looks—traits that have no bearing on a pundit's accuracy.

The truly bad news for forecasters, however, is that although foxes beat hedgehogs, math often beats all but the best foxes. If there are three possibilities (say, that China will experience more, less or the same amount of civil unrest), throwing darts at targets representing each one produces a forecast more accurate than most pundits'. Simply extrapolating from recent data on, say, economic output does even better. But booking statistical models on talk shows probably wouldn't help their ratings.


Dennis Sanders said...

I've heard the hedgehog-fox comment before, but it sometimes casts the hedgehog as the "good guy" and the fox in a more negative light.

I think frankly that there are good qualities of both hedgehogs and foxes and they both have blind spots. Making one more "noble" than the other doesn't really help things.

Chuck Tackett said...

I agree with the idea that citing one over the other really doesn't do much to help our information society but I would say that the second-to-last paragraph is exactly right on target.

My pop psychology answer is that people don't know what to think but want to feel secure that someone does know. Thus the self-assured speaker, regardless of position, attracts and retains support of those who generally think the same way.

Allan Bevere said...

Dennis and Chuck,

Thanks for your thoughts. I agree with both of you. I would only say as a kind of response to both (as well as affirmation) is that people are attracted to confident speakers who think the way they do because it keeps them from having to think too hard about issues. It allows them to continue to believe what they believe while not having to entertain data and argumentation that raises questions about what to believe. This is why political conservatives watch the evening lineup of FOX News and liberals watch the nighttime shows on MSNBC. Both of which contain hedgehogs that either find ways to ignore or twist contrary data.

I prefer the context of the round table where people who disagree have a civil discussion, and are willing to listen and even consider differing points of view.

Dennis Sanders said...

Acutally, this brings up an issue where it seems we can't have a civil discussion and that's issues over homosexuality. I say this as a gay man, but it seems we can't really talk about anything related to sexuality without devolving into name-calling or someone expressing their being hurt. How does the church talk about this and hear each other without resorting to name calling or incivility?

Allan Bevere said...


That's an excellent point; and I am not sure I have an answer for you, except to say that discussion that resorts to name calling will not move us forward in any fashion; and perhaps if the discussion is to be civil, certain persons on both sides of the matter need to be excluded until they can come to the table and act like adults.