Science Needs a Truth Squad
The Washington Post has a regular column called “The Fact Checker,” by Glenn Kessler, a longtime Post reporter. It’s a relatively new feature. Earlier this year, Kessler described the column’s origins and purpose:
My colleague Michael Dobbs started the column during the 2008 [Presidential] campaign and now, in 2011, The Washington Post is reviving it as a permanent feature.
We will not be bound by the antics of the presidential campaign season, but will focus on any statements by political figures and government officials–in the United States and abroad–that cry out for fact-checking. It’s a big world out there, and so we will rely on readers to ask questions and point out statements that need to be checked. Over time, we hope to build this page into a more interactive feature than the blog it has been.
The purpose of this website, and an accompanying column in the Post, is to “truth squad” the statements of political figures regarding issues of great importance, be they national, international or local. As the 2012 presidential election approaches, we will increasingly focus on statements made in the heat of the presidential contest. But we will not be limited to political charges or countercharges. We will seek to explain difficult issues, provide missing context and provide analysis and explanation of various “code words” used by politicians, diplomats and others to obscure or shade the truth.
All this makes total sense, of course. And it’s a great public service. But why only for politics? Science is also a battlefield, with claims, counterclaims and all manner of misstatements that cry out for fact checking. Climate science, a subject that is often hotly debated in the public arena, would obviously be a recurring topic in any such Truth Squad column. So would nuclear power, biotechnology, evolution, and many medical and health-related issues that are often in the news.
If it’s important to gauge the accuracy of what politicians say about the budget deficit, foreign policy, Medicare, etc., it’s equally important to gauge the accuracy of what newsmakers say about climate change, stem cell research, vaccines, evolution, and so on.
The big newspapers, such as The New York Times, Washington Post and USA Today, have eminently qualified science reporters that could be charged with a column that fact checks questionable scientific statements made by government officials, politicians, and even widely read pundits.
Science is just as important to society as politics. And just as is the case with political and policy related issues, the public often has trouble separating out fact from fiction on many scientific claims and statements.
Science, like politics, needs a Truth Squad.
” But why only for politics?”
I think this is one reason the IPCC was created. If people don’t believe the IPCC and every science academy in the world, why are they going to believe a science fact checker? I suppose it would be helpful to some people, but I don’t think it would change the underlying dynamics of the climate issue. Too many bottom lines and lifestyle issues are affected.
Dean,
I’m not thinking only in terms of the climate debate. Nor do I think a science fact checker is going to change the dynamics of any one debate (be it on climate change or nuclear power) any more than a fact checker of political statements is going to change the debate over health care or the deficit.
But there is something important (and intangible) about accountability, which I believe in strongly. And at the end of the day, highlighting false or misleading statements of public figures continues the conversation of important issues (and hopefully sheds more light on them), all which is a net plus for me.
But as one mainstream science journalist has already reminded me offline, science reporting remains ghettorized in newspapers and is not accorded the same value as political reporting.
But that’s a topic for another post…
I would suggest they look at the cellphone-cancer pronouncement…
Why? Well it’s not nearly as politically charged as climate change and it’s timely… it would be a real good place to expand on the headlines that were presented…
It is a great idea!!
Can we get Mann to defend the “accuracy” of the hockey stick ?
.
/sarc off
.
We have LOTS of groups spinning now. I have yet to see a “truth squad” that does not have an agenda that does not push one side over another.
.
These “truth squads” also need to be funded. Not much funding out there if you are not pushing an agenda.
Keith – I have no problem in principle with a fact-checking team or system. I would stay away from phrases like “Truth Squad”. The first question is how to staff the team in order to maximize credibility.
The second question comes up when the truth is inconvenient. People are fond of criticizing climate scientists who say things that come across as extreme because they are politically unpopular. But sometimes they are still true. Isn’t this the basic message of the Breakthrough folks? The truth is politically unpalatable, so hide it for something that they think is more politically feasible? Wouldn’t you be one of those who would eventually tell a Truth Squad to say something other than the unvarnished truth in order to build bridges?
Ed,
My suggestion is that a seasoned science reporter at a mainstream newspaper truth-squad false or misleading claims made by newsmakers.
I don’t see that as spinning, though I realize that in this climate, it will be hard to reach people whose minds are already made up on certain issues. For example, I have little doubt that diehard Sarah Palin fans will be moved in the slightest by the fact-checking here of some of her recent statements.
Dean (5)
Whoa, whoa. Bridge building, in my mind, does not involve shading the truth. It’s an acknowledgment that you and someone else sees something differently, and then you try to find ground that can accommodate you both.
I also don’t see bridge-building as part of the fact-checker’s purview.
Again, you are looking at this idea (science fact-checker) solely through the climate lens. I see it as having much wider application.
@7
I suppose we tend to evaluate things on this blog in relation to climate. While such efforts are probably not going to work for the most politicized science issues, there are many that it could be helpful for. And just maybe some issues might become less politicized if such an effort could prevent myths from getting established.
While in principle, it could be a function of journalism, it seems more realistic if it came from the NAS or something similar, since they already have the expertise available.
“…My suggestion is that a seasoned science reporter at a mainstream newspaper truth-squad false or misleading claims made by newsmakers…”
Like all those seasoned science reporters working for mainstream newspapers who jumped on the false notion that tornado events were increasing due to CO2…..O’wait….they were the ones pushing this crap.
If it needs a “truth squad”, by definition it is political, and there are axes to grind, news to spin, and interest groups to placate.
So if research overturns the current “truth”, what is the “truth” squad to do. As an example, there was a consensus view that there was a global MWP. Now some or many climate scientists say there was no global MWP. One unnamed climate scientist even said[paraphrasing]that we had to get rid of the MWP.
Now what is the “truth”?
If it is the consensus view then what do we do when the consensus view es. Does the “truth” change.
Does this mean that the “truth” squad would be better named then “consensus” view squad.
Philosophers have debated for centuries about just what the “truth” is and if science is a means for finding the “truth”. That science can veer between diametrically opposite “truth” (e.g. the MWP did or did not happen) then the question of just what scientific “truth” is can be seen to be a difficult question
If the “truth” is taken as the consensus view then the creation of a “truth” squad to hunt down and reveal errors is something that would be very frightening to me in a political sense. Who guards the guardians?
Hi Keith, it’s always sobering when you translate a political idea into German:
Wahrheit-Staffel would be the “truth squad”
Ben Goldacre does quite a good job on medical and drug trail issues:
http://www.badscience.net/
but he does get sued for his pains!
*****
Where science interacts with the media especially when combined with policy it is not what is untrue that is often the issue but what is unfalse.
Particular choices of wording that convey things that are misleading but are not readily falsifiable, and are likely to tie a challenge up in knots.
Now the difference between:
“Based on available evidence it was concluded that A was unlikely”,
and
“It was assumed that A was unlikely”
is small but provided one can be reasonably sure that it is not open to direct challenge it is an unfalse but casts a different light.
It is a useful ploy if one simply wishes to cast doubt in a readers mind or to reinforce a readers prejudice. A fair number can be included in one published essay. A blogger would most certainly be called out for this, but an academic or policy author frequently can not be so easily challenged on the same page as the essay.
Alex
The chances of any ‘truth squad’ presenting the unvarnished truth is next to nil. They would be nothing but political advocates masquerating as scientists.
“Whoever undertakes to set himself up as a judge in the field of Truth and Knowledge is shipwrecked by the laughter of the gods”.
Einstein
Who truth-checks the truth checkers?
In areas of controversy, each side expresses a view that they each consider to be the truth. They each look at what the other side writes, and thinks “somebody ought to truth-check that.” What they mean, of course, is to have some authority that shares their own opinions on what is true to be given the imprimatur and authority of Truth, the better to defeat the other side.
If you want a classic example, Bjorn Lomborg did exactly that. He went through all the claims of the Malthusian Environmentalists and checked them against the official statistics, to see if they were true. He had over 150 pages of notes and references giving all his sources.
And what happened? Everyone who agreed with him thought it was one of the greatest truth checks ever written, and everybody who didn’t agree with him thought he was biased, a discredited propagandist, a delayer, a shill for the uncaring capitalist polluters, a dangerous threat to popular support and funding for essential efforts to save the world.
And so of course Lomborg gets “fact-checked” and the fact-checkers get fact-checked in return, and we’re off again.
Basically, truth-checking is the role assigned to the opposition in any debate. That’s why we have free speech and open debate. The truth about anyone who sets themselves up as a truth-checker is always that they are a player, not a referee.
I’m puzzled by by some of the objections expressed here. I’m not suggesting that there be arbiters of “truth” in the absolutist sense. I question how some of you get by day to day. When you look at the sky and it’s blue, do you say to yourself, hmm, I wonder if that’s really a blue sky, or is someone to trying to pull a fast one on me?
Perhaps some of you need examples of what I’m referring to. Here’s one. I’ll provide a bunch more later tonight when I get home.
“I question how some of you get by day to day. When you look at the sky and it’s blue, do you say to yourself, hmm, I wonder if that’s really a blue sky, or is someone to trying to pull a fast one on me?“
We can use that as an example, if you like. Everybody knows that the sky is blue. It’s proverbial. There’s even a shade of blue named after it. So what are we to do when some people come along and say that they’re just seen a red sky, with pink clouds?
Why, you dismiss it as nonsense, of course. Blue is the mainstream scientific consensus. We asked thousands of scientists and scientific organisations what colour the sky was, and they all said “blue”. They even have it on their list of most frequently asked questions: “Why is the sky blue?” It’s because of Rayleigh scattering, which is absolutely basic and indisputable physics. And clouds are white, right?
So if somebody says “I’ve just seen a red sky, with orange and pink clouds”, they must be lying, right? And you don’t want the public to be misled into unscientifically thinking the sky might be red instead of blue. Science is just as important to society as politics. So you need someone to go round everyone they’re talked to and firmly put them right. The sky is blue. The mainstream consensus says so. Science is a battlefield, with claims, counterclaims and all manner of misstatements that cry out for fact checking. In science, as in politics, statements are either true or not true, with no room for honest disagreement. Science, like politics, needs a Truth Squad.
The problem is, I’ve seen the sky red. I get up before dawn for half the year. So what am I supposed to think of a Truth Squad that insists I’m wrong and it must actually actually be blue? And people who say I must be wrong because the Truth Squad says so? Could it be that they only say it because they’ve been told themselves that’s the consensus? Are their methods – in taking votes and following authorities – not more political than scientific? Are they not just stating one side of the controversy, and declaring by fiat, by virtue of their name that it is the Truth?
How can you have a Truth Squad rule on a subject of genuine controversy that does not risk being simply a partisan for one side? On what basis do you decide which Truth Squad is telling the Truth?
It’s a liberal thing: We’re smart, you’re not, we need to tell you what’s “truth”.
Come to think of it, haven’t there been things like this is some of the lesser known countries. Re-education, or something like that. I don’t know. But I’m glad to know that someone out there is keeping an eye on the “truth”.
Is the old adage “Red sky at night, sailors delight. Red sky in morning, sailors take warning” true, or is it just an old wives’ tale?
Nullius, Kdk33, et al–
I think i get it now. You’re not really climate skeptics or contrarians, or whatever: you’re relativists.
#20,
Nope, definitely not relativists.
I can’t speak for the others, but I would hold that there is an absolute truth, but that human reasoning is subject to unconscious cognitive biases and so has difficulty determining the truth reliably by itself. We address this problem by holding a debate between people with many different biases, on the basis that people who disagree with me are the most likely to find the flaws in my reasoning. Ideas that survive challenge, in circumstances where one can reasonably expect flaws to be detectable, gain in confidence.
It’s a bit like an evolutionary approach to scientific progress. People ask how blind randomness in nature can lead to the illusion of design in much the same way we ask how fallible human reasoning can lead to the brilliant scientific discoveries we see all around us today. The answer is methodological scepticism: – everything is open to challenge, there are no authorities or experts, every statement requires proofs founded in empirical evidence that can meet and survive every challenge.
I’m pretty sure that’s something you agree with too – in theory at least.
The problem with your Truth Squad is that it is expressed in the singular – it’s one squad, one position, one perspective, one side of the debate. Every example I’ve seen purports to be telling the absolute truth in a world of lies, and every one I’ve seen is used as a device to give their own particular perspective in active debates the extra kudos of being an impartial fact-checker with access to the absolute truth, the better to beat up their opponents.
Yes, quite often they’re right. Often they’re wrong. Often they’re selective about what myths to debunk and what arguments to present. None of that is very surprising – they’re human. But this is precisely why we built the principle of falsification into the scientific method. You can’t find out the truth by reading what the Truth Squad has to say, you can only find out by reading what their opponents have to say and seeing if their best arguments are any good – something which everyone has to judge for themselves.
I’m not saying it wouldn’t be useful to have resources where the best arguments for each side are collected for easy access – but don’t go calling them “Truth Squads”. That’s just asking for ridicule.
“The problem with your Truth Squad is that it is expressed in the singular it’s one squad, one position, one perspective, one side of the debate.
No, the problem with this whole conversation is that you distorted it by focusing on one line. The idea by behind the post is simple: politicians and other newsmakers sometimes make patently false and/or misleading statements–either out of ignorance or to score points. Do we agree on that much?
Now, we know this happens in the political realm. Hence The rationale behind the fact checker in the WaPo.
Do we agree the same thing happens with respect to science?
Let me stop there and see if we can at least get agreement on this much.
#22,
Yes. Agreed.
So why would you object to a “fact-checker” type column in a newspaper calling out misstatements on issues related to science, be it about climate change, biotechnology, stem cell research, etc?
In this age we live in, if the designated fact-checker at a newspaper gets it wrong, there is no shortage of eyes in the media landscape and in blogland to challenge the fact checker. So that would seem to serve as a check on the checker.
I’m not seeing what the big deal is.
That the fact checker gets to decide what facts to check is a problem. If they already have a particular perspective they aren’t likely to check what they already agree with, even if it is actually wrong.
The fact checker can also avoid correcting the erroneous media mocking of the truth of someone saying Paul Revere told the British the Colonials were expecting them. Even when that fact checker supposedly did a fact check of the same bus tour where it was said.
Selective checking certainly isn’t unbiased checking.
It’s not a big deal, and I don’t “object” as such. I’m just saying that any such function will naturally do so as a partisan of a particular viewpoint, and on issues of controversy will naturally use their position as “fact-checker” to beat up their opponents in the debate.
It would be more honest to do it as an editorial, to represent it simply as the journalist’s or paper’s own viewpoint, rather that claim to be presenting “facts” or “truth”, beyond the extent that everyone presenting their views does so claiming them to be true.
It is highly unlikely that a newspaper will be able to correct every error, so it must be selective. Which errors will they choose to highlight and correct? Obviously, the ones they notice, and care most about, because they conflict with their own beliefs. And if it becomes known that fact-checkers are more believed, then any politician or partisan can set their own up. Because we all know this, so-called fact-checkers are cynically seen as attempts at dressing up opinion in a cloak of authority and applying a more subtle pressure in the debate. That’s exactly how we see those operating in the political field, and it doesn’t look good to apply political methods to science. It’s ultimately counterproductive.
As you say, if a fact-checker gets it wrong, there are plenty of eyes to object. But why does this not apply to the original statement? If it is good enough for fact-checkers, why is it not good enough for those whose facts they check? What is gained?
KK writes
re 22
==============
Now, we know this happens in the political realm. Hence The rationale behind the fact checker in the WaPo.
Do we agree the same thing happens with respect to science??
=====================
This is the role that Climate Audit plays. It examines published papers fro errors. it performs that auditing function or in other words “truth checking”. It revealed that the hockey stck widely published in the IPCC TAR and touted widely everywhere else was based on faulty mathematics and poor proxies.
Now there are a few thingsa that are relevant here
a) Steve McIntyre is brilliant but it takes a massive amount of work, even for someone as brilliant as him, to accomplish something like this
b) for hi efforts he has been vilified across the Internet by the AGW establishment
and c) this auditing function should be an inherent part of the IPCC process. The current IPCC process of being a repository for peer reviewed reports is totally inadequate given the size and the gravity of the problem. it needs a radical revamping and new personnel in a new organizational structure.
The AGW establishment has already shown their regard for external oversight by the press or elsewhere from the talk of cutoffs and unreliable people in the Climategate emails. If this is left to the press it will inevitably fail. There will be a tame group of reporters asking puff ball questions of AGW establishment members.
To succeed, this would require a distinguished scientist whose opinions could have career limiting consequences on the members of the AGW establishment. If he/she notes shoddy research then the scientists or group in question could expect severe criticism with financial consequences. This may include a withdrawal of all future research funding.
Inevitably even this supervisory position will be captured by the AGW establishment. We have seen to many commissions of inquiry that followed the party line. So the position and organization will have to be restaffed periodically. So the position itself should only be temporary with the supervisor’s scientific reputation and legacy depending on his/her behavior in the post.
Hear, hear! (to Keith)
re 29
Bart Verheggen writes
===============
Hear, hear! (to Keith)
================
It is “Hear, hear!”s like this that make me pessimistic about the AGW issue. Climate scieitists just do not see themselves as part of the problem of public acceptance.
RealClimate was supposed to be, in part, a truth checking web site. They try to be but have failed very badly. Part of the reason that they have failed is that they do not see themselves as failing.
This presumes 2 things: that facts matter, and that they will make a difference to the polarized sides.
As much as I wish that was true, it’s just not. I mean, if the Paul Revere history facts didn’t matter to Palin’s acolytes, what more evidence to we need to show that facts don’t matter.
And that it makes a difference. Chris Mooney has been writing about how contrary information makes the most polarized adherents of a topic even more steadfast.
Listen–I’ve been on the side of science in every political battle of note in the last few years: evolution, stem cells, climate, alt-med, vaccinations, GMOs, and more–and we keep trying to use facts, peer-reviewed material, and the weight of scientific evidence. And it keeps not mattering.
I’m always looking for what works, but facts don’t seem to be among the candidates.
Mary,
You’re right that such fact-checking won’t matter to those with entrenched views.
But I still see value for the average news consumer that perhaps only pays passing attention to even the most controversial issues.
I’d still like to know what “facts” are.
The current issue of the American Scientist has an article from a climate scientist on reconstructions of the climate in the Plicoence – three million years ago. According to the article the climate of the Pliocene was very much like the climate that is projected for the end of this century.
The article has an illustration of the shoreline of the south eastern US in the Pliocene. The shoreline was 25 meters higher than today. Is this a “fact”?
The article has an illustration of the Mann hockey stick with measured temperatures appended for recent time. The illustration does not show any confidence intervals or make any note of the difficulty in matching measured temperatures to proxy reconstruction. Is this illustration a “fact”.
Tom
your solipsistic attitude is tedious.
Solipsistic — I thought that it was lonely here
All I am asking for is just what are the “truths” that will be the basis of the “truth checker”. It “truth” to be defined as being coherent with some scientific authority. So, for example, the “truth checker” will be able to cite the chapter and line of a consensus IPCC report. is this what is meant by “truth checking”?
So fro AGW, is the” truth” based on the current scientific “consensus” that is produced by the political appointees to the IPCC. Is the”truth” contained withn the four corners of the current IPCC assessment report?
This isn’t solipsism but some form of scientific scholasticism.
Mary #31,
If you think Paul Revere didn’t tell the British the Colonials would be waiting for them, you are a victim of a misconception. Sarah Palin was correct. The mocking by media opponents of Palin just demonstrated their own ignorance of the event. Similar to the mocking dished out when Palin said “party like it is 1773” about which she was also correct. That was precisely when the Boston Tea Party occurred. The Palin mockers were also ignorant on that one.
One would almost think the media bias against Palin is so strong they don’t even mind looking ignorant as long as they can try to besmirch her intelligence. The other possibility is maybe they are actually that ignorant.
That the WaPo fact checker supposedly fact checked her bus tour and ignored checking about Paul Revere certainly doesn’t inspire confidence in his neutrality.
Here is Paul Revere in his own words. He even volunteered what size force the British might be facing. Pertinent part is beginning of page 4.
http://www.masshist.org/database/img-viewer.php?item_id=99&img_step=1&tpc=&pid=&tpc=&pid=
It isn’t hard to see why public distrust of the media is on the rise. Adding another layer of media fact checking to the layers of fact checkers the media already touts they have won’t improve the situation.