Fact checking: How to think like a journalist
When in doubt — and you should always be in doubt — here’s how to assess what “facts” you can trust
If you were collecting the news, what would you ask your sources? Now, are the answers to these questions in the stories you read? If not, maybe you’re looking for news in the wrong places.
PRUDENCIOALVAREZ/iStockphoto
Today, getting information is as easy as opening a browser on your computer or phone, typing in a question and waiting a nanosecond for a long list of links to load. These websites will promise to answer your question. But actually, they may not be all that trustworthy. What do you do?
Figuring out which site is believable and which is bunk takes work. Vetting that information is not impossible, though. In fact, it’s what every good journalist does daily. And students and other non-journalists can learn from the methods reporters use to determine the truth of what they read and hear.
There are plenty of people who call themselves a “journalist.” What separates a good journalist from a bad or lazy one is often where they get their information. Good reporters are always on the lookout for a “scoop” — new information or new interpretations of existing data. But for them, a scoop is worthless if it’s based on bogus or misinterpreted data.
To verify a new claim, a reporter contacts experts on the topic. Often many. In journalism, such experts are known as “sources.” This may be a scientist who discovered something new in the lab. Or it could be the witness to a crime. The more strange or controversial that some claim is, the more sources a reporter may need to contact.
In the decades I’ve been reporting on science, I’ve written more than 1,000 stories, with anywhere from 2 to 50 sources for each. As the editor of this magazine, I’ve read, tweaked — and even totally rewritten — some 2,000 more stories authored by other journalists. I have a lot of experience hearing claims of new and amazing things. Some have proven too good to be true. Others eventually were validated by research.
Sooner or later every cub reporter hears: “If your mother says she loves you, don’t believe it unless you can get confirmation from at least two independent sources.” It means question everything!
Just because I read something — or hear a scientist say something — doesn’t mean I believe it. Reporters are taught to ask questions, then verify every factoid they can. Indeed, sooner or later every cub reporter hears: “If your mother says she loves you, don’t believe it unless you can get confirmation from at least two independent sources.” That line is meant to make us question everything! Journalists are skeptics. And you should be too.
How certain can you be?
Some things are relatively easy to source. If you want to see who won a ballgame, consult the scoreboard. Or you could talk to people who watched the game, those who played in it or league officials.
But few things are as black and white as who won a ballgame.
If there’s a forest fire, how do you know where it started, when and how? You can ask government officials, but they may not know. You can ask a fire investigator or talk to people who were near the fire shortly after it started. But unless someone spotted the fire’s ignition as it happened, people may never know for sure. They may simply offer speculation. Saying something “might have” or even “probably” started that fire is not fact. So you would have to qualify any statement about the fire’s origins.
I describe such qualifiers as “weasel words.” These words and phrases include “might,” “could have,” “was likely to have been” or was “suggested” to have happened this way. Whenever something isn’t known for sure (meaning the sources of that information don’t either), journalists will highlight their uncertainty by using weasel words. They are a sign of a good journalist. They are being honest with the reader about the uncertainty rather than overblowing claims.
How do I know if such weasel words are necessary? When I consider a claim from a scientist, I ask a lot of questions. I consider whether I can trust that person. Did she collect the data personally? Did she publish the data in a respected peer-reviewed journal (which means that other experts had to accept the data as looking reliable)? How strong are the statistics behind her findings?
If I can get an original paper in which the findings were published, I might check for the answers. But I am not an expert, so I often turn to outside experts for help. These are people who are not directly related to the new research. I’ll ask them to look at the paper and give me their assessments.
Reporters who don’t have much time or motivation may just quote the first scientist, accepting anything she said as fact. If the reporter didn’t get an outside comment, the reader will not know how well to trust the original source. Keep that in mind when you read news reports.
Figuring out who to trust
Good reporting is time-consuming. It also can be costly. If the news you are reading was free to the world on the web, consider what that may mean. A good reporter should be paid for all of the work that he does, as are all of the people at this magazine (and at its sister publication, Science News). Some blogs and even news sites are authored by unpaid writers. Many of these writers are untrained in reporting. Their reports may not provide the depth and care that a professional journalist would have employed to check the facts.
Also, some “news” stories are written by people working for organizations that are more interested in promoting a certain point of view than in accuracy. Or the entire site may be biased, from the owners down to the journalists. This can cause them to willfully mislead their readers. You might search for the writer online and figure out who pays them. If the money comes from a vaping company, for instance, what is written about e-cigarettes might not be entirely the truth.
Likewise, check whether the site you’ve obtained information from appears to have done a good job in vetting what they’re reporting. Particularly where the news seems “unbelievable,” look for signs that the reporter was appropriately skeptical of such claims.
And don’t stop there.
Is the person who wrote a story or interviewed a source named? Was this writer a journalist — or just someone who heard about some new claims? Did the reporter cite his sources by name? Did he describe what qualified the cited sources as experts for this story? Did he give you enough information to fact check the data he mentioned, such as citing a published research paper that you can look up?
Are the scientists who are quoted in a news story expert in the field at issue? It may seem obvious, but details of a story on climate science are best interpreted by other climate scientists. When a physician, politician, economist or theologian comments on new data in a story on climate or another field outside their own, keep in mind that these people may be “experts” — but not necessarily in a field that enables them to fully understand the new findings.
One place you can turn for solid news is this publication. We are professional science journalists with degrees in science or journalism — and often both. We speak with and quote experts. We ask for outside comment. We cite our sources. But first and foremost: We are skeptical.
You should also be skeptical — of everything you read, even here. That’s acting like a journalist.
