Skip to main content
It looks like you're using Internet Explorer 11 or older. This website works best with modern browsers such as the latest versions of Chrome, Firefox, Safari, and Edge. If you continue with this browser, you may see unexpected results.

Research: Evaluate

Easy to fake

Evaluating online information these days is not like it used to be!

The things we used to be told to look for in our evaluation of information are easy to fake. It's really cheap to get a good-looking site nowadays. You can be a disinformation agent from China or Russia and still use spell check. Anyone can add footnotes to make something look more serious (students do it all the time!) The same holds for using scientific language.

And a lot of things we used to be told to look for have no basis in reality at all. Current fact checking researchers have found that students believe

  • .orgs are better than .coms (No, they're not.)

  • non-profits are better than for-profits (Wrong.)

  • Fewer ads on a page means the page is more reliable (Not even close.)

To put it simply — the things that might mean something are easy to fake, and the checklist we used to teach in evaluating online resources just doesn't work anymore.

Further reading: Educating for misunderstanding: HOW APPROACHES TO TEACHING DIGITAL LITERACY MAKE STUDENTS SUSCEPTIBLE TO SCAMMERS, ROGUES, BAD ACTORS, AND HATE MONGERS.

The history of .org

Why the CRAAP test doesn't work.

But hang on, haven’t teachers been telling us not to use Wikipedia because it is not a credible source. Now you’re suggesting using Wikipedia to see what others say about a source or an organization. How should I understand these different recommendations?

You may have heard in the past that you should avoid Wikipedia as a source. We (like Mike Caulfield) will instead encourage you to use Wikipedia, but to do so while recognizing both its strengths and its limitations. Most Wikipedia articles are highly accurate, as Wikipedia has editors who work to ensure that Wikipedia content adheres is its editorial practices, including providing evidentiary sources. Wikipedia articles that are longer and that are older tend to be of higher quality because they have been developed and improved over time by individuals who follow Wikipedia's best practices. That said, it's still true that someone can put inaccurate information on Wikipedia that is not immediately corrected. Wikipedia articles that are about contentious topics and that are undeveloped should be evaluated with greater care. You can also use the references at the end of a Wikipedia page to help you determine its level of accuracy.

 

But fact checkers use Wikipedia differently than many students do. They skip the main article and dive straight to references, where more established sources can be found. 

They know that the more controversial the topic, the more likely the entry is to be "protected" through the various locks Wikipedia applies to prevent changes by anyone except high-ranking editors. 

Fact checkers also know how to use a Wikipedia article's "Talk" page, the tab hiding in plain sight next to the main entry—a tab that few people know about, let alone consult. The “Talk” page is where an article's claims are established, disputed and, when the evidence merits, altered.

See the guide for using Wikipedia in your research.

Map of Wikipedia

Creative Commons Licence
This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License.

Please read our license before re-using or sharing.

What is a peer-reviewed article?

In secondary school we are always looking for relevant, authoritative information sources, but not always peer-reviewed articles. This is not the case at university.

Perspective and bias

While some sources are primarily informative and others are more opinion-driven, almost all sources reflect a certain perspective (and along with it some degree of bias). This perspective influences what information the creator includes or excludes and how they present that information.

Rather than looking for sources that are completely free of any bias, recognize that most sources have some degree of bias. This is not necessarily a bad thing: people's personal experiences and viewpoints often provide important insights into an issue. Consider what the source creator's perspective is, what expertise they have on the topic, and what evidence they use to support their claims or arguments. Verify evidence by reading laterally and looking at other sources, including ones that may present a different perspective that is still well supported by evidence.

The Human Brain & Confirmation Bias

Our brains are wired to believe things that fit with our preexisting views and to disbelieve those things that challenge our views. This phenomenon is called confirmation bias.

Confirmation bias plays a powerful role in how we evaluate and use information. It's a major reason that misinformation easily spreads online. Learn more from this video about how confirmation bias influences us and we can counteract it.

 

Source

The Quality Information Checklist

 

Source

In this method, these skills are called “moves” and each one is tied to a couple of simple skills you can usually execute in less than a minute.

These skills will make a dramatic difference in your ability to sort fact from fiction on the web (and everything in between).

Stop

The first move is the simplest. STOP reminds you of two things.

First, when you first hit a page and start to read it — STOP. Ask yourself whether you know and trust the website or source of the information. If you don't, use the other moves to get a sense of what you're looking at. Don't read it or share it until you know what it is.

Second, after you begin the process and use the moves it can be too easy to go down a rabbit hole, chasing after more and more obscure facts or getting lost in a "click cycle". If you feel yourself getting overwhelmed in your fact-checking efforts, STOP and take a second to remind yourself what your goal is. Adjust your strategy if it isn't working. Make sure you approach the problem at the right amount of depth for your purpose.

Investigate the Source

More about this later. The key idea is to know what you're reading before you read it.

This doesn't mean you have to do a high powered investigation into a source before you engage with it. But if you're reading a piece on economics by a Nobel prize-winning economist, you should know that before you read it. Conversely, if you're watching a video on the many benefits of milk consumption that was put out by the dairy industry, you probably want to know that as well.

This doesn't mean the Nobel economist will always be right and that the dairy industry can't ever be trusted. But knowing the expertise and agenda of the source is crucial to your interpretation of what they say. Taking sixty seconds to figure out where it is from before reading will help you decide if it is worth your time, and if it is, help you to better understand its significance and trustworthiness.

Find trusted coverage

Sometimes you don't care about the particular article that reaches you. You care about the claim the article is making. You want to know if it is true or false. You want to know if it represents a consensus viewpoint, or if it is the subject of much disagreement.

In this case your best strategy is to ignore the source that reached you and look for other trusted reporting or analysis on the claim. In other words, if you receive an article that says koalas have just been declared extinct from the Save the Koalas Foundation, the winning strategy may be to open up a new tab and find the best source you can that covers this, or, just as importantly, scan multiple sources to see what the consensus seems to be. In these cases we encourage you to "find trusted coverage" that better suits your needs — more trusted, more in-depth, or maybe just more varied. 

Do you have to agree with the consensus? Absolutely not! But understanding the context and history of a claim will help you better evaluate it.

Trace claims, quotes, and media back to the original context

A lot of things you find on the internet have been stripped of context. Maybe there's a video of a fight between two people. But what happened before that? Who started it? What was clipped out of the video and what stayed in? Maybe there's a picture that seems real but the caption is dubious at best. Maybe a claim is made about a new medical treatment supposedly based on a research paper — but you're not certain if the paper supports it.

In these cases we'll have you trace the claim, quote, or media back to the source, so you can see it in its original context and get a sense if the version you saw was accurately presented.

It's about REcontextualizing

There's a theme that runs through all of these moves: it's about getting the necessary context to read, view, or listen effectively. And doing that first.

One piece of context is who the speaker or publisher is. What's their expertise? What's their agenda? What's their record of fairness or accuracy? So we investigate the source. Just as when you hear a rumour you want to know who the source of it is before reacting to it, when you encounter something on the web you need the same sort of context.

When it comes to claims, a key piece of context includes whether they are broadly accepted or rejected or something in-between. By scanning for other coverage you can see the expert consensus on a claim, learn the history around it, and ultimately land on a better source.

Finally, when evidence is presented with a certain frame — whether a quote or a video or a scientific finding — sometimes it helps to reconstruct the original context in which the photo was taken or research claim made. It can look quite different in context!

In some cases these techniques will show you claims are outright wrong, or that sources are legitimately "bad actors" who are trying to deceive you. But even when material is not intentionally deceptive, the moves do something just as important: they re-establish the context that the web so often strips away, allowing for more fruitful engagement with all digital information.

Source

Fact-checking sites

Fact-checking Sites

The following organizations are generally regarded as

reputable fact-checking organizations focused on U.S.

national news:

Politifact

Factcheck.org

Washington Post Fact Checker

Snopes

Truth be Told

NPR Fact-Check

Hoax Slayer

Respected specialty sites cover niche areas such as climate

or celebrities. Here are a few examples:

Climate Feedback

Quote Investigator

Respected specialty sites cover niche areas such as climate or celebrities. Here are a few examples:

There are many fact-checking sites outside the U.S. Here is a small sample:

Full Fact (UK)

Practise your fact-checking skills

Practise your skills.

Play fakeout  
https://newsliteracy.ca/fakeOut/

Ctrl-F Find the facts. 

Learn to spot fake faces at a glance

Which face is real (play the game).

Subject guide created by

Guide created by Tania Sheko. Contact me