Jenett’s quick guide to evaluating information

I got nudged by a friend to do a ramble about information evaluation. It might have gotten a little away from me.

Basic principles:

  • We all have biases and things we know more about than others.
  • Some people are more up front about this than other people.
  • Ditto goals. We all have them, some people are more up front about them.
  • Be really suspicious of the people who claim they have the absolute truth and are telling you for your own good.

(They probably don’t and they probably aren’t. Especially if you don’t have a preexisting trusting relationship. Real world stuff has fewer absolutes, for one thing.)

Information: A quick guide to information evaluation (image of a fountain pen and blank lined notebook)

Who is this person (or What is this source?)

Start with the basics. Who’s telling you this thing? What’s their background? If it’s a website without an individual author, what do you know about the site?

You may need to file this in “Need to do some more research” but knowing you need to do that is a great first step. First thing: check out the ‘about’ page, or a bio. Usually this will give you some hints on what they’re about and what they care about most.

If you’re not sure where to start with that, try searching the person’s name (plus maybe a term from the topics they write about, if you need to narrow it down) or search on the name of the site. Sometimes adding in words like ‘review’ or ‘about’ will help.

Even just knowing what kind of source this is can help. Personal website? Newspaper that’s actually well-known and reasonably respected (even if you don’t agree with them)? Pocket of internet culture you weren’t previously aware of? Political group hidden behind astroturfing techniques?

I sort things into “Probably reasonably competent”, “Dubious” and “Need more information”, personally.

Probably reasonably competent sources are those I’ve checked out before, and came up reasonably well sourced. I still need to check the specifics here, but they get some starting benefit of the doubt. Dubious sources are those that have come up short before. Everything else gets filed in ‘need more info’.

What are their goals?

Education? Information? Sell something? Share something gorgeous or fun or amusing? Are they trying to persuade you of something?

What do they get out of you believing them and taking them (or their information) seriously? Are they being up front and honest about that?

Here’s an example: sales sites are not the most fun thing ever, but there is something refreshingly honest about “Buy this thing from me and here’s why.” It’s clear what the people want, and usually pretty clear what’s involved in getting it.

On the other hand, a lot of sources in the political realm are trying to persuade you of things, but it’s not always clear what they’re trying to persuade you of. (Or whether they’re not trying to persuade you at all, but are instead signalling to their core base what they care about.)

This is often where you see a lot of vagaries and unsourced information that plays on emotions rather than treating you like the intelligent, thoughtful, considerate person I want to think you are.

Where did they get their information?

This is where we get to the meat of things. People who are saying trustworthy things should give you a way to check, or more information about how they know that.

When we’re talking to a friend, we put what they tell us in the context of all the other things we know about them. They’re reliable as anything with a ride when it’s important, lousy at getting stuff to the post office.

They have a lot of specific experience in dealing with Mercutian rabbits, and the last fifty things they told you about those rabbits turned out to be right, but they’re not nearly so reliable about Venusian wombats. And they’re normally great about Saturnian leopards, but there’s this one weird quirk, don’t trust their grooming recommendations.

When we’re reading a random website, we don’t have that. We can’t put some of what they’re saying in context without more information.

That’s why their sources matter. Do they tell us where they’re getting their info? If it’s unnamed experts and sources, be dubious. (Though there’s a link below with some more about how to evaluate this with more nuance.)

If they claim specific expertise, can you verify that or does it seem in line with what someone with that expertise would say? (If someone claims to be a lawyer or doctor or librarian and says stuff that is way outside what you’d expect, be dubious without more specifics. Maybe a lot more.)

When is this information from? Is this a topic where currency matters a lot? Some topics change fast, some don’t. Sometimes the info that debunks a current thing has been around for a while (so older info may still be helpful in sorting this out.)

What kind of source is this, and is the information presented in a way consistent with quality information in that kind of source? Reputable newspapers don’t generally go in for explicit personal insults or completely unverified sources. (Unless they’re quoting someone who used one.) Less reputable current events sources might.

Expect better of where you go to learn things. If they’re not giving you meaningful information, go to sources that that will. You can do better than speculation and gossip.

Other key tips

Beware of absolutes, especially in complex situations.

There just aren’t that many absolutes in the world. This is especially true when looking at expert statements: few experts will give 100% certainty. If they do, they will likely also be explaining why. Look for that explanation.

If a media source says something absolute, check into what the experts actually said, and what information they looked at to get there. Chances are pretty good the expert was not nearly so absolute about things.

Be dubious of things that are too good to be true, too weird, or too perfect.

Again, the world just isn’t like that very often. The more we realise that we live in a world that has a lot of shades of colour and nuance and different experiences in it, the sooner we’re going to get better at evaluating information effectively and using it well.

Is this a situation where there are strong emotions?

Sourcing is often not the top priority in these cases. Which is understandable, but just because someone’s having emotions all over the place doesn’t mean you have to use everything they tell you as the basis of your decisions.

Emotions don’t mean someone’s wrong, mind you.

It is, for example, pretty reasonable for someone to be emotional about a topic that has a major impact on their daily life, health, safety, family, or religion, if other people are treating it as a purely intellectual discussion. But a story that’s playing on your emotions to make you feel upset or riled up or righteously victorious, you should be suspicious of that.

If emotions are in play, and you’re not in the middle of the discussion, it’s usually better to pause and take a moment to look at what’s being said.

Who has real experience with this thing? Who doesn’t? How does what people are saying match up with other kinds of information you can find or your experience of people or situations? Who has what at stake? Is this a real person who has specific experiences, or is it a made up storm of emotion that’s trying to get you to react a certain way?

Some additional resources:

Here are a few additional links worth reading

This is only a beginning – there are lots of nuanced issues involved in how we find and evaluate information I haven’t even touched on here (like who decides what gets researched that you can refer to later.)

What I do: taking things in (text, sound, video)

Round about now, it might be useful to note how I take in information, and what I do with it.

I’m laying this out not because I think anyone else should do things the way I do, but because doing so gives a way to talk about some other possible approaches and issues, and figuring out what choices might work for you.

You should know about me…

I have – for reasons related to the chronic health issues – a pretty set routine about my life. This means my day to day is pretty consistent (and that I try to keep it like that.)

I read very fast compared to most people, but find watching videos for active content very tiring (and I also have a very limited amount of time that’s something I can reasonably do.) Audio’s somewhere in the middle: I can only do some things while listening to podcasts, but they include driving and time at work.

(I’m also laying this out because I’ve been thinking a lot recently about different kinds of media and how I use them, for an upcoming post, and explaining it here will make it easier to reference when I get there…)

Where I get things

Reading online

My core online reading includes my Dreamwidth reading list, and The Cauldron (the online pagan forum I’ve been on for approaching two decades in some form, and am currently staff.)

The former takes me about 10-15 minutes to read several times a day, unless someone’s made an extra long thinky post, the latter runs between about 20 minutes a day and an hour or more, depending on what replies I make.

I have a large number of blogs in my Feedly rss reader, and they produce about 120-200 posts a day. A number of these are from very busy sites (Metafilter and Ask.Metafilter) where I read only the ones I find intriguing from the opening, and it also includes a handful of Tumblr blogs where the posts tend to be short.

On an average day, I probably read 30-50 actual posts, and skim a lot of others.

I find Metafilter and Longform.org both fabulous for finding longer in-depth reading material that gets me looking at things from a different perspective, or getting me to read about something I might not have selected.

I dip into Twitter, though I have lists set up so I can keep up with a few close friends on there, and skim other things as I have time.

Finally, on the news front, I have digital subscriptions to two papers and one of my local NPR stations. They all send me at least daily updates on new stories, and I click through the ones that interest me. (Plus various other newsletters, information from organisations I donate to monthly, and the other stuff that happens in email.)

ebooks

I have a long To Be Read pile, read some amount of fanfiction on a regular basis (some of which can be quite long). These days, most of my reading is in electronic form (three long distance moves will convince even the most ardent adorer of books that moving the physical objects is a pain in the neck. And the arms and the back.)

So I mostly only buy print of titles I might want to lend out or reference with people in future (Pagan titles, mostly, or things where the print layout really matters) and everything else is digital. Conveniently, this also means I can walk around with 600+ books in my pocket, and never have to have the “Might I run out of book while out today? I’d better bring another one” mental discussion with myself, like I do with print.

My ebooks go into Marvin, an app that has a very functional list feature. I move items into “to be read” lists by broad genre (fiction, non-fiction, and Pagan) so I can skim different things, and then have a “read next” list for things I want to read sooner than later, a “read” list for things I’ve read, and so on.

I read about 10 books a month, give or take, though I’ve had months where it’s only about 4, and months with 15 or more (especially if I’m travelling.) In months where I’m short on brain, it’s a lot of reading things I already know something about, or rereading old favourites.

I’ve read for at least 10 minutes (and usually more like 30) every night of my life I could read, except for a handful of times. Since I do most of my reading on my phone these days, I also do a lot of bits of reading at other times (waiting for my work computer to boot in the morning, while things are processing, etc.)

Listening

I got the podcast bug fairly hard within the last year or so. I actually have three different kinds of podcast listening I do.

Swimming :
I swim three days a week before work, and I have a waterproof MP3 player and headphones. I put them in when I get ready to swim, and turn it off after I’ve showered and changed and gotten back to my car, which is right around an hour total. Keeps me from getting bored. I mostly go for longer ones here, and usually with multiple hosts, because I find that easier to follow early in the morning.

Work :
Some of my work tasks involve routine sorting of information that gets a little tedious, so I listen to podcasts while I’m doing that.

Since I work in a school (not directly with students, but with a range of people nearby at times), I want to be attentive to content, so I avoid some topics and podcasts. I mostly aim for history podcasts about topics I know something about (but not tons) which works out well for me.

Driving:
My commute is about 25-40 minutes (depending on traffic) and I listen to podcasts there. This is when I’ll listen to other content (there’s a couple of spooky or true crime ones that I won’t listen to at work, or Pagan/esoteric topics, and I avoid explicit political discussion there too.)

Watching

A lot of my video watching is material I am already familiar with – it’s what I have on while I’m home after work (or on the weekends) doing other things online.

Right now, I’m watching Classic Doctor Who from the beginning (BritBox made all the surviving episodes available streaming last year: I’ve seen them all at one point or another, but in many cases, not for decades.)

I usually get through 1-2 hours in a given night, but if I need to really focus on content, then the video is the only thing I’m doing, and time for that is a lot harder to come by.

What that looks like in a day

On the average workday, I get up, read my core online spaces, get dressed, get my things together, and drive to work, while listening to about 30 minutes of a podcast. (Some days I go swimming, first, and listen to about an hour of podcasts while swimming. Either way, I’m out the door within 30 minutes of getting up.)

I go to work. Sometimes I can’t listen to anything (because there are other people – volunteers, interns, visiting researchers – working in my office.) Sometimes I listen to music. Sometimes it’s podcasts. It probably comes out to 3-5 hours of podcasts a week.

During this time, I’ll take breaks while I’m waiting for things to run, check in with the core online spaces, read blog posts, and so on.

I drive home (another 30 minutes of a podcast), get home around 5 and do things like make dinner, be sat on by the cat, eat dinner, and other necessities.

From here, on a good day, I’ve got about 3 hours of time I could do something with (including making and eating dinner) before my concentration goes away. On a bad day, I’ve already run out of focus. That’s due to multiple chronic health issues that can play havoc with concentration, focus, and ability to process new stuff at all efficiently.

This is my time for writing (hi!), reading more blog posts, knitting, making images for the blog, some kinds of other project work, and basically any other small hobby things.

This is also the only slot where I can actually watch video. So for me, video has to be really extra important for me to make the time for it. Written material (blog posts, books) or podcasts, I’ve got multiple places I can do them, and they take me a lot less processing energy.

It’s often a question of “Watch this video thing attentively or write a blog post?” or “Watch this video attentively or help out a friend with a question” or “Watch this video attentively or do this project” Most of the time, the video loses.

About 9pm, I start wrapping up, do one more pass through my core online spaces, and I try to be lying down in bed by 10. I read for 15-30 minutes (This is my one solid book-type reading time, though I read in small chunks at other times during the day. I read about 10 books a month, to give you an idea of my text to audio processing ratio.) Then I fall asleep.

How does that add up?

In total, my average weekday involves:

  • 1-3 hours of audio material
  • 30-60 minutes of reading a book
  • 1-2 hours of background video time
  • A hard to count amount of time in text-based online spaces (reading, commenting, etc.) but probably 3-4 hours most days.
  • Little focused video time (an hour once or twice a week at most, usually)

My weekend time is obviously a bit more flexible – there’s no going to work in there! – but it’s also when I work on larger projects at home, or just relax.

I’m curious

Where do you get your information, and what forms work for you?

I share some of the reading I’ve found particularly interesting in my fortnightly newsletter (and a new one will be out on Wednesday) so if that’s a thing you’re intrigued by, you can sign up over here.