In the internet era, information is easier to come by than ever before. We have personalized feeds set up for us to continually keep us aware of things happening in the world around us. We have routinely updated data portals set up by local, regional and national governments to allow us to obtain spreadsheets that contain information about the communities we live in. And we have a constant stream of interactive data visualizations created by a whole host of organizations and individuals that give us a powerful picture of what’s going on. That’s a really good thing.
But there’s a huge problem with this Infotopia we live in. Misinformation is also far easier to come by than ever before. There’s a severe quality problem with the information at our fingertips. Or more particularly, there’s a lack of good information about the quality of our information. And it’s way too easy to just run with whatever we read on social media, download from those portals, and see in that viz we come across.
Jumping to Conclusions
Case in point: last week I attended the Tableau Conference in Austin, Texas along with 13,000 other people. Unrelated to our event there were people peacefully protesting the election in the streets of Austin not far from our conference location, and in between where the conference was held and where our evening event was to be held. Tableau had arranged for a fleet of buses to convey conference attendees to the party that night, and the buses were lined up on the street waiting for us. That sets the stage for what happens next.
A man from Austin by the name of Eric Tucker noticed the buses in the area, assumed they were being used to bring in professional protesters, and tweeted photographs of the buses along with the statement “Anti-Trump protestors in Austin today are not as organic as they seem. Here are the busses [sic] they came in. #fakeprotests #trump2016 #austin”:
The tweet got picked up by a variety of partisan sites. But 17,000 retweets later, it became apparent to Eric that his assumption about the purpose of the buses was totally wrong, and he deleted his tweet. Local news in Austin covered the situation and interviewed Eric, and Eric himself wrote a blog post about his mistake and why he took down the tweet. Read his blog post – it’s quite impressive. I don’t mean to disparage Mr. Tucker whatsoever. I’ve jumped to similar conclusions many times in my own life. We all have. At least he owned up to it.
But this situation highlights a broader problem with information on the internet in the 21st century. Eric assumed that when supposed “news” organizations picked up on his tweet and spread it, that they would do their homework and research whether he was right about the buses or not. They didn’t. And in a world where 44% of adults in the US get news on social media either often (18%) or sometimes (26%), the lack of fact checking is quite scary.
On Getting Bamboozled by Satire & Fake News
Unfortunately, it goes way beyond honest mistakes like Eric’s. There are entirely fake news sites and “satire” news sites that now exist to profit from the clicks, shares and ad revenue that they generate with their intentionally partisan and provocative articles, and sometimes just click-baity lies.
I actually got bamboozled by one such article myself late last year about a “Map to Multiplication” that Nikola Tesla supposedly created in 1912. I even created a fancy viz about it.
As cool as the map really is (and my fabulous interactive remake, if I do say so myself), the article is complete malarkey. It was published by “cbsnews.com.co”. It’s a complete scam news site. But the article has been shared on social media over 50,000 times.
The map was actually created by math teacher Joey Grethner who admitted to the hoax and who said that the experience of creating the hoax was “enchanting” and declared that it would not be the last time he did it. I’m lucky that I stopped just barely short of contributing to the madness after double-checking the source, but not before I wasted an entire afternoon making a viz. Here’s what the footer of the fake “CBS News” site says. I apologize if you find it crass or insensitive. I find it absolutely moronic:
Copyright 2016 | All Rights Reserved | Powered by HITTEKK | Proudly owned by CBS News President & CEO, Dr. Paul Horner. We need writers! Contact us! Looking to advertise? Contact us! All trademarks, service marks, trade names, trade dress, product names, images and logos appearing on the site are the property of their respective owners. | Do you have a complaint? We love to hear them! You can call our complaint department directly at (785) 273-0325 | Do you have a problem with self-rape? Are you looking to get off the Devil’s playground? Fappy The Anti-Masturbation Dolphin can help! Praise Fappy!
The Importance of Fact Checking in Journalism
It’s ironic to me that during the conference, and just prior to the bus fiasco, I tweeted a link to a praiseworthy discussion between some very talented journalists who routinely work with data:
These journalism tips for avoiding data pitfalls apply to all: Fact check, find the human story, talk to sources https://t.co/63yrIVZEo2
— Ben Jones (@DataRemixed) November 8, 2016
In their discussion, they talk about the importance of fact checking, interviewing your data, proactively seeking out other sources of information that challenge the veracity of what you already have, and reaching out to the individuals who are responsible for creating the data. Anyone who publishes information to the internet should read this. That includes anyone who blogs, creates data visualizations, or shares news on social media.
So basically all of us.
Just How Sure Are You? The Role of Uncertainty
Along these same lines, I was interviewed by the Seattle Times about the fact that Donald Trump outperformed the polls quite handily in the recent election. Now, I’m not a pollster or an expert in political forecasts and election data by any means, and I’m not going to pile on and bash those who tried to make predictions. They have a tough job, and I was as surprised as anyone. But I shouldn’t have been. There are always statistical and systematic errors in polling, and the very nature of predictions is probabilistic. A model that predicts a 70% chance of victory for Clinton means that it also predicts a 30% chance that she will lose. So saying the predictions “got it wrong” is not quite right.
I’ve been fortunate enough to work with data journalists all over the world for almost four years now, and the topic of voting comes up multiple times per year. What I’ve noticed is that we don’t always convey the inherent uncertainty very well. Or we do, but it’s not received properly by our readers. So that’s on us. These concepts aren’t incredibly complicated compared to, say, differential calculus, but they’re certainly not well understood by many. So there’s a problem that we as information purveyors need to correct.
Our readers need to understand that there are sources of error and uncertainty to be taken into account. So do we. And it’s not just voting, it’s every topic.
Taking into account these recent situations and discussions, I’d like to echo the journalists in the article referenced above and relate some important tips to avoid the egregious data pitfall of creating and sharing misinformation:
- Check your sources. Where does the data come from? Who created it? What are their incentives? Talk to them.
- Rigorously scrutinize the data. Seek to debunk it before others do. What are the limitations of the data? What are the errors in it. And yes there are always errors in it.
- Get more than one person to look at what you have found or made. Ask at least one subject matter expert and one complete novice.
- Describe your methods and list all of your data and information sources. Don’t be lazy in this regard.
- Be clear about any uncertainty in the data. Give your readers the benefit of the doubt that they can understand these concepts.
- SLOW DOWN. The urge to share / viz / post / tweet is pretty great. The information is juicy. Your viz is amazing. But it might also be dead wrong. Make sure it isn’t before you share it.
- Call out fake news when you see it. On social media, on twitter, wherever. Yes, your grandma will be annoyed that you’re raining on her partisan parade. Just also be nice to her. Yep, she got bamboozled. Next time it could be you.
- Support real, professional, thorough investigative journalism. IRE, the Knight Foundation and Pro Publica are good places to start.
In conclusion, it’s on all of us to be watchdogs of the information in our world. It’s on all of us to call out the bullshit that we come across. In our social media streams, on the TV programs and radio shows that we watch, when our leaders open their mouths and cite statistics, when a viz we see is using problematic data or coming to dubious conclusions. Do so tactfully.
It’s on all of us to make sure that what we say, create and share is factually correct and intellectually honest.
I really enjoyed this article. It is well written, informative and very valuable. In my parlance, I say that I treat all data as guilty until proven innocent. This is just my way of saying: “Fact check!” and “Check Data Quality!”.
One way that I have decided to get around the B.S. is to do direct testing myself. For example, I am doing a series of articles that compares Tableau to Power BI. When I read statements attributed to Microsoft that says: “A data engine Microsoft says is 10 to 100 times faster than Tableau’s”, I’m going to find out for myself if that is a true statement. Once I complete the testing, I will know the answer for myself. This is another way that I do my own version of fact-checking.
Thanks again for sharing this,
P.S. Click here to fact check my quote!
Awesome, thanks for the comment, Ken, and glad to know you’re a fellow fact checker! I agree with your sentiment – there’s no better way to verify a data point than to measure it yourself or run your own experiment. Be sure to let us know how it goes.