The Trap of Ad Hoc Visualizations. Why Asking the Right Questions Matters

Listen to AI Narration

0:00

/6:08

1×

We live in a world where data is omnipresent. With a plethora of dashboards, charts, and infographics at our fingertips, it's easy to assume that we're more informed than ever before. However, this influx of data visualizations can be a double-edged sword. On the one hand, they can offer quick insights and simplify complex information; on the other, they can mislead, confuse, and result in misinformation. This is especially true when the visualizations are encountered in an ad hoc manner, like on social media, news articles, or blogs. These are more prone to causing misinformation compared to when we ask specific analytical questions and seek visualizations tailor-made for them.

The Pitfalls of Ad Hoc Visualizations

Lack of Context

The greatest disadvantage of ad hoc visualizations is that they are often devoid of context. Unlike academic papers or professional reports, or when visualizations are created specifically based off your analytical questions, these visuals don't always come with accompanying text that provides background information, data source, or methodology. This lack of context can lead viewers to make false assumptions and draw incorrect conclusions.

Sensationalism Over Accuracy

Many ad hoc visualizations are created with the primary goal of grabbing attention. Whether it's a brightly colored pie chart on a blog or a flashy infographic on social media, the priority may be more on aesthetics and impact than on accurate representation of data. As a result, data can be manipulated to fit a narrative rather than to reveal truth.

Incomplete Data

Because these ad hoc visuals are often aiming for simplicity and immediate impact, they may represent an incomplete data set. Cherry-picking data to fit a specific narrative can leave viewers with a skewed perspective, making them prone to misinformation.

The Power of Purposeful Inquiry

When we start with a well-defined analytic question, the path to proper data visualization becomes more focused and reliable for several reasons:

Tailored Data Selection

Beginning with a specific question allows us to collect data that directly addresses that inquiry. This reduces the chances of data omission or manipulation as we are collecting data with a specific purpose in mind.

Contextual Awareness

When visualizations are generated in response to a specific question, they are often accompanied by textual explanations, sources, and methodologies. This added context helps in the accurate interpretation of the visual data and minimizes the chances of misinterpretation.

Depth Over Breadth

Asking a specific question often results in a more in-depth exploration of a particular issue, allowing for nuanced insights. Tailor-made visualizations can include multiple data points, trends over time, or comparative data that provide a fuller, more accurate picture.

The Case for Data Literacy

Regardless of whether a visualization is encountered ad hoc or is custom-made, it's essential to approach it with a critical mind. Always ask the following questions:

• What is the data source?
• Is the scale distorted?
• Are there any visible biases?
• What is omitted?
• Does the visualization actually answer the question it purports to address?

Remember, while ad hoc visualizations offer quick, easily digestible insights, they are fraught with pitfalls that can lead to misinformation. In contrast, starting with a specific analytical question and generating tailor-made visualizations can result in more accurate, reliable, and insightful data interpretation. Thus, the key to navigating this world rich in visual data lies not just in the visualizations themselves but also in the questions we ask before creating or interpreting them.

Example "Explosive Growth in Screen Time. An Epidemic of Unproductivity"

You stumble upon a bar graph on a productivity-focused blog that shows "Average Screen Time Hours Per Day" for the years 2018 through 2023. The bar for 2023 is almost double the size of the bar for 2018. The blog headline screams, "Waste of Time: Screen Time Doubles, Productivity Plummets!"

You start to feel guilty about the time you spend in front of screens. The visualization makes it appear as if everyone, perhaps including yourself, is becoming less productive because of excessive screen time. You start considering extreme measures, such as digital detox plans and very restrictive time management techniques.

The Misleading Aspects

• Broad Data Grouping. The term "screen time" is an aggregate of various activities: work, socializing, entertainment, education, and so forth. It's not broken down to indicate how much of this screen time is unproductive versus productive (e.g., work-related, educational).
• Lack of Context. There's no information provided about other lifestyle changes that may have occurred in the same time frame. For example, the transition to remote work and online education due to the COVID-19 pandemic might account for a significant increase in necessary, productive screen time.
• Definition of "Unproductivity". The blog assumes all screen time is linked to a decrease in productivity without defining what "productivity" means. In reality, productivity is a complex metric influenced by a variety of factors, including but not limited to screen time.

The Ad Hoc Problem

The bar graph you encountered was designed to capture attention and provoke a reaction. However, it oversimplifies a complex issue by lumping all screen time into one category and directly linking it to "unproductivity," without offering nuanced information or context.

A Tailored Visualization

On the other hand, if you start with the specific problem and question you want answered, results are typically much more useful.  Assume  you were looking to understand the impact of screen time on productivity, a more useful chart might include the levels of productivity that you can compare to the average daily screen time, as well as any annotations that mention major events that may have influenced the data:

This would offer a clearer picture of how screen time affects productivity specifically.. In this example, productivity is a made-up metric for demonstration purposes. Let's go one step further and assume the employer sees the spike in screen time in 2020 and wants to understand more context. A more useful chart to answer that specific question might include:

• Different types of screen time (work, social media, video streaming, etc...)
• Contextual factors like changes in work environments, tools, or policies
• An abstract that includes the source of the data

This would offer a clearer picture of the different types of screen time. You can see in 2020, when COVID-19 started, work-related and educational screen time dramatically increased.

Click here to download a best practices guide to help you apply critical thinking to evaluate and interpret ad hoc visualizations.