Data analysis tools have advanced to the point where seemingly anyone with basic computer skills can develop meaningful insights.  As with any tool, however, operator skill determines the ultimate outcome.  Anyone can create graphs with trend lines, but knowing how to graph data properly requires skill.

For example, the graphic below appeared in a recent blog post about the impact of Health Care Reform on President Obama’s popularity.  While the trend lines look impressive, this graph misleads the reader.

Four scatter graphs contrasting public opinion about Obama with public opinion about Health Care Reform

It is true that statisticians use scatter plots to show the relationship between two variables, but in this case a third variable plays an overriding role.  This third variable is Time.  Public opinions shift over time, depending on the headlines and the proposed changes in legislation.  A scatter plot cannot take this into consideration. A timeline chart such as the one below proves to be more informative: it shows trends over time and provides possible explanations for shifts in public opinion.

Annotated timeline chart showing public opinion over time

Think about the many graphics we consume on a daily basis, whether in business meetings or in the media. What do we really know about the skill and motivation of the author behind the graphic?  If a graphic supports an opinion we already hold, we may never question it – even if our opinion deserves questioning!

Herein lies the Catch-22: during times of change we have even less time than usual to deal with the intricacies of data analysis.  Yet, precisely at those times do we need to question our assumptions, adapt to new realities and update our opinions.  One way of solving this dilemma: delegate data analysis to people with the appropriate experience and skills.

By necessity, we always make choices about how to present data. After all, we *are* trying to make a point when we share information. But even if we do not intent to spin the message, we may be unable to see the whole story until someone else adds their insight. By making our data available for download, we can level the debating field somewhat and hopefully reach better informed conclusions.

Whether by accident or by design, one way to spin the message involves the use of data ranges. In the example below, we have divided US obesity rates into three different ranges. The first range uses intervals of 11, the second range uses intervals of 10 and the last range uses intervals of 5.

Look at the graphs about soda taxes in vending machines and see how each graph may lead to a different conclusion about obesity and soda taxes in vending machines. Then take a look at the graphs for the other taxes and notice how those graphs support similar conclusions regardless of the range size.

When deciding how to present information we have to balance “information overload” with the need to present important details. Which graph we choose ultimately depends on the point we are trying to make. Some might call that spin, others call it effective communication. If we are the audience, we need to be skeptical and ask questions.

Related Posts:

http://pharma-bi.com/2010/03/how-to-avoid-misleading-conclusions-explore-your-data/
http://pharma-bi.com/2010/03/tableau-public-interactive-obesity-data-on-the-web/

By selecting different obesity ranges, one can observe that soda taxes tend to be low in states with high obesity rates.  Just check out Mississippi, Alabama and South Carolina.  Low soda taxes also occur in states with low obesity rates, so it would be premature to conclude that soda taxes are a good way to reduce obesity. Another interesting observation about populations with the highest obesity rates: general food taxes are high while soda taxes are low. Is this encouraging soda purchases over food?

Of course there may be issues with the underlying data as I explain here.  I do not know this data well enough to draw definitive conclusions from it. Besides, this post is more about illustrating how we can move beyond static charts & graphs when discussing issues on the web. By the way, using the icons at the bottom of this viz, you can download this data and even change the cursor behavior so it will zoom in on a particular area on the map.

Here are a few tips for using Tableau Public to share your data:

• focus on one issue per viz
• decide how to guide visitors into exploring your data
• include only relevant data and create an extract
• determine how much display space you have
• first set the display size in your Tableau workbook – here are some tips
• then design your viz within that space
• beware of borders and shading – they may look very differently in a web browser
• preview your viz & make sure everything works as planned
• keep it simple and have fun!

If you are using a blog with a theme, be sure you know how much display space your theme allows.  If you use an HTML editor like the one in WordPress, make sure to paste Tableau’s HTML code into the HTML section of the editor, not the visual editor. I found it best to add the Tableau HTML at very end before publishing/updating the post.

Happy authoring and exploring.  Please shoot me a note with your comments.

Acknowledgment:

Although based on USDA data, the data set in the above viz comes from Tableau and can be found here: http://www.tableausoftware.com/public/viz-contest

Related Posts

Just because a graph looks pretty doesn’t mean it tells the truth. Here is an example.

Visual exploration tools make it easy to create graphs, but we still need to spend time exploring the data to make sure we are drawing meaningful conclusions. Here is an example.

Here is an example of how such an exploration may look: we are exploring data about obesity, soda consumption and sales taxes on soda.  We are told this data came from the US Department of Agriculture and a quick look reveals that we are looking at county level data.  As one might expect, a scatter plot reveals a strong relationship between rising soda consumption and increased obesity.

Adult obesity rates increase as soda consumption increases

Now we get to the real questions: do sales taxes on soda help with lowering obesity rates?  What relationship do we see between sales tax rates on soda and obesity? As luck would have it, the data we received also provides two measures about sales taxes for soda: one rate for vending machines and another rate for retail stores.

First we look at the relationship between soda taxes for retail stores versus obesity rates.  One might expect that taxes discourage soda consumption and, yes, there appears to be a small downward trend as tax rates increase.  Maybe soda taxes actually help with bringing down obesity?

Adult Obesity Rates, Retail Sales Tax Rates and Soda Consumption by US County

Now let’s take a look at sales taxes on soda coming from vending machines.  Interesting observation: diabetes rates seem to increase slightly as these tax rates increase.  Counter intuitive?  How do vending machine purchases differ from purchases in a retail store?  Are we observing a real relationship here, or is the data fooling us?

Adult Obesity Rates, Vending Machine Sales Tax Rates and Soda Consumption by US County

Before answering these questions, let’s take a closer look at all those data points on the y-axis.  Do they really indicate that these counties levy a 0% soda tax?  A quick inspection of the underlying data shows that, yes indeed, all records indicate a 0% tax rate.  Not a single “null” value among them.   However, without knowing how the data was processed, we cannot be sure that “zero” really means “no taxes” – it could also mean “no data.”

To explore further we start by placing the three graphs side by side.  This way we can see more easily what happens when we exclude “zeroes.”

Soft Drink Consumption, Obesity and Soda Sales Taxes

First we exclude “zeroes” for retail sales taxes. Then we’ll do the same with taxes levied on soda in vending machines.  The following graphs illustrate this.

Excluding Records with 0% Soda Sales Tax Rate (Retail). The center graph shows the relationship between the remaining retail records and obesity. The trend line still points downward.

Excluding Records with 0% Soda Sales Tax Rate (Vending). The right hand graph shows the relationship between the remaining vending machine records and obesity. We now see a downward trend as tax rates increase.

Wait a minute, though.  When we exclude “zeroes” from one set of taxes, all data points for “greater than 0% taxes” disappear from the other graph.  In other words, this data indicates that the two types of taxes are mutually exclusive!  Hmm, does this even make sense in real life?  Why would every US county tax soda either in retail stores or in vending machines but never in both?

Without further knowledge about this data we have to reframe our questions and conclusions:

• When soda taxes are levied, higher tax rates appear to go hand in hand with decreasing obesity rates
• We cannot draw any conclusions about the impact of “no sales taxes” versus “sales taxes”
• Before we continue with a detailed analysis, we probably need to ask questions about this data.  At first glance it makes little sense that counties levy soda taxes either on vending machines or on retail stores but never on both.  Then again, I’m not a tax expert.

Chances are that we will uncover other areas about which we need to ask questions.  Instead of taking the scattershot approach to learning about this data, data exploration helps us to develop very specific questions to ask.  With specific questions, we stand a better chance of finding the right subject matter experts to consult.

This was a quick example for exploring data about which we knew nothing when we started.  To gain new insights, we sometimes need to apply this “beginners mind” approach even to data about which we already know a lot.  After all, errors can happen, collection and processing systems can change without our knowledge and sometimes we find nuggets that were hidden until we started looking for them.  One final thought: the next time your boss or client asks to hurry up with the analysis, ask these two questions:

• What are the consequences of making poor decisions because we hurried too quickly through the data exploration?
• Do we need to go for more accuracy or is a ballpark analysis good enough at this time?

Let’s look at an example.  Below are two graphs based on exactly the same underlying data – but why do the colors look different?   Each graph appears to show a quadrant analysis that compares two web sites based on their search engine rank and trust.

Graphs comparing two web sites based on their SEO rank and trust

The difference lies in the way each graph is generated: the first graph really represents a set of eight data points, while the second graph represents two sets of four data points each – a subtle, but important distinction.  The second graph shows the quadrants for each individual site using a separate scale for each site.  This allows us to compare each site quadrant by quadrant without having to worry about one site having vastly more links than the other.  In other words, we can answer questions like: which site did a better job of getting high quality links vs. low quality links?

The first graph combines the data for both sites and plots each quadrant on a scale for the combined data.   If one site has many more links than the other site, it will skew the scale toward the higher linked site.  In essence, we are comparing all eight quadrants against each other as opposed to comparing how each site performed on a particular quadrant.

The second graph therefore is the “correct” quadrant analysis if we want to compare each site quadrant by quadrant.  But why even talk about the first graph?

That’s because in Tableau it may be tempting to generate the first graph to save time – especially when one is new to Tableau.  We only have to drag the “Site Name” dimension onto the column shelf and, voila, we can show both sites next to each other.  The problem is this: the shading is now determined based on all 8 data points together – rather than using a set of 4 data points for each individual site.  This becomes obvious once we add color scales to the graphs:

Graphs comparing two web sites based on their SEO rank & trust - includes color scales

The first graph really does not compare the two sites to each other. Instead it takes a look at all the links for both sites combined and creates 8 data points from all those links.  The second graph uses data from one site at a time.   A small – but critical – difference.

While this example may seem trivial, it actually has deep implications when we deal with more complex visualizations.  For example, when we use bins or when we filter records based on certain values, we may add misleading reference lines or create inaccurate charts – but that’s a topic for another day.

For example, this heat map makes it obvious that Session C was the least popular event in this conference – but notice also that the range of scores is fairly close together.  In this example, attendees used a 5 point scale, with 1 being the least favorable score and 5 being the most favorable score.   Seeing that the lowest average score was above 4 tells us that, overall, attendees were quite happy with this conference.

A heatmap showing how each individual session was rated along multiple criteria -- click on the picture to enlarge it

Of course we also want to understand what worked well and what didn’t work so well.  Let us take a more detailed look at Session C.

Click the picture to enlarge it

Very quickly we can tell that the audience was critical of the session content and how it was presented rather than the speaker’s knowledge about the subject.  More than 90% felt that the speaker had an adequate background to present on this topic.  But less than 75% were happy with the way the information was presented and how it related to their job. A few people awarded low scores of 2 and 1, but it is encouraging that these low scores came from fewer than 10% of the audience members.

Just to provide a contrast, let’s also take a look at Session B, clearly the favorite of this event.

Individual Scores for Session B

There is no doubt that the audience rated this Session very highly across the board: 74% or more audience members awarded it the highest score across all measures.  Notice that this session, too, received a handful of low scores.

As we can see here, sometimes we only need a few pictures to gain useful insights.   No doubt, if this were a more involved customer survey, we would need more and very likely different graphs to discern the finer points of attitudes and perceptions — especially when such information is tracked over time.  But pictures like these are a good start to find out where to focus such additional efforts.

While point-and-click is much easier than writing code, it still requires computer users to become familiar with what the computer can do and how to accomplish various tasks.  It also requires standards about where to click and what should happen when certain actions are taken.  All this knowledge and these standards had to develop over time and often through trial and error.

In some ways we have reached a similar stage for data analysis: visual analytics provides a new language through which non-analysts can explore and answer business questions.  It frees the non-technical user from the analytic equivalent of writing code, that is, it frees them from the need to learn how to create graphics that – until now – required significant technical knowledge to generate.

As with any new technology, some mayhem ensues: we have to gain experience and learn through practice.  We need to become fluent in the appropriate use of less familiar – yet oddly intuitive – graphing techniques like sparklines, heatmaps or small multiples.  In short, we must develop a visual vocabulary beyond the bar charts and line graphs we know from Excel or PowerPoint.

And this brings me back to the Xerox-GUI-Macintosh comment from the beginning: as in those early days of learning the language of icons and point-and-click, we have now reached the point where more powerful ways of encoding data in a visual format is available to the lay person.  Just as with the graphical user interface, we will some day look back and say “I can’t imagine a world without seeing data in pictures.”  For at least a little while, those of us in the analyst professions need to act as interpreters and guides to those who are learning this new language.   Sooner rather than later we will all get there.