Pharma BI » Visualization

A Short Exploration of Physician Practice EHR/EMR Adoption

Christine Muser — Fri, 09 Dec 2011 07:38:33 +0000

According to recent estimates by the CDC, the use of electronic health and medical records in physician offices continues to climb. According to data from the November 2011 NCHS Data Brief, around 57 percent of office based physicians report using some kind of electronic health/medical record system and around 34 percent report having a system that meets basic performance criteria. However, EHR/EMR usage varies widely by state, as can be seen in the graph below.

The graph shows states in descending order based on the usage of any type EHR/EMR system. The series of dots illustrates usage of any system, while the bars illustrate usage of systems that meet basic criteria. Median percentages are illustrated by dotted lines. Colored shading indicates the percentage range into which most (68%) states fall. The shaded area on the left applies to usage of basic systems, the shaded area to the right applies to usage of any EHR/EMR system.

As we can see from the jagged outline of the blue bars, adoption of systems that meet basic performance criteria is not necessarily consistent with overall usage of electronic record systems. For example, Utah ranks number 2 in terms of overall EHR/EMR usage, but four states rank higher than Utah in terms of systems with basic functionality. These four states are Minnesota, Wisconsin, Washington and Oregon.

North Dakota, Utah, Minnesota, Wisconsin, Washington, Oregon and Hawaii not only lead the nation in terms of overall EHR/EMR usage, but also in terms of having basic systems in place. Looking toward the lower end of the list, we can see that states like Mississippi, South Carolina, Nevada, New Jersey and Louisiana have the lowest percentages of systems that meet basic criteria.

click on the graph to enlarge it

To look up specific EHR/EMR adoption values for each state or to download the data, go to this interactive Tableau Workbook.

Given the current state of the debate around health care reform, one might be tempted to ask whether the political climate at the state level plays a role. Using information about “Red States and Blue States” from Wikipedia, we can assign party affiliations to states based on results from the past five presidential elections (1992 to 2008). At first glance, it appears that democratic states tend to be further along in terms of EHR/EMR system usage. Their median percentages for overall usage (61%) and basic system usage (36%) are higher than those of republican states (57% and 31% respectively).

click on the graph to enlarge it

However, reducing this question to simple partisan politics ignores the many factors influencing the adoption of complex systems. Issues like perceptions and attitudes, computer experience, workflow impact and concerns about patient-doctor relationships play a much bigger role in the day to day decisions doctors make. These issues are beyond the scope of this writing, though a very small sampling of relevant academic research appears at the end of this post.

Even without delving into those issues, we can get a sense that there is more at play here by looking at socio-economic data from the US Census Bureau. The graph below shows the usage of any type of EHR/EMR system, along with census data about poverty, annual pay, higher education, age, and number of physicians per capita.

click on the graph to enlarge it

Using this graph, we notice that our breakout of Democratic and Republican states shows some significant differences. The thick grey lines indicate the median value for each variable within each party category, while the grey shading shows the range into which the majority (68%) of states in each category fall. Based on this data, residents in democratic states tend to be

wealthier (lower percentage of people living below the poverty line)
earn higher incomes (higher average annual pay)
more educated (higher percentage of residents with a bachelor degree or higher)
able to choose among more doctors (higher rate of doctors per capita)

Since we are dealing with state level data we cannot say how much each of these factors influence EHR/EMR system adoption (see scatter plots at the end), except to say that higher levels of poverty appear to be linked to lower levels of EHR/EMR system usage – apparently more so if the state happens to lean republican.

Unfortunately this exploration suffers from the fact that we are dealing with high level data which hinders our ability to match up relevant details. None the less, it provides some hints that socio-economic issues play at least an indirect role in the adoption and use of EHR/EMR systems.

click on the graph to enlarge it

Resources:

Explore the data details behind the graphs in this post via this interactive Tableau Workbook.

Electronic Health Record Systems and Intent to Apply for Meaningful Use Incentives Among Office-based Physician Practices: United States, 2001–2011

NCHS Data Brief, Number 79, November 2011

http://www.cdc.gov/nchs/data/databriefs/db79.htm

Definitions used in this report (direct quote from the NCHS Data Brief):

“Physician office: A place where nonfederally employed physicians provide direct patient care in the 50 states and the District of Columbia; excludes radiologists, anesthesiologists, and pathologists.

Any EMR/EHR system: Obtained from “yes” responses to the question, “Does this practice use electronic medical records or electronic health records (not including billing records)?”

Basic EMR/EHR system: A system that has of all of the following functionalities: patient history and demographics, patient problem list, physician clinical notes, comprehensive list of patient’s medications and allergies, computerized orders for prescriptions, and ability to view laboratory and imaging results electronically (4). Having a comprehensive list of patient’s medications and allergies was asked as two separate questions in 2010 (one about medications and the other about allergies); the questions were collapsed into one question in 2011 (5).”

The 2012 Statistical Abstract: State Rankings

U.S. Census Bureau

http://www.census.gov/compendia/statab/rankings.html

Red states and blue states

http://en.wikipedia.org/wiki/Red_states_and_blue_states

Direct quote from the wiki article:

“Current classification

The following classification of red and blue states (as well as purple/battleground states) was determined by compiling the average margins of victory in the five presidential elections between 1992 and 2008. Three of these past elections were won by Democrats, Bill Clinton in 1992 and 1996, and Barack Obama in 2008, while two were won by Republican George W. Bush in 2000 and 2004.”

A Framework for Predicting EHR Adoption Attitudes: A Physician Survey

by Mary E. Morton, PhD, RHIA, and Susan Wiedenbeck, PhD

http://perspectives.ahima.org/index.php?option=com_content&view=article&id=162&Itemid=56

The influence of payer mix on electronic health record adoption by physicians

Menachemi, Nir; Matthews, Michael C.; Ford, Eric W.; Brooks, Robert G.

http://journals.lww.com/hcmrjournal/Abstract/2007/04000/The_influence_of_payer_mix_on_electronic_health.4.aspx

Predicting the Adoption of Electronic Health Records by Physicians: When Will Health Care be Paperless?

Eric W. Ford, PhD, MPH, Nir Menachemi, PhD, MPH, and M. Thad Phillips, MSHA, MSHI, MBA

http://www.ncbi.nlm.nih.gov/pmc/articles/PMC1380189/

Bridging the Dashboard Communication Gap

Christine Muser — Thu, 08 Sep 2011 06:41:16 +0000

The term “dashboard” provides a convenient metaphor because everyone has at least some idea of what a dashboard looks like – and therein lies the problem: our own idea of a dashboard may differ wildly from someone else’s idea of a dashboard. When people talk about dashboards, there may be a huge communications gap and it pays to build a bridge across that gap before taking any action toward developing a dashboard.

There’s a big difference between the dashboard of a car and that of a passenger jet! For one thing, understanding a car dashboard requires significantly less training and experience than a cockpit dashboard. Of course each is designed to meet different needs: a pilot has to worry about many more things than a car driver when ferrying passengers safely from Point A to Point B.

When developing information dashboards for a business we also need to keep user needs in mind: a VP of Marketing will need a much more high-level overview than a Product Director, who in turn has quite different requirements from a sales rep preparing an action plan. In practical terms this means that we first have to figure out who will use the dashboard and how.

Image of a Jet Cockpit

Image credit: http://flysusan.com/cockpits.htm

A dashboard is only useful when it can become an integral part of decision making. Some decisions need to be made on a daily basis, such as which prospects to call or how to follow-up with a client. Other decisions take more time. Those decisions tend to be more organizational and strategic in nature. They require input from many people and data sources, they require observation and a longer term perspective.

The implication here is that a dashboard needs to be capable of providing information at the speed with which decisions are made. A dashboard also needs to provide the appropriate amount of summary and detail. Before we can build anything, we need to define what information is actionable and how soon it needs to be available.

From a technical standpoint, this initial exploration provides the framework for determining how the dashboard should be build. The following questions are just a starting point:

What information is required?
How will the information be used?
What kinds of summaries and calculations make the most sense?
How much detail should be included?
How can we best present the information?
How often does data need to be refreshed?
Which databases and information sources are necessary?
Which software/hardware best meets our needs?

The answers to these questions always involve tradeoffs. Which tradeoffs make sense depends on the impact the dashboard will have and whether we can demonstrate a positive ROI. At some point, the dashboard has to help improve the bottom line.

Drawing a direct line between dashboard and bottom line dollars can be complicated. Dashboard benefits tend to be intangible. To measure ROI, we have to think about how the dashboard enables better decisions, how it helps users focus on profitable actions and whether it helps to save time or other resources. Often we need to offset these intangible benefits against very real budgets and real money that needs to be spent on development, training, software, hardware, and so on.

As we can see, if we are serious about building a dashboard, we also have to be serious about spending some time upfront to map out a plan for getting from the Basic Idea to an Actual Dashboard. To get there, everyone involved needs to develop a common language, that is, a set of common definitions and goals. For instance, if our dashboard is supposed to track progress following a product launch we need a common understanding of what market penetration means. Does it mean “number of actual customers vs. potential customers” or “number of actual customers vs. number of targets” or “number of customers who bought at least X number of widgets.” How we define our key metrics has a direct impact on the usefulness of our dashboard and on the effort required to build it.

Developing common definitions and goals is a key step toward building a bridge across the communication gap. Just because we are using terminology that seems to be well defined doesn’t mean we really are talking about the same thing. For example, we might be talking about lighthouses and we might even have the same general idea of what a light house is and how a typical lighthouse looks. But when it comes to actually building the lighthouse we need something more concrete.

What’s a lighthouse?

The Idea	An Actual Lighthouse	Another Actual Lighthouse

When discussing the concept of a lighthouse, usually this type of image comes to mind.	Most of us would expect an actual lighthouse to look something like this. It meets the typical expectations for a lighthouse.	Sometimes a lighthouse needs to address special situations: when a client needs a special lighthouse and the developer thinks of building a typical lighthouse, the project is in trouble.

Image credits (L to R):

http://www.freeclipartnow.com/buildings/lighthouses/

http://www.destination360.com/north-america/us/maine/images/s/maine-lighthouses.jpg

http://www.longislandlighthouses.com/lv112.htm

Double Take on Dual Axis Graphs: Part III

Christine Muser — Fri, 18 Mar 2011 18:11:39 +0000

This quick follow-up on dual axis graphs shows another take on their potential use. The first suggestion comes from Naomi Robbins in her book “Creating More Effective Graphs.” ¹ She suggests that dual axis graphs may be useful to represent data in different – but equivalent – measurement units such as Centigrade (Celsius) and Fahrenheit. Some of us can understand temperature better when it is expressed in Fahrenheit, while others relate better to temperatures on a Celsius scale. This example works because we are graphing the same thing, we’re just expressing it in different units and provide the two scales as an easy reference.

When doing this, however, we have to ensure that both scales are synched up as illustrated below.

The other example comes from Dona Wong’s recent Wall Street Journal. Guide to Information Graphics. ² In it she suggests using dual axis graphs to “ … help show how two directly related series move together.” The picture below – courtesy of Kaiser Fung’s Junk Charts blog – shows the example from her book. ³

I am somewhat ambivalent about this chart since we have to be careful in selecting each scale in order avoid distorting the data (note that the Market Share scale starts at 30% and that the increments for both scales line up).

When done properly, both examples help us when we need to make a point with a single picture: we can save space and keep two related pieces of information within easy view. This works best when illustrating a specific issue and when we can use a static picture in a newspaper, printed report or a presentation. We’re asking for trouble when using this approach in a dynamic environment, like a dashboard, where scales need to adapt to the latest data.

Acknowledgements:

1. Naomi B. Robbins, Creating More Effective Graphs (New Jersey: John Wiley & Sons, Inc., 2005), page 262

2. Dona M. Wong, The Wall Street Journal. Guide to Information Graphics (New York: W.W. Norton & Company, Inc.), page 59

3. Reading: WSJ Guide to Information Graphics, Posted on Mar 06, 2010 at 02:59 PM, Junk Charts: Infographics

Dual Axis Graphs: Are They Useful?

Christine Muser — Fri, 04 Mar 2011 03:11:31 +0000

I have to admit that even after reading Stephen Few’s article on dual axis graphs, I am not quite ready to rule them out entirely. As is so often the case with data visualization, what we use depends on what we’re trying to do. I agree with Joe Mako and Stephen Few that, when used as a communication tool, dual axis graphs often confuse rather than communicate. Therefore it makes little sense to use them in dashboards and other situations where we need to communicate information quickly and at a glance.

But I do find dual axis graphs useful as a shortcut when exploring data – precisely because I can more clearly see the shape of the two curves in relation to each other. Below is an example that looks at Dollars and Units. When graphing them separately, the lines look somewhat similar and it takes a second look to notice that Units are declining while Dollars are increasing.

Two graphs, one showing Dollars the other showing Units

In the dual axis graph below, I got that message without needing a second look:

Dual axis graph showing Dollars and Units

When exploring data with visual analytics tools like Tableau, a dual axis graph can help me compare metrics quickly to see whether something jumps out. Are the curves moving in synch with each other? Are they trending in a similar direction? Do the curves indicate anything useful from a business point of view? For instance, in this example it looks like our margin might be improving. If the curves moved in the opposite direction, we may need to ask whether we’re discounting too much.

As Joe Mako and Stephen Few point out, in a dual axis graph we cannot compare the magnitude of change, but we can get a general feel for the direction and whether there are further questions we need to ask. In this example, a simple ratio calculation of Dollars per Unit helps to confirm whether we’re onto something. When we report results from our analysis, a graph of this ratio will do a better job showing the relationship between Dollars and Units than the dual axis graph shown above.

Line graph showing Dollars per Unit

Mixed Metaphor: Line Graph + Bar Chart

Christine Muser — Thu, 03 Mar 2011 08:46:44 +0000

Many charting tools allow us to combine bar charts and line graphs in the same graphic – but should we? This question came up when I did a double take the other day while reading a market research report. In order to focus on function rather than content, I have re-created a similar graph below:

Click the image to enlarge it

If the author’s intent was to induce a double take, it worked for me – in a confusing sort of way:

It caught my eye that the line graph is trending down while the bars are going up
But it took a few moments to realize what the line graph actually represents
And it took a few more moments to realize the message behind the graph: Widget volume is going up, but the rate of growth has turned from “healthy” to “anemic”

That was way too much work to get the gist of this graph! Why was this so difficult to get?

Instead of providing a scale, the individual data points are labeled – this clutters up the picture and distracts the eye. Not only that, it is difficult to get a ballpark idea for the maximum values unless one scans each data point – a scale for each data series would have provided that contextual information much more quickly.

When I re-created the graph I realized something else: in order to display the line graph within the colored bars, I had to set the maximum value for the percentage scale higher than what would be expected: instead of setting the top percentage value to 10% I had to use 15% to push the line graph down far enough to completely overlap with the bars.

Also, fewer words in the legend labels would have been better (the original had even more text!) – and it would have helped to spell out the word “percent” instead of hiding the percent symbol amongst all that text. Better yet: be consistent and stick with the same nomenclature: the chart title says “Volume and Growth” so the series labels should say the same.

Maybe this is a nit pick, but I prefer legends at the bottom or the side of a graph. The top should be reserved for the title – IMHO.

Finally, the question of whether we should mix bar charts with line graphs: I don’t think so. The bars overpower the picture and clutter things up. Call me a minimalist, but it’s much easier to see how volume and rate of growth relate to each other when we draw two clean lines like so:

Click the image to enlarge it

With less clutter, the eye can very quickly take in the overall picture and focus on what matters:

Number of Widgets is going up,
Percent Growth is going down.

If the data values really matter to us, we can look them up in the table below the graph.

How Three Graphs Can Tell Us Where to Focus our Attention

Christine Muser — Tue, 04 May 2010 22:22:00 +0000

A while back, SEO guru Glenn Crocker and I were talking about how visual analytics can help with search engine optimization. Getting useful SEO information usually requires crunching data for thousands of links, so it’s quite useful to have something that takes us from a quick overview to the interesting details.

To illustrate how visual analytics can help with this, we decided to look at the web sites for two of our favorite charities: Feeding America, formerly known as Second Harvest, and an affiliated organization called the Harvesters Community Food Network in Kansas City.

Using data from SEOmoz, we combined the links for both sites into one database and compared their link performance. A quick glance at the graph below tells us that Feeding America has the better SEO profile: more links in general and also more high quality links. This shouldn’t be too surprising since Feeding America is a national organization, while Harvesters serves the greater Kansas City area. But comparing the two sites provided some interesting data for us to review.

Two things are worth noting here:

Some domains linking to Feeding America are very highly ranked – up to a rank of 10, while the domains linked to the Harvester’s site top out at 8. This is also reflected in the overall average for the domain ranks – 5.5 and 4.5 respectively.
When looking at the color near the peak of each curve, we notice that the rank for domains with the most links to Feeding America is higher than for highly linked domains at Harvesters.

The quality of the links is indicated by the color (green is better) and the Domain mozRank (10 is best). The number of links is indicated by the height of each curve.

Next we tried to figure out which sites contributed the most to the SEO performance. Below I have highlighted a few of them. These scatter plots show domains that have 5 or more links across both sites. In the interactive version we can hover over data points to see more details about each domain. I am not sure that the folks at SEOmoz would be happy about me uploading their data, so I am showing just this picture to get the general idea across.

Depending on time and interest, we can perform even more fine grained analysis. For example, just because a highly ranked domain sends us links doesn’t mean the links rank equally well. The graph below shows that only two highly ranked domains send links of a similarly high rank.

Let’s pretend we are Harvesters and we want to reach an audience beyond the Kansas City area. By looking at these three graphs, we now know that

we need more and better links
we need web content that attracts more highly trusted domains
we have plenty of links from the local community (many of the Top 15 Domains are from Kansas City based organizations). Maybe we should broaden our horizons and reach out to the owners of more nationally focused, highly trusted domains.

P.S.: Just in case anyone is curious: at least in the Kansas City area, Harvesters does rank at top for the search term “Harvesters”

Don’t be Fooled by Poor Analytics

Christine Muser — Mon, 19 Apr 2010 20:02:46 +0000

Today’s shrinking resources may tempt us into rushing things along, yet we need to be careful when relying on graphics to make decisions. Good graphics make their point more quickly than a wall of text. On the other hand, poor graphics easily create the wrong impression. Distinguishing between the two is not as easy as one might think.

Data analysis tools have advanced to the point where seemingly anyone with basic computer skills can develop meaningful insights. As with any tool, however, operator skill determines the ultimate outcome. Anyone can create graphs with trend lines, but knowing how to graph data properly requires skill.

For example, the graphic below appeared in a recent blog post about the impact of Health Care Reform on President Obama’s popularity. While the trend lines look impressive, this graph misleads the reader.

Four scatter graphs contrasting public opinion about Obama with public opinion about Health Care Reform

It is true that statisticians use scatter plots to show the relationship between two variables, but in this case a third variable plays an overriding role. This third variable is Time. Public opinions shift over time, depending on the headlines and the proposed changes in legislation. A scatter plot cannot take this into consideration. A timeline chart such as the one below proves to be more informative: it shows trends over time and provides possible explanations for shifts in public opinion.

Annotated timeline chart showing public opinion over time

Think about the many graphics we consume on a daily basis, whether in business meetings or in the media. What do we really know about the skill and motivation of the author behind the graphic? If a graphic supports an opinion we already hold, we may never question it – even if our opinion deserves questioning!

Herein lies the Catch-22: during times of change we have even less time than usual to deal with the intricacies of data analysis. Yet, precisely at those times do we need to question our assumptions, adapt to new realities and update our opinions. One way of solving this dilemma: delegate data analysis to people with the appropriate experience and skills.

Beware of Creative Analytics: Lies, Damned Lies and Statistics

Christine Muser — Fri, 19 Mar 2010 15:41:05 +0000

How often have we seen a graph in an opinion piece without knowing how it was created, but somewhere in the back of our mind we suspected that it was tweaked somehow to make a point? How can we ferret out “creative analytics” from the true story? Remember Mark Twain’s famous quote about ” … lies, damned lies and statistics.” It is much more difficult to identify “lies” when we cannot inspect the data behind them.

By necessity, we always make choices about how to present data. After all, we *are* trying to make a point when we share information. But even if we do not intent to spin the message, we may be unable to see the whole story until someone else adds their insight. By making our data available for download, we can level the debating field somewhat and hopefully reach better informed conclusions.

Whether by accident or by design, one way to spin the message involves the use of data ranges. In the example below, we have divided US obesity rates into three different ranges. The first range uses intervals of 11, the second range uses intervals of 10 and the last range uses intervals of 5.

Look at the graphs about soda taxes in vending machines and see how each graph may lead to a different conclusion about obesity and soda taxes in vending machines. Then take a look at the graphs for the other taxes and notice how those graphs support similar conclusions regardless of the range size.

When deciding how to present information we have to balance “information overload” with the need to present important details. Which graph we choose ultimately depends on the point we are trying to make. Some might call that spin, others call it effective communication. If we are the audience, we need to be skeptical and ask questions.

http://pharma-bi.com/2010/03/how-to-avoid-misleading-conclusions-explore-your-data/
http://pharma-bi.com/2010/03/tableau-public-interactive-obesity-data-on-the-web/

Tableau Public: Interactive Obesity Data On The Web

Christine Muser — Fri, 19 Mar 2010 03:26:19 +0000

The viz below uses Tableau Public and allows visitors to explore differences in food consumption and food/soda taxes based on obesity rates in the US. Notice the dramatic differences in soft drink consumption among populations with different obesity rates. At first glance, it appears that cutting back on sodas is our best bet for reducing waistlines!

By selecting different obesity ranges, one can observe that soda taxes tend to be low in states with high obesity rates. Just check out Mississippi, Alabama and South Carolina. Low soda taxes also occur in states with low obesity rates, so it would be premature to conclude that soda taxes are a good way to reduce obesity. Another interesting observation about populations with the highest obesity rates: general food taxes are high while soda taxes are low. Is this encouraging soda purchases over food?

Of course there may be issues with the underlying data as I explain here. I do not know this data well enough to draw definitive conclusions from it. Besides, this post is more about illustrating how we can move beyond static charts & graphs when discussing issues on the web. By the way, using the icons at the bottom of this viz, you can download this data and even change the cursor behavior so it will zoom in on a particular area on the map.

Here are a few tips for using Tableau Public to share your data:

focus on one issue per viz
decide how to guide visitors into exploring your data
include only relevant data and create an extract
determine how much display space you have
first set the display size in your Tableau workbook – here are some tips
then design your viz within that space
beware of borders and shading – they may look very differently in a web browser
preview your viz & make sure everything works as planned
keep it simple and have fun!

If you are using a blog with a theme, be sure you know how much display space your theme allows. If you use an HTML editor like the one in WordPress, make sure to paste Tableau’s HTML code into the HTML section of the editor, not the visual editor. I found it best to add the Tableau HTML at very end before publishing/updating the post.

Happy authoring and exploring. Please shoot me a note with your comments.

Acknowledgment:

Although based on USDA data, the data set in the above viz comes from Tableau and can be found here: http://www.tableausoftware.com/public/viz-contest

Related Posts

Just because a graph looks pretty doesn’t mean it tells the truth. Here is an example.

Visual exploration tools make it easy to create graphs, but we still need to spend time exploring the data to make sure we are drawing meaningful conclusions. Here is an example.

How to Avoid Misleading Conclusions: Explore Your Data

Christine Muser — Wed, 17 Mar 2010 07:29:39 +0000

Often we have to work with data without knowing all the details of how it was collected and processed. In those situations we first need to determine what information the data contains and what it can and cannot tell us. We need to ask questions of the data and determine whether it makes sense, given what we already know. To hone in on the time saving questions it helps to be a subject matter expert. But even if we are unfamiliar with the subject area, we can start by inspecting the different pieces of data to see how everything fits together. Visual analysis tools like Tableau software make that job much easier than it used to be.

Here is an example of how such an exploration may look: we are exploring data about obesity, soda consumption and sales taxes on soda. We are told this data came from the US Department of Agriculture and a quick look reveals that we are looking at county level data. As one might expect, a scatter plot reveals a strong relationship between rising soda consumption and increased obesity.

Adult obesity rates increase as soda consumption increases

Now we get to the real questions: do sales taxes on soda help with lowering obesity rates? What relationship do we see between sales tax rates on soda and obesity? As luck would have it, the data we received also provides two measures about sales taxes for soda: one rate for vending machines and another rate for retail stores.

First we look at the relationship between soda taxes for retail stores versus obesity rates. One might expect that taxes discourage soda consumption and, yes, there appears to be a small downward trend as tax rates increase. Maybe soda taxes actually help with bringing down obesity?

Adult Obesity Rates, Retail Sales Tax Rates and Soda Consumption by US County

Now let’s take a look at sales taxes on soda coming from vending machines. Interesting observation: diabetes rates seem to increase slightly as these tax rates increase. Counter intuitive? How do vending machine purchases differ from purchases in a retail store? Are we observing a real relationship here, or is the data fooling us?

Adult Obesity Rates, Vending Machine Sales Tax Rates and Soda Consumption by US County

Before answering these questions, let’s take a closer look at all those data points on the y-axis. Do they really indicate that these counties levy a 0% soda tax? A quick inspection of the underlying data shows that, yes indeed, all records indicate a 0% tax rate. Not a single “null” value among them. However, without knowing how the data was processed, we cannot be sure that “zero” really means “no taxes” – it could also mean “no data.”

To explore further we start by placing the three graphs side by side. This way we can see more easily what happens when we exclude “zeroes.”

Soft Drink Consumption, Obesity and Soda Sales Taxes

First we exclude “zeroes” for retail sales taxes. Then we’ll do the same with taxes levied on soda in vending machines. The following graphs illustrate this.

Excluding Records with 0% Soda Sales Tax Rate (Retail). The center graph shows the relationship between the remaining retail records and obesity. The trend line still points downward.

Excluding Records with 0% Soda Sales Tax Rate (Vending). The right hand graph shows the relationship between the remaining vending machine records and obesity. We now see a downward trend as tax rates increase.

Wait a minute, though. When we exclude “zeroes” from one set of taxes, all data points for “greater than 0% taxes” disappear from the other graph. In other words, this data indicates that the two types of taxes are mutually exclusive! Hmm, does this even make sense in real life? Why would every US county tax soda either in retail stores or in vending machines but never in both?

Without further knowledge about this data we have to reframe our questions and conclusions:

When soda taxes are levied, higher tax rates appear to go hand in hand with decreasing obesity rates
We cannot draw any conclusions about the impact of “no sales taxes” versus “sales taxes”
Before we continue with a detailed analysis, we probably need to ask questions about this data. At first glance it makes little sense that counties levy soda taxes either on vending machines or on retail stores but never on both. Then again, I’m not a tax expert.

Chances are that we will uncover other areas about which we need to ask questions. Instead of taking the scattershot approach to learning about this data, data exploration helps us to develop very specific questions to ask. With specific questions, we stand a better chance of finding the right subject matter experts to consult.

This was a quick example for exploring data about which we knew nothing when we started. To gain new insights, we sometimes need to apply this “beginners mind” approach even to data about which we already know a lot. After all, errors can happen, collection and processing systems can change without our knowledge and sometimes we find nuggets that were hidden until we started looking for them. One final thought: the next time your boss or client asks to hurry up with the analysis, ask these two questions:

What are the consequences of making poor decisions because we hurried too quickly through the data exploration?
Do we need to go for more accuracy or is a ballpark analysis good enough at this time?