There’s a big difference between the dashboard of a car and that of a passenger jet!  For one thing, understanding a car dashboard requires significantly less training and experience than a cockpit dashboard.  Of course each is designed to meet different needs: a pilot has to worry about many more things than a car driver when ferrying passengers safely from Point A to Point B. When developing information dashboards for a business we also need to keep user needs in mind: a VP of Marketing will need a much more high-level overview than a Product Director, who in turn has quite different requirements from a sales rep preparing an action plan.  In practical terms this means that we first have to figure out who will use the dashboard and how.

Image of a Jet Cockpit Image credit: http://flysusan.com/cockpits.htm

A dashboard is only useful when it can become an integral part of decision making.  Some decisions need to be made on a daily basis, such as which prospects to call or how to follow-up with a client.  Other decisions take more time.  Those decisions tend to be more organizational and strategic in nature.  They require input from many people and data sources, they require observation and a longer term perspective. The implication here is that a dashboard needs to be capable of providing information at the speed with which decisions are made.  A dashboard also needs to provide the appropriate amount of summary and detail.  Before we can build anything, we need to define what information is actionable and how soon it needs to be available.

From a technical standpoint, this initial exploration provides the framework for determining how the dashboard should be build.  The following questions are just a starting point:

• What information is required?
• How will the information be used?
• What kinds of summaries and calculations make the most sense?
• How much detail should be included?
• How can we best present the information?
• How often does data need to be refreshed?
• Which databases and information sources are necessary?
• Which software/hardware best meets our needs?

The answers to these questions always involve tradeoffs.  Which tradeoffs make sense depends on the impact the dashboard will have and whether we can demonstrate a positive ROI.  At some point, the dashboard has to help improve the bottom line.

Drawing a direct line between dashboard and bottom line dollars can be complicated.  Dashboard benefits tend to be intangible.  To measure ROI, we have to think about how the dashboard enables better decisions, how it helps users focus on profitable actions and whether it helps to save time or other resources.  Often we need to offset these intangible benefits against very real budgets and real money that needs to be spent on development, training, software, hardware, and so on.

As we can see, if we are serious about building a dashboard, we also have to be serious about spending some time upfront to map out a plan for getting from the Basic Idea to an Actual Dashboard.  To get there, everyone involved needs to develop a common language, that is, a set of common definitions and goals.  For instance, if our dashboard is supposed to track progress following a product launch we need a common understanding of what market penetration means.  Does it mean “number of actual customers vs. potential customers” or “number of actual customers vs. number of targets” or “number of customers who bought at least X number of widgets.”  How we define our key metrics has a direct impact on the usefulness of our dashboard and on the effort required to build it.

Developing common definitions and goals is a key step toward building a bridge across the communication gap. Just because we are using terminology that seems to be well defined doesn’t mean we really are talking about the same thing. For example, we might be talking about lighthouses and we might even have the same general idea of what a light house is and how a typical lighthouse looks. But when it comes to actually building the lighthouse we need something more concrete.

What’s a lighthouse?

The Idea	An Actual Lighthouse	Another Actual Lighthouse
When discussing the concept of a lighthouse, usually this type of image comes to mind.	Most of us would expect an actual lighthouse to look something like this.  It meets the typical expectations for a lighthouse.	Sometimes a lighthouse needs to address special situations: when a client needs a special lighthouse and the developer thinks of building a typical lighthouse, the project is in trouble.

Image credits (L to R):

http://www.freeclipartnow.com/buildings/lighthouses/

http://www.destination360.com/north-america/us/maine/images/s/maine-lighthouses.jpg

http://www.longislandlighthouses.com/lv112.htm

• How does one differentiate between the “voice of the customer” and the “noise of the customer?”

VOC or “Voice of the Customer” is a key concept in Six Sigma, the quality methodology used by Toyota and many other companies.  Needless to say that with millions of customers, there are millions of opportunities for feedback – hence the potential for noise.

Wordplay aside, any communication from a customer contains some useful information, but not all feedback carries the same weight.  For example, a broken radio most likely has less impact on car safety than a stuck gas pedal – but we can’t be sure until we have more information: the broken radio may be a symptom of an electrical problem that also affects the accelerator.

Therein lies the problem: how do we assign the “appropriate” value to the information we receive?  How much effort and money do we put into researching the (hypothetical) “radio problem” versus other problems?  How can we quickly assess whether the “radio problem” can turn into a “safety problem” that requires thorough attention?  With the myriad of active and passive ways in which we can listen to customers, we need a good triaging system to help us separate critical information from information clutter.

While everyone can agree that data needs to be used “appropriately,” it is much more difficult to agree on what “appropriate use” actually means.  Assuming for the moment that we can collect accurate data, what do we need to know in order to elevate an incident from “routine” to “requires immediate attention?” Here are several key factors that influence appropriate use:

• The ability to recognize the potential for significant harm
• The ability to draw a correlation between the incident and significant harm
• The ability to develop a solution to the problem
• The ability to implement a solution to the problem
• The ability to make that solution pay off in the long run

Each of these bullet points shares two characteristics: to accomplish them, we need good information as well as sound judgment – neither of which comes easily.  This applies to all types of corporate decisions – whether we are dealing with product safety issues or the most profitable allocation of sales and marketing resources.  The major differences between types of decisions typically revolve around their scale and the level of detail required to make a decision.

It is impractical to go through all the possible ways in which we can identify “appropriate” information.  Instead, here are a few guidelines:

• Assess the potential harm
• Identify actionable information
• Prioritize timeliness, accuracy and budget
• Identify who needs to know what and when
• Incorporate the means to review requirements from time to time

Keeping these bullets in mind goes a long way toward selecting the tools and resources needed to supply appropriate information.

Additional Reading

Toyota knew of accelerator pedal problem in UK a year ago
From The Times
February 2, 2010

http://business.timesonline.co.uk/tol/business/industry_sectors/transport/article7011671.ece

Unintended Acceleration: Toyota Addresses the Issues
November 06, 2009 by Irv Miller

http://pressroom.toyota.com/pr/tms/our-point-of-view-post.aspx?id=2234

Wikipedia entry for Six Sigma, the quality control methodology used by Toyota and many other companies.  Voice of the Customer (VOC) is a key concept of the Six Sigma methodology.

Software companies like Tibco/Spotfire and Tableau are putting visual analytics front and center when it comes to helping businesses find out where their bottom line hurts and where opportunities may be hiding.  A recent article by Ted Cuzzillo for The Data Warehousing Institute^TM talks about a “new breed of BI Analyst” who wants to dive into the data, try things out, see what works and explore it in an interactive, almost intuitive, manner.  Enterprise blogger Michael Vizard, who writes for the Ziff Davis suite of publications, even foresees “The Demise of Report Writers and Spreadsheet Jockeys.”

Too good to be true?

Remember the promise of the “E.I.S.”, the Executive Information System, which was supposed to put all the answers at the fingertips of executives and management?  Once the novelty wore off, executives and managers asked their administrative staff to pull data and print out reports from their company’s E.I.S.  When questions arose, the IT or Business Information departments still needed to provide answers.

I don’t mean to rain on anyone’s parade: visual analytics is the future and I enjoy using tools such as Tableau (see my review here).  Visual exploration opens the door to much faster insight and understanding of what’s happening in a business.  Rather, this is meant as a cautionary note not to get caught up in the belief that a tool alone can solve our information problems and turn someone into an expert analyst with all the correct answers.  The risk with visual analytics lies in the fact that it appears to turn data analysis into child’s play, when, in fact, real work has to be done.

Tackling the hidden hazards of visual analytics

Let’s face it: to draw useful conclusions one has to deal with the fact that data is imperfect, that factors beyond our control affect the quality of data and that sometimes good judgment is needed to discern the meaningful from inconsequential.

An analyst has to understand the data behind each graph and know how it is collected, processed and presented.  When is data updated? Why could data be missing or incorrect? When is an unusual observation due to problems with the underlying data rather than actual performance of the business?  These are just some of the factors that may influence the outcome of an analysis, visual or not.

Granted, if the underlying data tables are well designed and if good data quality processes are in place, visual analysis becomes much safer for someone who doesn’t have time to worry about data schemas, update cycles and data scrubbing.  None the less, the person performing the analysis still has to know how to question the underlying data or risk reaching misinformed conclusions.

Visualization tools often make it easy to group data and to create new summaries on the fly.  Discerning what really drives results can be tricky when dealing with such summaries and groupings.  For instance, when drugs are used to treat multiple indications, how do we know that the data group we are exploring really contains all the relevant elements?  There’s something to be said for good meta data management and clear definitions for the metrics used by decision makers.

Graphs and charts are simply clutter unless one has the good judgment of knowing which blips are worth exploring and which are simply noise.  Some outliers point to real issues while others are merely distortions.  Knowing which is which requires business knowledge and a feel for how data behaves.  For instance, a spike in unit sales has very different implications for the bottom line when high unit sales are achieved through heavy discounting for major customers as opposed to gaining new customers at full price.

Visual analytics is no longer the next frontier

The emergence of visual analytics feels a little bit like going from command line DOS to the visual interface of Windows.  The further away we get from knowing about the inner workings of a computer – or in this case “the data” – the easier it becomes for errors to slip past our awareness because we don’t know what questions to ask or what can get us into trouble. Just ask any computer security expert how often people neglect to install or maintain intrusion protections on their computers.  Yet, nowadays no one can imagine life without computers anymore.

Today’s abilities to easily pull information together from various sources, to create graphs on the fly and to select individual data points for further analysis would be impossible without the ability to remove ourselves from the tedious details of collecting, transforming, staging and retrieving data.  Without the ability to gain distance from the inner workings of data we would drown in information overload – which really is no better than having to run a business by gut feeling and dead reckoning.

The real power of visual analytics comes to the fore because an analyst now has the tools to combine business knowledge and data expertise without having to spend hours preparing code or staging data in more effective ways.  That time can now be spent doing analysis and deriving value from the data.

However, with power comes responsibility.  Just because today’s tools remove us from the tedium of data preparation doesn’t mean we abdicate our responsibility for analyzing what’s under the hood.  Instead, we have to develop QC methods and standards that allow us to recognize when something goes awry.

Personally, I am happy to see that business intelligence applications and visualization are merging together.  Point and click frees my inner data geek from having to write tons of code and allows me to get right to the fun part: seeing what I’m dealing with, understanding how the pieces fit together and figuring out answers. Now I’m just waiting for the day when we can say “Computer, tell me how to answer this question!”  That’s the day I get to retire