In the past, I have been called into situations where higher management were wanting a dashboard. What follows is my collection of key questions that I would ask (or at least have in the back of my mind) in such situations; with the goal being to increase our clarity about the end result and what to expect from such a project.
My questions can probably be grouped as follows:
- General purpose
- Subject matter experts
- Data sourcing
- Data wrangling
- Data refresh process
- Chart design
- Dashboard design
- Delivery + security
- Feedback process + on-going maintenance
Dashboards are so hot right now… and [head of some other department] has one and I want something like that, but for us.
Is there something that happened recently, where having had a dashboard might have facilitated a better outcome?
What is the current state of the reporting that's already taking place?
Is there a particular aspect of the business that needs to be reported on in a more timely manner?
Why aren’t you getting someone internal to do this? Do you have someone who could?
How will you verify that any processes implemented have been done appropriately; in a secure, efficient, maintainable and well-documented manner?
How viable is it, to be sharing your data with an external party (such as myself)?
What guidelines / requirements are in place (eg. NDA's, protocols for handling data (eg. setting up a shared drive instead of emailing files all over the place, clear understanding of who can access what))?
Is this your first dashboard?
Is a dashboard really the ideal solution? Could an automated pdf or PowerPoint report, refreshed to a schedule, be sufficient. [From now on, references to "dashboard" can be considered as stand-ins for any practical reporting solution].
Your subject matter experts (SME's)
What is the data-literacy level of the subject-matter / domain experts?
How available are these people for meetings (discussing the subject matter, their current process and potential for automation)?
Will the SMEs feel they are "automating themselves out of a job"?
If the project reveals inaccuracies in the SME's past/current reporting (and this looks like requiring additional work), will "getting back onto the right track" be well-supported?
How will the SME's feel about having to learn a new tool / process (as opposed to just sticking with Excel)?
Who will the dashboard be for?
- Internal business teams
- External clients
- Public facing
How will the dashboard be consumed?
- static visuals only
- on mobile
- interactive (with filters)
- will it take on app functionality (eg. be a delivery system for refined Excel output)?
Have your end-users had past experiences with dashboard (platforms) in the past?
What’s the timeline for getting version 1 up and running?
What’s your timeline for getting it to the “actually useful” (with real adoption) stage?
Should we prioritise a “mock up” example?
How many "domain areas" will be involved?
How many subject-matter experts will be involved?
Do we have a checklist of desired metrics for each area?
How many data sources are involved for each area?
Is the data currently being collected at the required level of granularity (or is it too aggregated) for the desired metrics / charts?
Does the data collection process require collaboration between and/or input from multiple people? Is the infrastructure for this data collection already set up?
Is gaining access to the data (at specific levels of detail) going to be challenging due to its sensitivity and privacy issues?
Will we have to “silo” the different data sources (so that only certain people can see the data from certain areas)?
What are the data source types?
- Excel files
- csv extracts
- email attachments
- Googlesheets / OneDrive
- other online sources
Will a data warehouse need to be set up -as an intermediary location to house all data collected -for the process?
What refresh schedule will be possible for each data source?
What refresh schedule will the metrics/charts of the dashboard require?
Will certain data connections need to be live?
Will any sources require manual or automatic augmentation (eg categorisation / grouping / recoding)?
Could errors ever be present in the data (typos, invalid data, duplication) that could cause errors in the processing? Does a review step need to be included to prevent potential “under the radar” type errors?
How complex do you think the data wrangling (getting all the data "neat and tidy") will be?
Does the wrangling phase need to include validation steps to check for errors / duplication and to generate alerts (for key personnel) that something is “up” with their data source?
Is the combination of different data sources (or the augmentation of the data) seeming relatively intuitive?
Do the metrics have designated targets that will also need to be sourced and incorporated on a regular basis?
Data refresh process
How automated can we make the data sourcing and wrangling process?
Do we want to “embed credentials” and make parts of the process completely “hands off”?
Do we need each data source's results to be verified by the SME, before any refreshed results are shared?
Will this automation be resilient to anticipated / possible simple changes in the data source? What sort of changes would it take for the process of a particular data source to fail?
Will the automation of the data processing bring with too much complexity?
Is the complexity involved with looking after the process, suitable for the relevant staff member who will be left with the responsibility. Or is some on-going partnership arrangement more appropriate?
If the source data changes format / structure, will the relevant staff be capable of adjusting the established process, so the charts continue to be refreshed?
Can we copy chart designs from your existing reporting?
Should we build "scaffolding" metrics/charts, based off of aggregated data (in such reports), to get a static dashboard produced quicker -so we can prioritise the design and the delivery aspects over establishing the data-wrangling processes?
Any draft drawings of the charts available?
What filters could be included (these will usually apply across multiple charts)?
What “parameter selections” could be included (eg, swapping out metrics, changing time-frame comparisons (eg, past month or past 6 month comparisons)).
Do any charts / statistics look like they will require complex LOD or DAX calculations (eg. time period comparisons, percent achievement towards goals, etc)?
What tooltips (or even in-tool-tip-vizzes) could be included (i.e when hovering over certain chart areas, further info is shown)?
Any sense of how the individual charts will fit together (i.e overall design)?
Do you have any pictures of preferable dashboard designs (or even a colour palette to be used)?
Do we need to display the recency of all the data sources?
What “click actions” (filtering between charts by clicking on parts) will be required?
Delivery + Security
Do we require “row-level security” / “context-filtering” (ie. only displaying certain data to certain users)?
How will the dashboard be delivered:
- embedded on a website,
- online platform,
- single dashboard file on a shared drive (or by transferring copies of the file all over the place via email)
- saved to pdf and printed?
What will the selected ongoing delivery solution cost?
Do we need to restrict end-users' access to the data underlying each chart?
If the data is sensitive, what steps can we take to reduce the risk of a data-breach (at all stages in the process) as much as possible?
Feedback process + on-going maintenance
How will we measure uptake and usage of the solution.
Is the dashboard content useful and actionable?
Does the content actually inform “how good or how bad” the business is doing?
Do permanent staff have to be trained up in order to maintain (and perhaps improve) their respective individual processes and vizzes?
Is there a central person who is going to be tasked with ensuring that the seperate areas all work cohesively?
How much process documentation will be needed?
Will staff be able to incorporate requested changes in a timely fashion?
Process (and project) contingency planning (ie. how to handle the "Julian just got hit by a bus" scenario).