Running an analytics retrospective: a habit most teams skip and most programs need
Analytics teams spend most of their time building forward. A request comes in, work begins, something gets shipped, and attention moves to the next request. It is a natural rhythm, and in most organizations it is relentless enough that there is rarely a pause to look back. That is a problem, because the questions that would most improve an analytics program are almost always backward-looking ones. What did we build over the past six months? Who used it? What decisions did it actually support? What are we maintaining that nobody is asking for?
A deliberate analytics retrospective is a structured answer to those questions. It is not a project postmortem, which typically focuses on a single initiative and what went wrong. It is a periodic review of the full portfolio of analytics work: what was built, what is being used, what is being ignored, and what that pattern reveals about where the program is and is not working. Done well, it takes a half-day and produces three things: a clearer picture of where analytical capacity is actually going, a list of things that can be retired or simplified, and an honest conversation about whether the right work is getting prioritized.
Most analytics teams that try this for the first time discover the same few things. A meaningful share of active reports and dashboards have not been opened in months. Several reports that get opened regularly are not connected to any identifiable decision. Some of the most-used outputs are informal: a spreadsheet someone built three years ago, a Slack message with a screenshot, a monthly email with a few numbers. And some of the work the data team is most proud of is the work stakeholders understand the least.
None of that is failure. It is information. The retrospective is valuable precisely because it surfaces these patterns and gives the team something concrete to act on.
The structure that tends to work best involves five questions, each examined across the full inventory of active work. These are some of the same questions our consultants ask when we are helping an organization formalize their data strategy. https://consulting.sva.com/solution/data-strategy
The first is simply: what exists? Most analytics teams do not have a clean, current list of everything they maintain. Reports accumulate. Dashboards get created for a specific purpose and never retired. Before any evaluation is possible, an honest inventory is necessary. This step alone is often surprising. Teams discover they are actively maintaining more than they realized, and that a significant portion of it was created for a purpose that has since changed or been resolved.
The second question is who is actually using this, and how often? Usage data is usually available. Most BI platforms log view counts, last-opened dates, and user activity. Pulling that data and matching it against the active inventory is often the most illuminating part of the exercise. The goal is not to shame low-viewed reports but to identify which outputs have genuine ongoing demand and which are maintained by habit rather than need. A report that was opened once in the last 90 days is probably not worth the maintenance burden it carries.
The third question is what decision does each output support? This is harder than it sounds, and the difficulty itself is informative. If the team cannot answer this question for a given report, either because the purpose was never clearly defined or because the original need no longer exists, that report is a candidate for retirement. If the answer is that it supports a decision but nobody on the business side is making that decision based on the data, that points to a different problem: a gap between what the data team is providing and what stakeholders are actually using to decide.
The fourth question is what is missing? The retrospective is not only about reducing; it is also about identifying where demand exists that the current portfolio does not serve. Stakeholders who are working around the analytics environment, using spreadsheets, pulling one-off data requests, or making decisions without data, are signaling unmet need. A structured conversation with business partners about what they are not getting is as valuable as an audit of what is being ignored.
The fifth question is what should change as a result? A retrospective with no output is just a review. The output should be a short list of concrete decisions: reports to retire, outputs to simplify, definitions to reconcile, or gaps to prioritize. The list does not need to be long. Three to five clear actions, owned by specific people and attached to a timeline, is more useful than a comprehensive improvement plan that never gets executed.
The full exercise works best as a shared effort between the data team and at least a small group of business stakeholders. A retrospective conducted entirely within the data team will miss the perspective of the people who are or are not using the outputs. A retrospective conducted once is useful. One conducted every six months becomes a governance habit that keeps the analytics portfolio aligned with what the business actually needs rather than what it needed at some point in the past.
A reasonable place to start: pull usage data for every active dashboard and report in your primary BI environment for the past 90 days. Sort by last-opened date. Look at the bottom third of that list. Ask whether those outputs are being maintained because they are needed or because nobody has stopped to ask the question. That audit, done in an afternoon, is usually enough to open a conversation worth having.
CLOSING
The thread running through this edition is the gap between what organizations have built and what is actually being used. Business formation data shows more economic activity than most planning models account for. AI adoption is running ahead of the data foundations that would make it reliable. The platforms organizations depend on are converging on metric consistency as a prerequisite for AI, not an optional enhancement. And analytics programs accumulate outputs that outlast the questions they were designed to answer.
None of these are problems that require a new platform or a larger team. They are problems that require a clearer look at what is already in place and an honest conversation about what is actually working. That is harder than buying something new, and usually more useful.
If any of this edition's topics connect to challenges your organization is working through, we are glad to continue the conversation.
data@work is published by SVA Consulting. Each issue is designed to help business leaders understand what is changing in the data and analytics landscape, why it matters, and what to do with it.