Seven Ways To Assess The Impact Of Analytics Projects Or Solutions
Vivek is the creator and head of Medha Analytics, offering advanced analytics and AI solutions for healthcare providers.
getty
With an analytics project or solution, an impact assessment is crucial, often as important as the solution or project itself. Essentially, an impact analysis does exactly what it says: It allows analytics teams to understand what impact the project or solution actually had in comparison to its desired result.
The impact assessment should happen at two phases in the lifecycle of an analytics project. First, the potential impact needs to be calculated when the project is conceptualized. Then, once the solution has been deployed, actual impact needs to be continuously calculated.
In order to understand how this is done, let’s look at two broad categories of analytics solutions.
1. Problem-Solving: Problem-solving analytics is built around a clear objective, such as improving revenue or throughput. Problem-solving analytics solutions need to have a clear impact associated with them.
2. Exploratory: Exploratory analytics is more of an investment aimed at uncovering insights in poorly analyzed areas or “dark data.” Exploratory analytics is broader but expected to lead to problem-solving analytics use cases down the line.
Measuring the impact, however, is often not straightforward. In this article, I will examine seven potential avenues for approaching impact assessment.
1. Savings In Man Hours
This is the most basic impact that an analytics project can deliver. Recognizing this benefit often helps organizations move into a shared services model, centralizing and automating more and more analyses.
2. Impact On The Frequency Of Decisions
Certain reports are often presented for decision making every month or every quarter. Automating the insights from the report to a daily or weekly report allows more frequent interventions. A course correction that happens every month could, therefore, happen every week, i.e., the inefficiency build-up over 30 days can now be restricted to only seven days. The difference between these inefficiency build-ups can be calculated as an impact of the analytics project.
3. Impact Of Earlier Interventions
In certain cases, it may not be possible or practical to improve the frequency of analysis or to increase the frequency of decision making. However, it may still be possible to compress the time to insights.
If an analysis project can compress the time to arrive at insights—versus arriving at these insights manually—then the analytics team can evaluate the time to intervention. For example, if a report takes three days to prepare manually and is subsequently analyzed by the management after one day, the impact analysis should show that it saved three to four days of inefficiency.
4. Impact Of Informed Decisions
There are often factors or perspectives that an analytics solution can bring by analyzing data at scale, analyzing a wider spectrum or employing a technique that may not be easy to replicate manually.
The potential impact of such solutions, however, is difficult to calculate. People would have still made those decisions before implementing an analytics solution, but they may have filled in any gaps through their experience or knowledge in the area.
To calculate the impact of an informed decision, it is important to define the objective of the analytics project at a very granular level. Consider a project aimed at revenue improvement. An organization’s revenue is often too broad an objective to be entirely attributable to a single analytics project, as changes in revenue are affected by several internal and external factors.
To successfully analyze the impact of an analytics solution, understand how the project attempts to enhance revenue. Is it intended to improve conversions of a particular cohort of customers from a particular channel? If so, the objective or the target metric needs to be exactly that.
5. Impact On Data Quality
Analytics solutions, especially in their first deployment, tend to bring data-quality issues to the fore. Teams may have been resolving such issues manually or making decisions based on incorrect data. An analytics solution can help the team track the extent of gaps in the data quality or non-compliance. It can also strengthen processes around data capture.
To measure the impact on data quality, we can look at two approaches: 1. Checking how the data with a higher quality would’ve changed decisions of the past and 2. Checking how data with a poorer quality would change decisions of the present.
6. Impact On Direct KPIs
This is probably the core measurement that needs to be done. If the objective or the direct KPIs of an analytics project have been clearly defined, it is easy to go back to the function that is responsible for driving the KPIs and ask them for a target or a goal they want to get to through the analytics solution.
When the solution is straightforward and the objective is clear, the impact on the direct KPI is the most important factor for an organization.
7. Impact On Indirect KPIs
While the objective of an analytics project may be very clearly defined, the KPI or metric that it improves may not appear important or relevant enough to excite the management team. It is important to connect the metric to a clearly quantified financial impact.
This may require extrapolations or simulations to get to the impact on an indirect KPI. For example, by reducing the wait times of customers in a process, the throughput will be increased, which frees up bandwidth to handle a more crucial demand, creating capacity for additional revenue. It may also improve customer satisfaction, which can increase revenue by a certain factor in the medium to long term.
Conclusion
Impact assessment requires logical thinking and a good dose of creativity. They also require common sense and due diligence. A colleague from the industry once mentioned that their analytics team had gotten extremely creative, calculating the impact on revenue and ending up with an impact that was higher than the actual revenue itself.
That said, when the potential impact of a solution is clearly quantified, it becomes a powerful sales pitch and a critical driver of adoption. Nothing is more persuasive than concrete visibility of impact, and a clear and concise impact assessment can provide exactly that.
Forbes Technology Council is an invitation-only community for world-class CIOs, CTOs and technology executives. Do I qualify?
