What to Measure: Structuring IT Metrics to Improve Performance

Covid-19 has necessitated change in the way that we to work in IT, along with the ways of working for the people we serve and support. In particular, because many more of us – both in IT and from a customer perspective – are now working from home. It means that customers have different needs and possibly higher expectations of IT service delivery and support. It likely means that IT needs to up its game and to do this it needs to ensure that it’s measuring the right things.

However, the question “what do we need to measure?” is a difficult one. There’s no standard framework or set of metrics that can be plucked off a shelf to immediately help your IT organization in normal circumstances, let alone right now.

Measuring IT performance in the right way will help you align IT activity with what the business wants – and help you prove value in a way that the business understands. Whereas measuring the wrong metrics can have the opposite effect – resulting in increased dissatisfaction with IT and a shrinking budget. Ultimately, taking a strategic, structured approach to metrics will help you ensure that you’re moving in the right direction.

Take a top-down approach

Structure your metrics to support the sort of IT performance that your business wants to see. And IT’s idea of good performance is largely irrelevant. Because, most often, IT’s idea of what great performance looks like will be very different from that of the business.

In day-to-day operations, IT people need to continually ask themselves “Is what I’m doing of value to the business?” This mantra should also apply to the strategy for metrics: “Are these metrics helping us to improve performance for the business?”

In order to understand whether metrics are indeed of value, IT needs to consult with business people. That means sitting down with business leaders to agree on “what good looks like,” and how you can measure against this. The output of this process should be a fairly small set of key performance indicators (KPIs) that collectively form a well-rounded representation of IT performance from the business perspective.

Too many numbers, i.e. metrics, will put an excessive measurement overhead on IT and dilute the focus of improvement efforts. Too few and the business (and IT) will not be clear on performance. Of course, this set of KPIs will differ from industry to industry and from business to business, but there are some common metrics for performance that IT can suggest.

Measuring IT customer satisfaction, or employee experience, is a good example, because it essentially aggregates how happy business people are with what IT does. There are a number of methods for measurement, mostly adapted from general customer service metrics – including C-SAT and a variation on Net Promoter Score (NPS).

What, how and when you measure will depend on what your organization needs, but this might involve:

• Making it easy for an end user to “rate IT” at any time they choose via a web portal, email, or an app.
• Asking for feedback after each interaction (or third/fifth/tenth interaction so as not to bother end users for feedback too frequently).
• Surveying end users across the whole organization on a monthly or quarterly basis to ensure a complete and accurate benchmark.

Generally, where tools are available to automate the collection of IT customer surveys, monthly is better – but if the process involves a lot of manual effort, then you’ll need to balance the desired frequency against what’s realistically achievable.

Create a logical hierarchy of metrics

Once you have a clear set of top-level KPIs you can then start to work downwards to fill out a structured hierarchy of internal IT metrics that will help you focus improvement. Each one of these should tie directly to a top-level KPI. If you can’t identify how an internal metric links to overall IT performance, then drop it.

Taking IT customer satisfaction as an example, you can define a sub-set of internal IT metrics that will help you shift the dial upwards. The performance of the IT service desk is the number one factor affecting IT customer satisfaction, so there are some common service desk metrics that can help guide improvements and will have a positive effect on the top-level KPI.

For example, there’s a proven correlation between the first-contact resolution (FCR) rate and IT customer satisfaction ratings. So, FCR would be a viable metric. Again, the number of metrics in the sub-set should balance the effort of measurement against the ability to focus activity on improvement. If you’ve too many internal metrics, you’ll spend too much time monitoring them, and it can be difficult to see where to start your improvement journey.

Form an IT metrics “steering group”

Having a solid system of metrics is critical to IT performance, and this system needs to be managed over time to keep it on track. By forming a “steering group” or similar – comprised of stakeholders from the business and IT operations – you can adapt your metrics model to ensure that it remains fit for purpose in an ever-changing business environment.

The IT metrics steering group should review the hierarchy of metrics regularly – ideally every quarter. Business models change over time, so IT performance KPIs will need to change with them. This is just one way in which metrics go “stale”. If there’s a fundamental change in the direction of the business (e.g. a merger or acquisition), this should also be a trigger for a review. Also, as performance against internal IT metrics improves, the law of diminishing returns kicks in, so as improvements are made it may be time to restructure the metrics model to target more effective gains.

Steering group meetings present a good opportunity to discuss any improvement projects tabled by IT – and the potential impact they’ll have on top-level KPIs. For IT, this is a chance to demonstrate that they’re on the same page as the business and positive action is being taken to generate increasing value.

Another issue that the steering group must keep on top of is people trying to “game the system.” With the best will in the world, metrics often become counter-productive, particularly if there are bonuses and other rewards on the table. Metrics put people under pressure, so they naturally find ways to game the system.

By altering the way metrics are gathered (or actually doctoring the results) it’s possible to output more favorable numbers – and the story that the metrics tell can get further and further from the truth. The steering group should set policies for how metrics should be gathered and communicate clearly the penalties for rigging the numbers. Spot checks and audits may be ordered to ensure things are kept on the straight and narrow. Without trustable metrics, the whole system can quickly collapse, so ensuring honesty in performance measurement is a critical success factor.

Good metrics help IT align activity with what the business needs. And reviewing them regularly will help IT stay synchronized when business objectives change – as they inevitably do. Done right, metrics can be a strategic business tool that brings IT and the business together to optimize the value that IT delivers. Getting the right people together (from IT and the business leadership) to devise and manage a strategic approach to metrics will pay dividends by ensuring sure IT is always on the path that’s most productive for the business.

Is there anything you would add to this? Or disagree with? Please let me know in the comments.