Making ITSM Metrics Work
Many people and organizations seem to struggle with metrics, reporting, analytics, and demonstrating value from IT service management (ITSM). If you’re doing service management, service level management (SLM), and continual service improvement (CSI), then metrics are essential elements and need to be focused, relevant and useful – although in practice this is not often the case.
We should use the information produced regularly from reports (based on relevant business metrics) to help us to review and demonstrate the value of our services and make valid, timely decisions about how to improve them. But once again, this is not the norm…. The result is that much resources and time are wasted on producing reports, documents, and information packs that few read or understand and which are seen to deliver little value, despite seeming to have a level of expectation and importance that demands better.
So, what is the problem?
How have we got to this position where a fundamental part of our service management practice seems to be lacking in quality and value? Here’s some thoughts:
Too much internal focus – Many IT organizations report on what they know about what they do, rather than on how what they do delivers value to their customers. So the focus tends to be internal and about IT and technology only and not on its results and outcomes. In practical terms we focus on reporting, e.g. our first level fix rate, our mean time to resolution, and our server or application downtime. There are of course valuable and essential components in our own understanding of our performance, but offer nothing to external customers as a measure of what is done for them. So we often don’t view or report on our basic operational metrics with any sort of business context.
Expecting SLAs to mean everything – Service Level Agreements (SLAs) are useful as targets for some aspects of service delivery, but not all. Yet often it is expected that ‘green’ SLAs mean happy customers, which is of course not always the case. There is a fundamental question here about how the SLAs were set and agreed upon, and by whom in the first place. Are these SLA’s relevant to the business? Do they identify appropriate measures for both to evaluate the success of the service? Have these been fully understood and agreed by all parties? Often these questions can’t be answered in the affirmative and this seriously impacts on the value of the SLAs and reports.
Asking the wrong questions – One of the reasons that SLAs are often not assessed as useful is because they are still defined and set in IT relevant terms that can never get a useful response from customers. For example, we ask them how long it should take to fix X, they don’t know or care and frankly can’t make that call, and we are frankly asking them the wrong questions. We should be asking more about value – what’s important to you? When do you need it? – Rather than simply talking about how IT responds to failure.
Spoon feeding expectations – Here there is too much expectation based around what is perceived to be written in ITIL, although most of the ‘best practice’ metrics that are used and held up as being important are not actually defined as such in the ITIL books. Many would do well to read the CSI book, which does contain a lot of good suggestions for potential things to measure and use as a basis for improvement. The toolsets also can build expectations that they will deliver a bright new world of clear and simple reporting, although this in turn requires thought and action around what’s going into the product in the first place. We have a tendency to use ITIL and the tools as props, which we expect to deliver us everything we need, when in fact we need to do some work ourselves to actually discover what the requirements are. Plus we are still too focused on individual KPIs and process-based metrics, rather than outcomes.
Inertia about doing anything with the outputs – People contribute to and support improvement initiatives more if they can see that their inputs are working and delivering some value. If we simply churn out reports that do not, in-turn, generate actions, then the whole process is undermined. It’s vital to actually use the information to make decisions, ask more questions, kick off activities and projects that actually deliver some results. Otherwise it’s a waste of time. If the reports that are produced in your organization are not used for any useful purpose then really you should ask why bother, and in fact try stopping them for a while and see if anyone notices or complains.
What do we need to do to make this work?
Here are some simple suggestions to make your ITSM metrics work:
Be clear on business expectations and outcomes. Throw away cynicism for a moment and go and talk to your customers about what’s important to them that you can measure and improve on. The results might not be hugely different from what you currently do but you will definitely learn something. It’s the process of engagement that’s important here more than the output, as you will develop some common objectives and understanding that will force you to re-focus your reporting, and that’s got to be a good thing for you and your customers! Do try and ask simple questions and listen to the answers before diving into solution mode. The goal is to find what’s useful to measure and what will matter to customers, not to IT.
Think services and bundles of metrics. Operational metrics on their own are useful to the operational team. However they are only building blocks for the bigger ‘bundle’ of value-based measurements that will be useful to the customer. So your first-level fix rate is a component of the email service. For example, rather than the only measure. Customers will tell you what they think is their priority.
Look at inputs and categories. We all know about rubbish in/rubbish out. This is very true in ITSM. A good practical task is to review input categories and the quality of data that is captured within incidents, problems, changes, etc. Things to look out for: too many input categories that can’t provide any sensible results, double entry data where data elements are effectively replicated in different categories, inappropriate levels of detail that don’t add value, simple (reason) closing codes that provide a window into causality rather than impact of issues.
Think presentation and audience. Presentation is super important, as no one will read a 200-page listing, or even a 5-page listing. You must present appropriate information in summary and other interesting formats as much as possible. That way there’s a chance that it might be read rather than binned. So think summary (1 page) outputs, targeted information (don’t give people the task of finding their results), and use graphics and color as much as possible. Also try and mix up the format regularly, keep refreshing it to spark interest.
Do something with the results. Probably the biggest failing in all of this is inertia and lack of any action following on from results. If nothing is happening then maybe the results aren’t of value, or not clearly or appropriately presented. However you also need to have a process in place to review and take action based on the results. Otherwise the process of compiling metrics is a complete waste of time. Metrics and reporting should be the key to unlock quality and value from ITSM. It just needs a slight change in focus and activity to make it work.
If you’re interested in reading more about ITSM metrics, then I highly recommend the following articles from my good friend Stuart Rance:
