Don’t Use Other People’s KPIs

Some time ago I was involved in a discussion about whether First Call Resolution Rate (FCR) is a good KPI for incident management. After the conversation had died down I realized that the underlying issue is that you have to understand what you are trying to achieve before you can define KPIs.

So What Exactly Is a KPI?

Part of the problem with KPIs is that people forget what the letters K, P and I stand for.

• K stands for Key – we can’t measure and report every possible metric, and if you have too many KPIs then the less important ones will distract you from the Key ones that you really need to focus on.
• P stands for Performance – a KPI should help you to understand how well you are performing, and each KPI should help you to focus on a Critical Success Factor (CSF) that you need to achieve.
• I stands for Indicator – a KPI is not proof that you have achieved something, it is simply an indicator that suggests how well you are doing. Achieving your KPIs is not the end goal, it is simply a tool you can use to help you focus your activities.

What’s the Difference Between a KPI and a CSF?

CSFs are things you must achieve to help you meet your goals and objectives. These may not be measurable, but it should be very clear how they contribute to your success. Examples of CSFs for some IT service management (ITSM) processes could be:

1. The business is protected from the adverse impact of IT change (change management)
2. Downtime of the service does not have a serious impact on the customer’s business process (availability management)
3. Workarounds reduce the business impact of problems (problem management)
4. Although you can’t directly measure these CSFs you can discuss them with your customers and other stakeholders, and agree that this is what you are trying to achieve with your ITSM processes.

Once you have agreed your CSFs you should then define KPIs that indicate how well you are doing against these CSFs. There should only be a small number of KPIs for each CSF, as otherwise the numbers become unmanageable.

Here are some example KPIs for the CSF “Downtime of the service does not have a serious impact on the customer’s business process”:

• Maximum of four events in any year where there is a total loss of the service to all users
• Maximum downtime of 30 minutes for any total service outage
• Achieving these KPIs will not prove that you have met the CSF, but will help to indicate it.

You can agree with the customer that this is what you will measure and report to them to indicate how well you are doing, but in your customer review you need to discuss the CSF, not the KPIs. The customer discussion should be “Here is the KPI data, do you agree that downtime of the service has not had a serious impact on your business processes?” This way you can focus on what is important rather than on what you measured.

Why You Should Not Use Best Practice KPIs?

The ITIL publications include lots of example KPIs, but each section that lists KPIs says:

“These KPIs should not be adopted without careful consideration. Each organization should develop KPIs that are appropriate for its level of maturity, its CSFs and its particular circumstances.”

Let’s take the example of FCR and think about whether this is a good KPI. This is quoted as an example KPI in the ITIL publication, and is used by lots of other IT organizations, so you could just adopt it. You could then measure FCR and report it to show how you are closing increasing numbers of customer incidents and requests during the initial phone call. This has to be good doesn’t it?

Suppose you introduce a new self-service website to allow people to manage many of their own incidents and requests without the need for a phone call. You might automate password resets, and provide knowledge articles to help people resolve common issues for themselves. This could lead to faster resolution and happier customers, but as more of the simple incidents and requests move away from the service desk, the agents will have a higher proportion of difficult incidents to deal with, so FCR will get worse. FCR may be a good KPI for some organizations, but it would be completely inappropriate for others.

Another example of a KPI that sounds good but might be inappropriate is “reduction in the number of failed changes.” This one sounds as though it must be right for every organization, who could possibly want an increase in the number of failed changes?

Imagine an organization where the change management process is not working very well. Many IT staff ignore change management and simply get on with implementing things when they want. The organization decides to re-launch the change management process, so they train and motivate staff, appoint a new Change Advisory Board (CAB), and start to do a really good job of reviewing changes. The new process works well and the number of change requests goes up from 10 per week to 100 per week. At the same time the number of failed changes goes up from zero (only very safe changes were previously being logged) to 10 per week. This is clearly a good thing, but the KPI has gone in the wrong direction.

In Summary

When you are defining KPIs you really do need to think about your organization, your level of maturity, your customers, your goals, objectives and CSFs. Then you can define a small number of KPIs that will help to indicate how well you are doing. You also need to regularly review your KPIs to ensure they are still appropriate. When did you last review the KPIs you use to manage your IT services? Maybe it’s time to think about them again.

Image Credit