Measuring Change Performance – Easy or Not?

Without knowing how well we’re doing, we can’t show that we’re getting better. IT service management (ITSM) service providers need to show improvement, to keep customers happy that the service is in good hands. And to show we are improving, we need to be able to measure and compare our current and previous performance levels.

But getting good measurements can be harder than we think. Change management is an essential aspect of good ITSM, so let’s take that as an example. To judge our change management performance, we just need to measure how many of our changes are implemented successfully, and then make the numbers better.

What Could Possibly Go Wrong?

Let’s put aside, for now at least, any thoughts of comparing our performance with that of other organizations. Even restricting our thinking to measuring current performance against previous, we first have to clarify what is really meant by some of the words we use. Words like “change” and “success,” for example:

“Change”

ITIL^® tells us that a change is ”the addition, modification or removal of anything that could have an effect on IT services.” That’s pretty wide-ranging, and might include many simple things that are done automatically or via a service request and aren’t really that dependent on our change management process. If we want to honestly judge the change process, we need to be sure we are considering only the changes that actually depend on that process for their success.

We also need to understand how our change process fits in with other ones for big enterprise-wide changes. A significant business change might generate multiple IT changes. Should IT treat that as one big change or several smaller ones?

“Success”

This is the tough one for many. When is a change successful? And who decides? From the service provider’s perspective, a change asks for something – or more often a list of things – to be done. So a service provider tends to see success as having done those things. But does that mean that to be considered successful a change has to:

• Do ALL the things that were asked for?
• Do most of the things?
• Be implemented on time?
• Not cause too much damage elsewhere?
• Be within budget?

These are just some examples; there are more.  Each one has its merits and its complications in measurement. Most of them have a subjective element. And they are often conflicting – should we get it done on time, or wait a few days and get everything right?

So here’s my advice. Don’t get sidetracked too soon into actually measuring what you are doing. Instead, remember why you are doing it.

Expected Value

Forget work for a moment and imagine instead that you were buying goods or services for your domestic life. You will usually start with a clear picture of what you expect to get. That will form the starting point for answering the later question: “Am I willing to pay for this?”

If we’re paying someone to change our house or car, for example, we set our success parameters as part of the initial bargain we strike with the supplier.  So too in our work in ITSM – each and every change should include a definition of success. Without getting that customer view of success, we, as service providers, can only guess if we are doing the right things.

The simplest way to explain this to our customers is to ask that ultimate customer question. At which point would you be happy to pay? This should apply even if – with an internal supplier – there isn’t any real money changing hands. Still the concept of “when would I feel it right to pay” is a powerful indicator of having delivered success.

As a supplier, there are a range of factors (including those set out in that bulleted list above) that might give you a starting point when talking through with your customers what any given change should do to make it successful. But they have to decide. And then you have to deliver to that in order to be seen as “successful.”

No change should be allowed to go forward without success criteria being signed off by both supplier and customer; without knowing what you will be judged on, how can you possibly properly target your efforts and priorities?

Then You Have to Do It

This doesn’t do away with the need to set mechanisms in place to actually measure…and to measure consistently and to a previously agreed level of detail, timing, and accuracy. But it does make us agree on what we’re going to be measuring. Getting that agreed on beforehand allows us to then plan (with the customer ideally) how and when we’ll measure that success for each change.

Not getting that target set can leave us working hard on the wrong aspects, the wrong priorities, or even the wrong changes.

Since I’m on the topic of change management, it would be remiss of me not to refer you to my 15 tips on getting started with change management….so there you go. I hope you find that blog and this one on measuring change successes useful. Please let me know.