We've committed the mantra of “what gets measured gets done,” often attributed to Peter Drucker, to memory. We know that measuring things changes behavior and results. But what happens after the goal of the measurement is reached?
Imagine a scenario wherein two service owners are reporting their service scorecard in a management review with IT executives. The first service owner, Robert, proudly reviews his performance. His scorecard measures have all been green (met all targets) for three consecutive quarters! He's off the hot seat in under two minutes. The second service owner, Joan, has two measures that are red and one that is yellow. She has to explain the service improvement plans that she's working on. She's grilled on the issues and actions for over ten minutes.
Now ask yourself some questions:
- Which service owner would you rather be?
- Which service owner would be perceived as the ideal in your organization?
- Which service owner is better serving the business?
- How likely is it that Robert’s service has no opportunities for improvement?
- How likely is it that Joan’s service is or will be the better performing service?
In many organizations, meeting a performance target is the goal and staying within a threshold is the desired state. For measures driven by external factors, like SLAs, this is absolutely the case. There's no need to set more aggressive targets unless required by the customer, the business, the government, or other external parties.
For targets set for the purposes of continual improvement, however, the bar needs to be raised or the focus needs to change whenever a goal is met. Continuing to use the same set of process and service key performance indicators (KPIs) after you've met your targets, as our fictional service owner Robert did, is like getting A’s in first grade and declaring that you've mastered education. You won't continue learning or identifying new areas to improve, and you'll spend money producing measures that are simply vanity measures with no business benefit. Executives, likewise, must have a different mindset and view KPIs as something other than a typical performance measure and reward process and service owners who challenge themselves and continually evolve KPIs and put themselves back into the red.
From the ITIL Continual Service Improvement book, here are the four reasons why we measure:
-
To validate. Monitoring and measuring to validate previous decisions.
-
To direct. Monitoring and measuring to set the direction for activities in order to meet set targets; this is the most prevalent reason for monitoring and measuring.
-
To justify. Monitoring and measuring to justify, with factual evidence or proof, that a course of action is required.
-
To intervene. Monitoring and measuring to identify a point of intervention including subsequent changes and corrective actions.
Note that the prevailing tone in all of these reasons to measure is to initiate action or validate that an action was effective. There's nothing in these reasons that speaks to maintaining the status quo.
If you work in a culture where Robert would be perceived as the better service owner, then you have an opportunity to change your organization’s culture. The following are suggested steps to take toward that goal.
Step 1: Classify Measures
All measures fall into a handful of key classes. It is important to understand these classes because it will help you design scorecards that executives can understand. That understanding is a first step in changing the improvement culture of your organization.
-
Class 1: Performance to SLA/OLA/contract. These measures are intended to show that contractual commitments were met. The targets and measures are fixed and documented in the SLA, OLA, or contract.
-
Class 2: Organizational performance. These measures are mandated by the company and include things like cost, morale, safety, effectiveness, etc. Additional organizational performance measures can be mandated by the CIO or executive leadership in an organization. The targets and measures are generally fixed and established in the mandate.
-
Class 3: Regulatory/compliance performance. These measures are mandated by policy or law, either externally or internally. The targets and measure are either explicit or implicit.
-
Class 4: Improvement measures. These measures are determined by a process or service owner and often referred to as process or service KPIs. They collect data to determine a course of action or validate that an action was effective. They can be driven by critical success factors, strategies, leadership mandate, organization change, assessment or audit, etc.
The key differentiators between these measures and the previous three classes are that they are flexible and can be measured using whatever method makes sense with a target or threshold that is generally set relative to a baseline of performance.
Once you've classified all of the measures, you can build scorecards around those measures.
-
Customer-facing scorecard. This scorecard should focus on the first class of measures. The results of key improvements from the fourth class should be used when appropriate to garner trust or demonstrate your focus on continual improvement, but those measures should not be part of how the customers evaluate your performance.
-
Organizational performance scorecard. If the scorecard is an internal report to the CIO or a company executive committee on how you are performing, it will likely focus on the first three classes of measures with a few very important measures from the fourth class.
-
Management review (or quality) scorecard. If the scorecard is for management review, as defined in ISO20000/ISO9000, where the quality of the management system is the focus, it should focus on all four classes of measures.
Step 2: Educate Leadership
Management review is the forum where process and service owners should feel comfortable discussing the good and the bad, because the bad equals an improvement opportunity. If measures are classified so that executives can understand process and service performance in context, they can begin to accept scorecards that have measures that aren't meeting all targets.
Scorecards should clearly delineate measures that are required (classes 1-3) from those that are self-imposed (class 4). This establishes the capacity for a new conversation in management reviews.
To illustrate what the new conversation is, let's go back to the service owner scenario mentioned earlier. Using the measurement classifications when reviewing her service performance, Joan’s report could go like this: “We've met or exceeded all service SLAs for the past three quarters. All security and continuity requirements have been met on schedule. For the service improvement initiatives we're focusing on, we have not yet met three of the seven targets, but the trends are positive, so I’m satisfied that our action plans are going to be successful. For two of the improvement goals, we've met the target for two consecutive quarters, so I'm going to drop those from my scorecard and replace them with a new improvement opportunity that was set as a high priority by the CSI (continual service improvement) manager. I’ll have a new baseline measure and target for that improvement ready by next month.”
In that script, Joan did three things:
- Assured management that first and foremost they are meeting their service commitments to their customers and are compliant to internal policies.
- Acknowledged where she was not meeting self-imposed improvement targets but showing confidence that she is on the right track.
- Reinforced the goal and essence of continual improvement by moving on to the next opportunity once an improvement was implemented and being sustained.
Step 3: Establish Control
The final step toward changing the culture is to implement some governance around improvements, measurements and targets. Why is this important? Because people are involved and some people will want to toss or change an improvement or measure because they aren't comfortable reporting negative results. Establishing some governance and criteria for changing what is measured or the threshold/targets will maintain leadership’s trust that their process and service owners are reporting with integrity.
It's up to each organization to determine how to implement governance over what improvements to pursue and what to measure and how much rigor to apply to that governance. If there is a service management office or a CSI manager, you likely already have a governing body in place that can govern the fourth class of process and service measures. As to the amount of rigor to apply, the “trust but verify” approach is a good starting place if you do not know where to begin.
Next you will need to define the criteria for when a measure can be removed or changed. Some good criteria to start with are:
-
There are new business strategies or organizational goals. For example, when a company is trying to significantly reduce costs, improvements that are costly should probably be dropped.
-
A target has consistently been met. When this situation exists, the improvement has likely taken hold and continuing to measure it month after month will not result in new opportunities for improvement. If no action results from a KPI, it's no longer useful.
-
Something more critical needs attention. Everyone has resource constraints and must reprioritize from time to time. If an improvement opportunity is discovered that has more value or is driven by the customer or executive management, it is appropriate to drop a measure so you can focus on the new improvement.
-
A process or service has changed. If you are measuring performance on something you no longer do or provide, then it is obvious that the measure should change or go away.
Once the criteria are established, the governing body can use them to authorize or verify measurement changes. Confirming that the CSI process is being followed and that the CSI register is being used will also ensure that process and service owners select the right improvements to pursue. Finally, process and service owners can use the criteria to self-check and to explain why their scorecards have changed.
* * * * *
As service management professionals, we must always remember that continual improvement is a cycle. Once we complete the cycle, it's time to start a new cycle. In order to do that, and to be accountable for doing that, we must be comfortable with changing what we measure and how our performance is judged. Using the steps outlined above will move your organization toward a culture that rewards that behavior and has the ability to meet the CSI objectives of continually reviewing measurements and ensuring that you are identifying opportunities for improvement.
Ilene Jones is an enterprise service architect in the IT organization at The Boeing Company, where she has been working on ITSM programs for thirteen years. In her current position, Ilene is responsible for guiding the business strategy, governance, and service architecture for Boeing’s ITSM implementation, and she formerly served as the chief process architect. She’s active in itSMF USA, having served as a track chair for FUSION 08 and as a member of the knowledge management committee.