The Many Approaches to Performance Measurement
Business processes have a purpose—in the case of MOC, that purpose is to ensure that a change is performed safely and effectively.
Beyond the general notion of “purpose”, a business process also must satisfy a number of requirements. Measuring the performance of business processes is a fundamental part of ensuring that a business process is functioning effectively and efficiently.
There are many different approaches to performance measurement, each with its own methodology, each with its own benefits, each with its own proponents. Currently popular business process measurement approaches include:
- Activity-Based Costing (38)
- Balanced Scorecard (47)
- Key Performance Indicators, “KPIs”, (5)
- Lean Six Sigma (56)
- Six Sigma (297)
- Business Process Modeling (16)
Much has been written about these various approaches to business process measurement. In fact, a recent search of www.barnesandnoble.com revealed there were 38 books with “Activity-Based Costing” in the title, 47 books with “Balanced Scorecard” in the title, and so on, for a total of 443 books. Even conducting a review of all of this material is daunting.
Terminology
Each author is attempting the explain the origins and the usage of the concepts contained in the book’s title. The authors invariably explain concepts in quantitative terms, and thus require the appropriate vocabulary. This can get confusing because words with very similar meanings (at least to a layperson) are used interchangeably or differently by different authors.
Here are some terms that shall be useful to us:
Measure, Measurement: “the extent, quantity, amount, or degree of something, as determined by measurement or calculation.1” We’ll assume that measurements are always objective. Measurements can include quantities (e.g. 52.3 days), and counts (e.g. 27 MOCs). Measurements can even include results of surveys (60% of respondents chose “strongly agree” to a proposition).
Indicator: An indicator is directly calculated from measurements and/or additional objective information. E.g. “days late”, “cost to process an MOC”.
Metric: A generic term applied to the collection of all measurements and indicators.
These can’t be regarded as dictionary definitions, nor am I making any claims that these definitions are “correct”, since their meaning is derived from how I and other authors use the terms.
A Pragmatic Approach
Although lacking the media buzz that the list of business process measurement approaches garners, there’s always (plain old) Project Management, i.e. the desire to accomplish a project on time, on spec., and on budget. We’ll take this more pragmatic approach to address the specific needs and characteristics of MOC.
Metrics for MOC Performance Measurement
Traditional project management aims to complete each project “on time, on budget and on spec.” Since MOC is a project-oriented process, these appear to be reasonable objectives for the MOC process as well. So, we can propose the following metrics for MOC:
- Time: this is the duration of various parts an MOC, or even the entire MOC process. This is measured in units of time, often days.
- Cost: this is the cost of conducting all or part of the MOC process. This does not include the cost of materials (e.g. pipe, weld rod, instrumentation) for implementing the change. Cost is measured in units of currency, often dollars.
- Goals: this is the degree to which the goals and objectives of the MOC are attained. Goals are measured by the number actual goals/objectives attained as a percentage of planned goals/objectives.
There are additional metrics relevant to many business processes, including MOC. Often these are overlooked since the measurements are not necessarily easy to make, and the units of measure are far from obvious. However, they are important and worthy of attention:
- Satisfaction: This is a measure of how much the participants like the process. The logic is that the greater the user satisfaction with the process, the more the process will be used and/or used correctly.
- Flexibility: Flexibility is the ability to adapt to new or unforeseen circumstances. The more difficult it is to accommodate new circumstances, the less flexible the process is.
- Risk: This is a measure of the probability and consequences of negative outcomes, whether they be related to safety, environmental, financial, regulatory or other issues.
The following sections will elaborate on each of these metrics.
Time
Various indicators can be constructed from measurements of time, as indicated in Table 1.
If your site typically uses time-based metrics (average duration, number of temporary MOCs that are overdue, etc.), then you certainly are not alone. Time‑based metrics are what most companies use; in many cases, time-based metrics are the only ones used to monitor MOCs.
MOC practitioners, which include MOC Coordinators, PSM coordinators, MOC originators and MOC owners, would typically be interested in the values of T1 – T6n since these can be used to manage the progress of specific MOCs.
Process Safety managers and EH&S managers would typically be interested in summaries of the time indicators. Summaries would provide some indication of where bottlenecks are in the process, and PSM managers and EH&S managers would typically have control over the resources to remove bottlenecks for this process.
EH&S directors, plant managers and executives would typically be interested in summaries and trends of T1, to satisfy themselves that the MOC process is “working” or “on track”.
Indicator | Name | Calculation Method | Units |
---|---|---|---|
T1 | MOC Duration | current date – initiation date | days |
T2 | Close-out Timeliness | actual close-out date – target close-out date | days |
T3 | Request cycle duration | approval date – initiation date | days |
T4 | Approval timeliness | actual approval date – target approval date | days |
T5 | Startup timeliness | actual startup date – target startup date | days |
T6a | Duration of initiation state | end of initiation state – start of initiation state | days |
T6b | Duration of scoping state | end of scoping state – start of scoping state | days |
T6n | Duration of close-out state | end of close-out state – start of close-out state | days |
T7 | Excessive duration MOCs | Count of MOCs where { MOC duration > a threshhold value (like 2 years) } | # |