Title: Continual Improvement of ISMS through Statistical Trending
Facility: Hanford – Fluor Hanford
Point of contact:
Brief description of the Best Practice
Continual improvement of ISMS can be obtained in a simple and cost effective manner. The key elements of the process involve the following activities:
-
Break out the ISM System Description into implementing mechanisms
-
Set mechanism performance expectations and defining performance measures
-
Use statistical process control charts to monitor and evaluate performance on an ongoing basis
Driving continual improvement in ISMS can be a complex process or it can be an efficient and effective process. Fluor Hanford’s decision to use Statistical Process Control (SPC) reduced the resources needed to drive continual improvement, and has provided an
efficient and effective process. SPC provides management with valid and credible performance analysis and directs resources to those areas of ISMS that need the most improvement first.
The process looks at existing performance reports and data available from central reporting systems. Data gathering is highly automated, and added little cost to the existing data systems. The performance reports are reviewed, data analyzed and trends
identified by an ASQ Certified Quality Engineer (CQE).
Meetings are held with the mechanism owners for discussing trends and issues. Information is collected to define the performance expectations and objective(s) of the mechanism and selection of mechanism elements for improvement.
Development of measures occurs throughout the life cycle of each ISMS mechanism. When the mechanism’s performance data become stable for twelve months or longer it is time to evaluate the need for improvement in the mechanism’s performance. Stability is
determined through the use of SPC. If the performance data are stable, the statistical baseline is then compared against benchmarks, customer expectations, and other performance objectives. If it is determined that improvement is needed, then a system change is put in place. The
effect of change is evaluated through use of the performance measure(s). When a significant improving trend develops, that is feedback that the system change was effective.
If improvement in a stable measure is not needed, the measure is monitored to ensure that no adverse trends develop, and management may consider selecting new measures for that mechanism. When management determines that no improvement is required for a certain
mechanism, consideration should be given to moving resources to other, higher risk mechanisms that are in need of improvement. This becomes a very important means of applying the "graded approach" and risk management called for in ISMS and Enhanced Work Planning (EWP).
If there were trends in the data, appropriate corrective and reinforcing actions can be taken in a timely manner to arrest adverse trends, and reinforce improving trends.
Fluor has changed from selecting "one size fits all" performance measures and target levels. We have changed from chasing monthly fluctuations in the data to understanding the source of the fluctuations. We determine performance based upon on SPC trend
information. The important part is the process that analyzes and trends the data for improvement. Making the decision that a mechanism is either performing at an acceptable level or is in need of an improvement is more important than comparison to a single number target. Separation
of significant trends (signals) from noise also minimizes waste of resources that would otherwise be spent chasing random fluctuations in the data. We monitor that the trending and improvement processes that are in place and functioning for each critical mechanism.
This method of managing ISMS ensures continual, ongoing improvements that make the workplace safer, more effective, and more efficient. The key aspect of sustaining and maintaining the system is to know how the system is performing. Mechanism performance must
be managed differently, depending on if a trend is in progress (change is occurring) or if the process is stable. Each mechanism has a relationship to an affect on the total system.
Most system descriptions contain a list of documents such as policies, requirements, plans, directives, procedures and guidance. Reviewing the documents for the key drivers or mechanisms and developing a set of expectations can identify performance measures
and indicators for each mechanism. Then SPC is used to analyze and display the performance data on a monthly or quarterly basis. SPC is the core of performance monitoring and provides the tool for focusing on system performance excellence.
FH developed procedures for use of SPC and uses the resulting charts as the primary source of performance data for making decisions affecting ESH and ISMS. This process has been successful as evidenced by the continual performance improvement achieved. A
detailed article titled "Cleaning Up with SPC" was published by the American Society for Quality in September 2001.
Why the best practice was used:
The continual improvement of ISMS and annual performance review was initiated to drive performance improvement and at the same time reduce cost for performance measurement. This meant selecting performance measurement processes that produce reliable results
and that could stand the test of various assessments and audits. The SPC method was chosen because it was easy to focus on total ISM system performance.
The DOE Richland field office is very receptive of performance data presented in SPC format. In FY 2003, Fluor Hanford introduced a color coded "dashboard" where the colors have been established automatically through SPC criteria. Initial use of the
"dashboard" approach has led to quick recognition of trends, both improving trends and adverse trends. Timely corrective actions have been effectively implemented in the case of adverse trends. The dashboard also allows for identification of stagnant performance areas.
What are the benefits of the best practice
The benefits of continual improvement of ISMS using SPC include direct cost savings in performing analysis, cost savings through improvement of monitored mechanisms, and increased output of the mechanisms. There are also considerable losses if an organization
does not use SPC, and improperly reacts to performance information trends.
The use of the automated SPC process reduced the number of personnel required to develop performance charts. It is estimated that the use of SPC has saved Fluor Hanford at least one million dollars per year. The use of SPC enabled FH to meet performance
incentives, achieve injury reduction, improve procedure compliance, and achieve dose reduction. Proper use of SPC has caused a reduction in the number of internal and external assessment findings related to analysis of performance data.
The use of the SPC process can reduce the normal processing and reporting time by 80%. It takes less than a minute to perform a periodic update of a control chart. Effort is not expended "explaining" random noise in the performance data.
The requirement to conduct an annual review of your ISM System is made simple if the key items to review at the end of the year has already been reviewed on a continual and routine basis. The use of SPC is the most cost effective trending method and provides
the greatest creditability.
The cost of not using SPC includes missed increase in incident rates when using moving averages or other methods and easy identification of data errors when not obvious in other methods. Attempting to take individual (instead of systematic) corrective actions
on current data when no trends exist will waste resources and not improve the system. Also, the direct cause of performing analysis increases as analysts are directed to "find another rock" instead of performing a standardized analysis approach.
What problems/issues were associated with the Best Practice
The SPC process required a change in thinking from arbitrarily selecting performance goals and comparing this month’s results to the goal (or to this month last year, or other two-point analyses). Managers wanted to "explain" all data results, even if the data
simply represented random noise. It was found that the best way to overcome these problems were to be sure that management was briefed on the new process, and Dr. Deming’s Red Bead Experiment was used as a common "experience" to convince management and the workforce of the need to
change how they deal with performance data. Further information on the Red Bead Experiment is available at
http://www.hanford.gov/safety/vpp/redbeadreach.pdf
How the success of the Best Practice was measured
Specific cases where SPC was implemented were tracked for analysis costs and for improvements in the performance measures where SPC was implemented. In all cases, analysis costs dropped, and performance levels were improved.
Description of process experience using the Best Practice
In one specific example (Radiological Problem Report trending at a Fluor Facility:), the analysis burden was reduced from 40 hours per month to 15 minutes per month. This reduction was achieved through the use of automation of data gathering, formal SPC
trending rules and standardized chart formats. In every case where FH changed to control charts the cost was reduced and reliability improved. The time reduction has permitted the persons originally involved in chart making to focus their time on making use of the data results. Time
was also saved in chasing false alarms (noise) in the data, and focusing corrective actions on verifiable trends. Recognition of stable performance allowed management to focus on systematic improvements.
