Root Cause Tip: Audit Your Investigation System (A Best of The Root Cause Network™ Newsletter Reprint)November 26th, 2014 by Mark Paradies
AUDIT YOUR INVESTIGATION SYSTEM
AUDIT TO IMPROVE
We have all heard the saying:
Tom Peters changed that saying to:
“If it ain’t broke, you aren’t looking hard enough.”
We can’t improve if we don’t do something different. In the “Just Do It” society of the 1990’s, if you weren’t improving, you were falling behind. And the pace of improvement has continued to leap forward in the new millennium.
Sometimes we overlook the need to improve in places that we need to improve the most. One example is our improvement systems. When was the last time you made a comprehensive effort to improve your incident investigations and root cause analysis?
Improvement usually starts by having a clear understanding of where you are. That means you must assess (inspect) your current implementation of your incident investigation system. The audit needs to establish where you are and what areas are in need of improvement.
AREAS TO AUDIT
If we agree that auditing is important to establish where we are before we start to improve, the question then is:
What should we audit?
To answer that question, you need to know what makes an incident investigation system work and then decide how you will audit the important factors.
The first research I would suggest is Chapter 6 of the TapRooT® Book (© 2008). This will give you plenty of ideas of what makes an incident investigation system successful.
Next, I would suggest reviewing Appendix A of the TapRooT® Book. Pay special attention to the sample investigation policy and use it as a reference to compare to your company’s policy.
Next, review Appendix C. It provides 16 topics (33 suggestions) to improve your incident investigation and root cause analysis system. The final suggestion is The Good, The Bad, and The Ugly rating sheet to rate your investigation and root cause analysis system. You can download a copy of an Excel spreadsheet of this rating system at:
Next, review the requirements of your regulator in your country. These will often be “minimum” requirements (for example, the requirements of OSHA’s Process Safety Management regulation. But you obviously should be meeting the government required minimums.
Also, you may have access to your regulators audit guidance. For example, OSHA provides the following guidance for Process Safety Management incident investigations:
“12. Investigation of Incidents. Incident investigation is the process of identifying the underlying causes of incidents and implementing steps to prevent similar events from occurring. The intent of an incident investigation is for employers to learn from past experiences and thus avoid repeating past mistakes. The incidents for which OSHA expects employers to become aware and to investigate are the types of events which result in or could reasonably have resulted in a catastrophic release. Some of the events are sometimes referred to as “near misses,” meaning that a serious consequence did not occur, but could have.“
“Employers need to develop in-house capability to investigate incidents that occur in their facilities. A team needs to be assembled by the employer and trained in the techniques of investigation including how to conduct interviews of witnesses, needed documentation and report writing. A multi-disciplinary team is better able to gather the facts of the event and to analyze them and develop plausible scenarios as to what happened, and why. Team members should be selected on the basis of their training, knowledge and ability to contribute to a team effort to fully investigate the incident. Employees in the process area where the incident occurred should be consulted, interviewed or made a member of the team. Their knowledge of the events form a significant set of facts about the incident which occurred. The report, its findings and recommendations are to be shared with those who can benefit from the information. The cooperation of employees is essential to an effective incident investigation. The focus of the investigation should be to obtain facts, and not to place blame. The team and the investigation process should clearly deal with all involved individuals in a fair, open and consistent manner.“
Also, OSHA provides more minimum guidance on page 23 of this document:
Finally, another place to network and learn best practices to benchmark against your investigation practices is the TapRooT® Summit. Participants praise the new ideas they pick up by networking with some of the “best and brightest” TapRooT® Users from around the world.
Those sources should provide a pretty good checklist for developing your audit protocol.
AUDIT TECHNIQUES (PROTOCOL)
How do you audit the factors that are important to making your incident investigation system work? For each factor you need to develop and audit strategy and audit protocol.
For example, you might decide that sharing of lessons learned with employs and contractors is a vital part of the investigation process. The first step in developing an audit strategy/protocol would be to answer these questions:
- Are there any regulatory requirements for sharing information?
- What is required by our company policy?
- What good practices should we be considering?
Next, you would have to develop a protocol to verify what is actually happening right now at your company. For example, you might:
- Do a paper audit of the practices to see if they meet the requirements.
- Go to the field to verify workers knowledge of past best practices that were shared.
Each factor may have different techniques as part of the audit protocol. These techniques include:
- paperwork reviews
- field observations
- field interviews
- worker tests
- management/supervision interviews
- training and training records reviews
- statistical reviews of investigation results
To have a thorough audit, the auditor needs to go beyond paperwork reviews. For example, reading incident investigation reports and trying to judge their quality can only go so far in assessing the real effectiveness of the incident investigation system. This type of assessment is a part of a broader audit, but should not provide the only basis by which the quality of the system is judged.
For example, a statistical review was performed on the root cause data from over 200 incident investigations at a facility. The reviewer found that there were only two Communication Basic Cause Category root causes in all 200 investigations. This seemed too low. In further review it was found that investigators at this facility were not allowed to interview employees. Instead, they provided their questions to the employee’s supervisor who would then provide the answers at a later date. Is it any surprise that the supervisor never reported a miscommunication between the supervisor and the employee? This problem could not be discovered by an investigation paperwork review.
Don’t forget, you can use TapRooT® to help develop your audit protocol and find the root causes of audit findings. For example, you can flow chart your investigation process as a Spring SnapCharT® to start developing your audit protocol (see Chapter 5 of the 2008 TapRooT® Book for more ideas).
WHO SHOULD AUDIT & WHEN?
We recommend yearly audits of your improvement system. You shouldn’t expect dramatic improvements every year. Rather, if you have been working on improvement for quite some time, you should expect gradual changes that are more obvious after two or three years. This more like measuring a glacier moving than measuring a dragsters movement.
Who should perform these audits?
First, the system’s owner should be doing annual self-assessments. Of course, auditing your own work is difficult. But self-assessments are the foundation of most improvement programs.
Next, at least every three years you should get an outside set of eyes to review your program. This could be a corporate auditor, someone from another site, or an independent (hired) auditor.
System Improvements (the TapRooT® Folks) provides this type of hired audit service (contact us by calling 865-539-2139 or by CLICKING HERE). We bring expertise in TapRooT® and an independent set of eyes. We’ve seen incident investigation systems from around the world in all sorts of industries and have access to the TapRooT® Advisory Board (a committee of industry expert users) that can provide us with unparalleled benchmarking of practices.
GET STARTED NOW
Audits should be an important part of you continuous improvement program. If you aren’t already doing annual audits, the best time to start is NOW! Don’t wait for poor results (when compared to your peers) that make your efforts look bad. Those who are the best are already auditing their system and making improvements. You will have to run hard just to keep up!
(This post is based on the October 1994 Root Cause Network™ Newsletter, Copyright © 1994. Reprinted/adapted by permission. Some modifications have been made to update the article.)
Root Cause Analysis Tip: Rate Your Root Cause Analysis / Incident Investigation System – The Good, The Bad, and The UglySeptember 3rd, 2014 by Mark Paradies
Over a decade ago, I developed a rating sheet for root cause analysis implementation. We had several sessions at the TapRooT® Summit about it and it was posted on our web site (and then our blog). But in the last web site crash, it was lost. Therefore, I’m reposting it here for those who would like to download it. (Just click on the link below.)
Instructions for using the sheet are on the sheet.
I’m working on a new rating system for evaluation of individual incident investigations and corrective actions. Anyone have any ideas they would like to share?
Press Release from the US CSB: CSB Draft Report Finds Deepwater Horizon Blowout Preventer Failed Due to Unrecognized Pipe Buckling Phenomenon During Emergency Well-Control Efforts on April 20, 2010, Leading to Environmental Disaster in Gulf of MexicoJune 5th, 2014 by Mark Paradies
CSB Draft Report Finds Deepwater Horizon Blowout Preventer Failed Due to Unrecognized Pipe Buckling Phenomenon During Emergency Well-Control Efforts on April 20, 2010, Leading to Environmental Disaster in Gulf of Mexico
Report Says Similar Accident Could Still Occur, Calls for Better Management
of Safety-Critical Elements by Offshore Industry, Regulators
Houston, Texas, June 5, 2014— The blowout preventer (BOP) that was intended to shut off the flow of high-pressure oil and gas from the Macondo well in the Gulf of Mexico during the disaster on the Deepwater Horizon drilling rig on April 20, 2010, failed to seal the well because drill pipe buckled for reasons the offshore drilling industry remains largely unaware of, according to a new two-volume draft investigation report released today by the U.S. Chemical Safety Board (CSB).
The blowout caused explosions and a fire on the Deepwater Horizon rig, leading to the deaths of 11 personnel onboard and serious injuries to 17 others. Nearly 100 others escaped from the burning rig, which sank two days later, leaving the Macondo well spewing oil and gas into Gulf waters for a total of 87 days. By that time the resulting oil spill was the largest in offshore history. The failure of the BOP directly led to the oil spill and contributed to the severity of the incident on the rig.
The draft report will be considered for approval by the Board at a public meeting scheduled for 4 p.m. CDT at the Hilton Americas Hotel, 1600 Lamar St., Houston, TX 77010. The meeting will include a detailed staff presentation, Board questions, and public comments, and will be webcast at:
The CSB report concluded that the pipe buckling likely occurred during the first minutes of the blowout, as crews desperately sought to regain control of oil and gas surging up from the Macondo well. Although other investigations had previously noted that the Macondo drill pipe was found in a bent or buckled state, this was assumed to have occurred days later, after the blowout was well underway.
After testing individual components of the blowout preventer (BOP) and analyzing all the data from post-accident examinations, the CSB draft report concluded that the BOP’s blind shear ram – an emergency hydraulic device with two sharp cutting blades, intended to seal an out-of-control well – likely did activate on the night of the accident, days earlier than other investigations found. However, the pipe buckling that likely occurred on the night of April 20 prevented the blind shear ram from functioning properly. Instead of cleanly cutting and sealing the well’s drill pipe, the shear ram actually punctured the buckled, off-center pipe, sending huge additional volumes of oil and gas surging toward the surface and initiating the 87-day-long oil and gas release into the Gulf that defied multiple efforts to bring it under control.
The identification of the new buckling mechanism for the drill pipe – called “effective compression” – was a central technical finding of the draft report. The report concludes that under certain conditions, the “effective compression” phenomenon could compromise the proper functioning of other blowout preventers still deployed around the world at offshore wells. The complete BOP failure scenario is detailed in a new 11-minute computer video animation the CSB developed and released along with the draft report.
The CSB draft report also revealed for the first time that there were two instances of mis-wiring and two backup battery failures affecting the electronic and hydraulic controls for the BOP’s blind shear ram. One mis-wiring, which led to a battery failure, disabled the BOP’s “blue pod” – a control system designed to activate the blind shear ram in an emergency. The BOP’s “yellow pod” – an identical, redundant system that could also activate the blind shear ram – had a different miswiring and a different battery failure. In the case of the yellow pod, however, the two failures fortuitously cancelled each other out, and the pod was likely able to operate the blind shear ram on the night of April 20.
“Although both regulators and the industry itself have made significant progress since the 2010 calamity, more must be done to ensure the correct functioning of blowout preventers and other safety-critical elements that protect workers and the environment from major offshore accidents,” said Dr. Rafael Moure-Eraso, the CSB chairperson. “The two-volume report we are releasing today makes clear why the current offshore safety framework needs to be further strengthened.”
“Our investigation has produced several important findings that were not identified in earlier examinations of the blowout preventer failure,” said CSB Investigator Cheryl MacKenzie, who led the investigative team. “The CSB team performed a comprehensive examination of the full set of BOP testing data, which were not available to other investigative organizations when their various reports were completed. From this analysis, we were able to draw new conclusions about how the drill pipe buckled and moved off-center within the BOP, preventing the well from being sealed in an emergency.”
The April 2010 blowout in the Gulf of Mexico occurred during operations to “temporarily abandon” the Macondo oil well, located in approximately 5,000-foot-deep waters some 50 miles off the coast of Louisiana. Mineral rights to the area were leased to oil major BP, which contracted with Transocean and other companies to drill the exploratory Macondo well under BP’s oversight, using Transocean’s football-field-size Deepwater Horizon drilling rig.
The blowout followed a failure of the cementing job to temporarily seal the well, while a series of pressure tests were misinterpreted to indicate that the well was in fact properly sealed. The final set of failures on April 20 involved the Deepwater Horizon’s blowout preventer (BOP), a large and complex device on the sea floor that was connected to the rig nearly a mile above on the sea surface.
Effective compression, as described in the draft report, occurs when there is a large pressure difference between the inside and outside of a pipe. That condition likely occurred during emergency response actions by the Deepwater Horizon crew to the blowout occurring on the night of April 20, when operators closed BOP pipe rams at the wellhead, temporarily sealing the well. This unfortunately established a large pressure differential that buckled the steel drill pipe inside the BOP, bending it outside the effective reach of the BOP’s last-resort safety device, the blind shear ram.
“The CSB’s model differs from other buckling theories that have been presented over the years but for which insufficient supporting evidence has been produced,” according to CSB Investigator Dr. Mary Beth Mulcahy, who oversaw the technical analysis. “The CSB’s conclusions are based on real-time pressure data from the Deepwater Horizon and calculations about the behavior of the drill pipe under extreme conditions. The findings reveal that pipe buckling could occur even when a well is shut-in and apparently in a safe and stable condition. The pipe buckling – unlikely to be detected by the drilling crew – could render the BOP inoperable in an emergency. This hazard could impact even the best offshore companies, those who are maintaining their blowout preventers and other equipment to a high standard. However, there are straightforward methods to avoid pipe buckling if you recognize it as a hazard.”
The CSB investigation found that while Deepwater Horizon personnel performed regular tests and inspections of those BOP components that were necessary for day-to-day drilling operations, neither Transocean nor BP had performed regular inspections or testing to identify latent failures of the BOP’s emergency systems. As a result, the safety-critical BOP systems responsible for shearing drill pipe in emergency situations – and safely sealing an out-of-control well – were compromised before the BOP was even deployed to the Macondo wellhead. The CSB report pointed to the multiple miswirings and battery failures within the BOP’s subsea control equipment as evidence of the need for more rigorous identification, testing, and management of critical safety devices. The report also noted that the BOP lacked the capacity to reliably cut and seal the 6-5/8 inch drill pipe that was used during most of the drilling at the Macondo well prior to April 20 – even if the pipe had been properly centered in the blind shear ram’s blades.
Despite the multiple maintenance problems found in the Deepwater Horizon BOP, which could have been detected prior to the accident, CSB investigators ultimately concluded the blind shear ram likely did close on the night of April 20, and the drill pipe could have been successfully sealed but for the buckling of the pipe.
“Although there have been regulatory improvements since the accident, the effective management of safety critical elements has yet to be established,” Investigator MacKenzie said. “This results in potential safety gaps in U.S. offshore operations and leaves open the possibility of another similar catastrophic accident.”
The draft report, subject to Board approval, makes a number of recommendations to the U.S. Department of Interior’s Bureau of Safety and Environmental Enforcement (BSEE), the federal organization established following the Macondo accident to oversee U.S. offshore safety. These recommendations call on BSEE to require drilling operators to effectively manage technical, operational, and organizational safety-critical elements in order to reduce major accident risk to an acceptably low level, known as “as low as reasonably practicable.”
“Although blowout preventers are just one of the important barriers for avoiding a major offshore accident, the specific findings from the investigation about this BOP’s unreliability illustrate how the current system of regulations and standards can be improved to make offshore operations safer,” Investigator MacKenzie said. “Ultimately the barriers against a blowout or other offshore disaster include not only equipment like the BOP, but also operational and organizational factors. And all of these need to be rigorously defined, actively monitored, and verified through an effective management system if safety is to be assured.” Companies should be required to identify these safety-critical elements in advance, define their performance requirements, and prove to the regulator and outside auditors that these elements will perform reliably when called upon, according to the draft report.
The report also proposes recommendations to the American Petroleum Institute (API), the U.S. trade association for both upstream and downstream petroleum industry. The first recommendation is to revise API Standard 53, Blowout Prevention Equipment Systems for Drilling Wells, calling for critical testing of the redundant control systems within BOP’s, and another for new guidance for the effective management of safety-critical elements in general.
CSB Chairperson Rafael Moure-Eraso said, “Drilling continues to extend to new depths, and operations in increasingly challenging environments, such as the Arctic, are being planned. The CSB report and its key findings and recommendations are intended to put the United States in a leading role for improving well-control procedures and practices. To maintain a leadership position, the U.S. should adopt rigorous management methods that go beyond current industry best practices.”
Two forthcoming volumes of the CSB’s Macondo investigation report are planned to address additional regulatory matters as well as organizational and human factors safety issues raised by the accident.
Do you want a World-Class Improvement Program? Then read “Tide and Time Wait for No Man” on page 1 of this month’s Root Cause Network™ Newsletter. Download your copy of the newsletter by clicking on this link:
- 5 Ways to Improve Your Interviews (Page 2)
- Best Practice from the 2014 Global TapRooT® Summit: The TapRooT® Expert Help Desk (Page 2)
- How things naturally go from “Excellence to Complacency” (Page 2)
- A new idea … “Budget for Your Next Accident” (Page 3)
- Dilbert Joke (Page 3)
- An answer to “Is Human Error a Root Cause?” (Page 3)
- A list of upcoming public TapRooT® Courses – Is one near you? (Page 4)
(Photo of remains from cockpit fire of an Egypt Air 777 while parked at a gate in Cairo)
One of our TapRooT® Users sent the attached PDF of a SnapCharT® for the loss of Malaysia Air MH 370.
Have a look. See what you think. Then leave comments here…
How do you know if your root cause analysis is adequate? Read the article on page 3 of the March Root Cause Network™ Newsletter and find out! Download your copy of the newsletter by clicking on this link: Mar14NL120.pdf
.What else can you learn in this edition?
- What’s Right and What’s Wrong with Human Performance Tools (Page 1)
- Why Do Supervisors Produce Bad Investigations? (Page 2)
- How Should You Target Your Investigations? (Page 2)
- What’s Wrong with Your Trending? (Page 2)
- Admiral Rickover’s Face-the-Facts Philosophy (Page 2)
- Proactive Use of TapRooT® (Page 3)
- Stop Slips, Trips, and Falls (Page 3)
- Risk Management Best Practices (Page 3)
- Upcoming TapRooT® Courses Around the World (Page 4)
- What Can You Learn at the 2014 Global TapRooT® Summit? (Page 5, 6, & 7)
Plus there’s more! An article you really should read and act upon. See the article on Page 3: “Are you Missing an Important Meeting?”
Why should you read that article among all the others? Here’s the first paragraph …
“What if you missed a meeting and it caused someone to die. Or maybe you lost your job if you weren’t there? Or your company lost millions of dollars because you simply didn’t attend a three-day meeting. Would you make sure that you were there?”
If those questions don’t grab your attention, what will?
Go to this link:
Print the March Root Cause Network™ Newsletter and read it from cover to cover!
You’ll be glad you did. (And you’ll find that there are several actions you will be compelled to take.)
Click on the image above for a PowerPoint of the temperature alarms during reentry.
The following is the text of a speech delivered in 1982 by Admiral Hyman G. Rickover – the father of the Nuclear Navy – at Columbia University. Rickover’s accomplishments as the head of the Nuclear Navy are legendary. From developing the first power producing submarine based nuclear reactor from scratch to operations in just three years to creating a program to guarantee process safety (nuclear safety) for over 60 years (zero nuclear accidents).
I am reprinting this speech here because I believe that many do not understand the management concepts needed to guarantee process safety. We teach these concepts in our “Reducing Serious Injuries and Fatalities Using TapRooT®” pre-Summit course. Since many won’t be able to attend this training, I wanted to give all an opportunity to learn these valuable lessons by posting this speech.
– – –
Human experience shows that people, not organizations or management systems, get things done. For this reason, subordinates must be given authority and responsibility early in their careers. In this way they develop quickly and can help the manager do his work. The manager, of course, remains ultimately responsible and must accept the blame if subordinates make mistakes.
As subordinates develop, work should be constantly added so that no one can finish his job. This serves as a prod and a challenge. It brings out their capabilities and frees the manager to assume added responsibilities. As members of the organization become capable of assuming new and more difficult duties, they develop pride in doing the job well. This attitude soon permeates the entire organization.
One must permit his people the freedom to seek added work and greater responsibility. In my organization, there are no formal job descriptions or organizational charts. Responsibilities are defined in a general way, so that people are not circumscribed. All are permitted to do as they think best and to go to anyone and anywhere for help. Each person then is limited only by his own ability.
Complex jobs cannot be accomplished effectively with transients. Therefore, a manager must make the work challenging and rewarding so that his people will remain with the organization for many years. This allows it to benefit fully from their knowledge, experience, and corporate memory.
The Defense Department does not recognize the need for continuity in important jobs. It rotates officer every few years both at headquarters and in the field. The same applies to their civilian superiors.
This system virtually ensures inexperience and nonaccountability. By the time an officer has begun to learn a job, it is time for him to rotate. Under this system, incumbents can blame their problems on predecessors. They are assigned to another job before the results of their work become evident. Subordinates cannot be expected to remain committed to a job and perform effectively when they are continuously adapting to a new job or to a new boss.
When doing a job—any job—one must feel that he owns it, and act as though he will remain in the job forever. He must look after his work just as conscientiously, as though it were his own business and his own money. If he feels he is only a temporary custodian, or that the job is just a stepping stone to a higher position, his actions will not take into account the long-term interests of the organization. His lack of commitment to the present job will be perceived by those who work for him, and they, likewise, will tend not to care. Too many spend their entire working lives looking for their next job. When one feels he owns his present job and acts that way, he need have no concern about his next job.
In accepting responsibility for a job, a person must get directly involved. Every manager has a personal responsibility not only to find problems but to correct them. This responsibility comes before all other obligations, before personal ambition or comfort.
A major flaw in our system of government, and even in industry, is the latitude allowed to do less than is necessary. Too often officials are willing to accept and adapt to situations they know to be wrong. The tendency is to downplay problems instead of actively trying to correct them. Recognizing this, many subordinates give up, contain their views within themselves, and wait for others to take action. When this happens, the manager is deprived of the experience and ideas of subordinates who generally are more knowledgeable than he in their particular areas.
A manager must instill in his people an attitude of personal responsibility for seeing a job properly accomplished. Unfortunately, this seems to be declining, particularly in large organizations where responsibility is broadly distributed. To complaints of a job poorly done, one often hears the excuse, “I am not responsible.” I believe that is literally correct. The man who takes such a stand in fact is not responsible; he is irresponsible. While he may not be legally liable, or the work may not have been specifically assigned to him, no one involved in a job can divest himself of responsibility for its successful completion.
Unless the individual truly responsible can be identified when something goes wrong, no one has really been responsible. With the advent of modern management theories it is becoming common for organizations to deal with problems in a collective manner, by dividing programs into subprograms, with no one left responsible for the entire effort. There is also the tendency to establish more and more levels of management, on the theory that this gives better control. These are but different forms of shared responsibility, which easily lead to no one being responsible—a problems that often inheres in large corporations as well as in the Defense Department.
When I came to Washington before World War II to head the electrical section of the Bureau of Ships, I found that one man was in charge of design, another of production, a third handled maintenance, while a fourth dealt with fiscal matters. The entire bureau operated that way. It didn’t make sense to me. Design problems showed up in production, production errors showed up in maintenance, and financial matters reached into all areas. I changed the system. I made one man responsible for his entire area of equipment—for design, production, maintenance, and contracting. If anything went wrong, I knew exactly at whom to point. I run my present organization on the same principle.
A good manager must have unshakeable determination and tenacity. Deciding what needs to be done is easy, getting it done is more difficult. Good ideas are not adopted automatically. They must be driven into practice with courageous impatience. Once implemented they can be easily overturned or subverted through apathy or lack of follow-up, so a continuous effort is required. Too often, important problems are recognized but no one is willing to sustain the effort needed to solve them.
Nothing worthwhile can be accomplished without determination. In the early days of nuclear power, for example, getting approval to build the first nuclear submarine—the Nautilus—was almost as difficult as designing and building it. Many in the Navy opposed building a nuclear submarine.
In the same way, the Navy once viewed nuclear-powered aircraft carriers and cruisers as too expensive, despite their obvious advantages of unlimited cruising range and ability to remain at sea without vulnerable support ships. Yet today our nuclear submarine fleet is widely recognized as our nation’s most effective deterrent to nuclear war. Our nuclear-powered aircraft carriers and cruisers have proven their worth by defending our interests all over the world—even in remote trouble spots such as the Indian Ocean, where the capability of oil-fired ships would be severely limited by their dependence on fuel supplies.
The man in charge must concern himself with details. If he does not consider them important, neither will his subordinates. Yet “the devil is in the details.” It is hard and monotonous to pay attention to seemingly minor matters. In my work, I probably spend about ninety-nine percent of my time on what others may call petty details. Most managers would rather focus on lofty policy matters. But when the details are ignored, the project fails. No infusion of policy or lofty ideals can then correct the situation.
To maintain proper control one must have simple and direct means to find out what is going on. There are many ways of doing this; all involve constant drudgery. For this reason those in charge often create “management information systems” designed to extract from the operation the details a busy executive needs to know. Often the process is carried too far. The top official then loses touch with his people and with the work that is actually going on.
Attention to detail does not require a manager to do everything himself. No one can work more than twenty-four hours each day. Therefore to multiply his efforts, he must create an environment where his subordinates can work to their maximum ability. Some management experts advocate strict limits to the number of people reporting to a common superior—generally five to seven. But if one has capable people who require but a few moments of his time during the day, there is no reason to set such arbitrary constraints. Some forty key people report frequently and directly to me. This enables me to keep up with what is going on and makes it possible for them to get fast action. The latter aspect is particularly important. Capable people will not work for long where they cannot get prompt decisions and actions from their superior.
I require frequent reports, both oral and written, from many key people in the nuclear program. These include the commanding officers of our nuclear ships, those in charge of our schools and laboratories, and representatives at manufacturers’ plants and commercial shipyards. I insist they report the problems they have found directly to me—and in plain English. This provides them unlimited flexibility in subject matter—something that often is not accommodated in highly structured management systems—and a way to communicate their problems and recommendations to me without having them filtered through others. The Defense Department, with its excessive layers of management, suffers because those at the top who make decisions are generally isolated from their subordinates, who have the first-hand knowledge.
To do a job effectively, one must set priorities. Too many people let their “in” basket set the priorities. On any given day, unimportant but interesting trivia pass through an office; one must not permit these to monopolize his time. The human tendency is to while away time with unimportant matters that do not require mental effort or energy. Since they can be easily resolved, they give a false sense of accomplishment. The manager must exert self-discipline to ensure that his energy is focused where it is truly needed.
All work should be checked through an independent and impartial review. In engineering and manufacturing, industry spends large sums on quality control. But the concept of impartial reviews and oversight is important in other areas also. Even the most dedicated individual makes mistakes—and many workers are less than dedicated. I have seen much poor work and sheer nonsense generated in government and in industry because it was not checked properly.
One must create the ability in his staff to generate clear, forceful arguments for opposing viewpoints as well as for their own. Open discussions and disagreements must be encouraged, so that all sides of an issue will be fully explored. Further, important issues should be presented in writing. Nothing so sharpens the thought process as writing down one’s arguments. Weaknesses overlooked in oral discussion become painfully obvious on the written page.
When important decisions are not documented, one becomes dependent on individual memory, which is quickly lost as people leave or move to other jobs. In my work, it is important to be able to go back a number of years to determine the facts that were considered in arriving at a decision. This makes it easier to resolve new problems by putting them into proper perspective. It also minimizes the risk of repeating past mistakes. Moreover if important communications and actions are not documented clearly, one can never be sure they were understood or even executed.
It is a human inclination to hope things will work out, despite evidence or doubt to the contrary. A successful manager must resist this temptation. This is particularly hard if one has invested much time and energy on a project and thus has come to feel possessive about it. Although it is not easy to admit what a person once thought correct now appears to be wrong, one must discipline himself to face the facts objectively and make the necessary changes—regardless of the consequences to himself. The man in charge must personally set the example in this respect. He must be able, in effect, to “kill his own child” if necessary and must require his subordinates to do likewise. I have had to go to Congress and, because of technical problems, recommended terminating a project that had been funded largely on my say-so. It is not a pleasant task, but one must be brutally objective in his work.
No management system can substitute for hard work. A manager who does not work hard or devote extra effort cannot expect his people to do so. He must set the example. The manager may not be the smartest or the most knowledgeable person, but if he dedicates himself to the job and devotes the required effort, his people will follow his lead.
The ideas I have mentioned are not new—previous generations recognized the value of hard work, attention to detail, personal responsibility, and determination. And these, rather than the highly-touted modern management techniques, are still the most important in doing a job. Together they embody a common-sense approach to management, one that cannot be taught by professors of management in a classroom.
I am not against business education. A knowledge of accounting, finance, business law, and the like can be of value in a business environment. What I do believe is harmful is the impression often created by those who teach management that one will be able to manage any job by applying certain management techniques together with some simple academic rules of how to manage people and situations.
The CSB has asked for comments on their 2012-2016 Strategic Plan by April 12, 2012. You can see the plan at:
You can send your comments by email to firstname.lastname@example.org.
My thought was that the strategic plan needed more specifics as far as the strategies to be adopted. For example, there were five sub-goals under “Goal 3: Preserve the public trust by maintaining and improving organizational excellence.” One of these was “Maintain effective human capital management by promoting development in leadership, technical, and analytical competencies.”
The document referred to several other plans, (the “CSB Human Capital Plan for Fiscal Years 2011 – 2015,” the “Office of Personnel Management (OPM) Workforce Planning Model,” and the “OPM Strategic Leadership Management Model”), that were not readily available on the CSB web site or by a link in the Strategic Plan. Thus, the specifics of the plan are largely unknown and unknowable.
For example, I would like to see an easy link to the qualification requirements and training program for CSB investigators. This could be helpful so that others could see what the CSB considers an adequately qualified investigator.
It would also be interesting for the CSB to detail what they are doing to learn industry best practices for root cause analysis, interviewing, corrective action development, and accident prevention. But these details are not easily available.
Also, I would think that many TapRooT® Users would suggest that the CSB have a core of investigators familiar with the TapRooT® System that is used extensively across the chemical, petrochemical, and oil industry. This would help them interface with industry personnel and provide them the knowledge they need to evaluate industry incident reports produced using TapRooT®.
Finally, I would also think that TapRooT® Users would like to see continued participation of the CSB in ongoing TapRooT® Summits where industry best practices about root cause analysis and accident prevention are shared. Participation in the TapRooT® Summit and other industry conferences should be spelled out as part of the strategy to keep up with the state-of-the-art in the chemical industry.
One other item that deserves comments is the timeliness of CSB accident reports. Frequently, these reports are more than a year after the accident. Important investigations, (for example, BP Texas City, and the still unreleased BP Deepwater Horizon investigations), take more than two years. By the time the investigations are released, the industry has already implemented corrective actions and moved along. I would like to see a specific strategy/plan to improve the timeliness of investigations to avoid late investigations that have limited industry impact because of their tardiness.
One final thought … because having an effective independent evaluation of major accidents is so important, I highly recommend that readers in the chemical industry take the time to read the CSB Strategic Plan and provide your comments before April 12. You can’t complain about the outcome if you don’t comment when asked.
Monday Accident & NOT Lessons Learned: Under-Reporting of Sentinel Events May Be One More Cause of Failure to Prevent Human Errors in the Healthcare SettingFebruary 20th, 2012 by Mark Paradies
A new investigation by the Inspector General of the Department of Health and Human Services says that:
“Hospital employees recognize and report only one out of seven errors, accidents and other events that harm Medicare patients while they are hospitalized.”
The report also says that:
“…even after hospitals investigate preventable injuries and infections that have been reported, they rarely change their practices to prevent repetition of the ‘adverse events.’”
For the whole report, see:
So, only a small fraction of sentinel events are investigated and most of those don’t cause permanent, effective change to prevent future errors.
Sometimes it can be frustrating to be a prophet when those that could make change happen just don’t listen. We’ve been suggesting proven ways to improve sentinel event investigation and performance improvement that could be applied by medical facilities ever sine the 1994 TapRooT® Summit. But only a limited number of healthcare facilities have taken advantage of the lessons they could learn.
The TapRooT® Summit is coming up on February 29 – March 2 and we have a full track devoted to improving performance in the healthcare industry. This isn’t just lessons from inside the industry. Rather, this is a place where healthcare folks can learn from a wide variety of industries and facilities with best practices from around-the-world.
If you are from a healthcare facility that needs to improve (and from the Inspector General’s report, that’s just about every facility) there’s still just enough time to sign up. See:
I wrote this paper for the for the BARQA Journal and they are nice enough to let me republish it here. Click on the pdf below to see the whole article.
The article is written for people interested in root cause analysis to improve pharmaceutical quality, but the problems discussed are common to all industries and apply to those looking to improve safety, operation, maintenance, process safety, and quality.
Sources of Root Cause Analysis Failures by Mark Paradies is published by:
Quasar (Members Magazine of BARQA, British Association of Research Quality Assurance) No. 118 Pages 7 – 10, Jan 2012.
Used by Permission.
If you have already signed up for the TapRooT® Summit, have you made sure that others in your company who should be there have signed up?
Here’s a link to the TapRooT® Summit Brochure that you can pass along to them as a reminder …
The Rail Accident Investigation Branch (RAIB) has released its annual report for the operational period 2010. It is published in two sections; Section 1 covers the work of the RAIB in 2010, and Section 2 cover the status of recommendations as reported to RAIB. During 2010, the RAIB published 20 investigation reports, a Special Investigation report, 11 bulletins and started a further 18 investigations.
To see the report, go to:
In order to help the UK RAIB shape the next report, please complete the feedback questionnaire available at the following link:
Press Release from the US Chemical Safety Board: CSB Report Finds that Oil and Gas Exploration and Production Facilities Present Hazards to Members of the Public, Especially ChildrenOctober 27th, 2011 by Mark Paradies
CSB Issues Recommendations to EPA, State Regulators, NFPA, and API Aimed at Increasing Oil Site Safety and Security
Hattiesburg, MS, October 27, 2011 – The U.S. Chemical Safety Board (CSB) today released a new study of explosions at oil and gas production sites across the U.S., identifying 26 incidents since 1983 that killed 44 members of the public and injured 25 others under the age of 25, and is calling for new public protection measures at the sites.
The report examined in detail three explosions that occurred at oil and gas production facilities in Mississippi, Oklahoma, and Texas, that killed and injured members of the public between October 2009 and April 2010.
The CSB report found that children and young adults frequently socialize at oil sites in rural areas, unaware of the explosion hazards from storage tanks that contain flammable hydrocarbons like crude oil and natural gas condensate. The unintentional introduction of an ignition source (such as a match, lighter, cigarette, or static electricity) near tank hatches or vents can trigger an internal tank explosion, often launching the tank into the air and killing or injuring people nearby. The report identified regulatory gaps at the federal and state levels and called on the U.S. Environmental Protection Agency (EPA) and state regulatory bodies to improve current safety and security measures at exploration and production sites such as warning signs, full fencing, locked gates, locks on tank hatches, and other physical barriers. The report also called on state regulators in Mississippi, Oklahoma, and Texas to require safer, modern tank designs that reduce the likelihood of an internal tank explosion if an ignition source is inadvertently introduced nearby.
On October 31, 2009, two teenagers, aged 16 and 18, were killed when a storage tank containing natural gas condensate exploded at a rural gas production site in Carnes, Mississippi. Six months later a group of youths were exploring a similar tank site in Weleetka, Oklahoma, when an explosion and fire fatally injured one individual. Two weeks later, a 25-year-old man and a 24-year-old woman were on top of an oil tank in rural New London, Texas, when the tank exploded, killing the woman and seriously injuring the man. The CSB deployed investigators to all three sites to collect information on the incidents.
Investigators found that the three accidents occurred in isolated, rural wooded areas at production sites that were unfenced, did not have clear or legible warning signs and did not have hatch locks to prevent access to the flammable hydrocarbons inside the tanks.
CSB Chairman Rafael Moure-Eraso said, “After reviewing the work of our investigators I believe that these incidents were entirely preventable. Basic security measures and warning signs – as well as more safely designed storage tanks – will essentially prevent kids from being killed in tank explosions at these sites.”
The CSB’s investigation found a few major cities and some states, such as California and Ohio, already require varying levels of security for oil and gas production sites, such as fencing, locked or sealed tank hatches, and warning signs. As a result, California did not appear to have any fatal tank explosions between 1983 and 2011. However, many other large oil and gas producing states have no such requirements. The major oil producing states Texas and Oklahoma require fencing and warning signs for certain sites that have toxic gas hazards but not for all sites with flammable storage tanks.
“Oil and gas storage sites are part of the landscape in many rural American communities; hundreds of thousands of similar sites are located across the country,” said CSB Lead Investigator Vidisha Parasram. “It was a concern to discover that issues related to public safety are rarely considered prior to placement and design of these sites. In many cases sites can be as close as 150 to 300 feet from existing buildings such as residences, schools, and churches, and still lack any meaningful warnings or barriers to prevent public access.”
Among the six formal safety recommendations in the report, the Board urged that state regulators require the use of inherently safety tank design features such as flame arrestors, pressure-vacuum vents, floating roofs, and vapor recovery systems. The safety measures, which are similar to those already in use in refineries and other downstream storage tanks, reduce the emissions of flammable vapor from the tanks or otherwise prevent an external flame from igniting vapor inside tanks.
“The goal of this investigative study is to issue recommendations that will effectively address the current gaps that exist at the state and federal level, said Dr. Moure-Eraso. “As I have seen firsthand, these sites can be dangerous to the people who live and work in these communities and should be properly designed and protected.”
The Board recommended that EPA issue a safety bulletin warning of the explosion hazards of storage tanks, describe the importance of increased security measures such as fencing, gates and signs, and recommend the use of inherently safer storage tank design. Similarly, the CSB’s recommendations seek to address the current gaps in regulations and codes in Mississippi, Oklahoma and Texas.
The CSB’s investigation also examined industry codes and standards, such as those from the American Petroleum Institute (API) and the National Fire Protection Association (NFPA). The final report recommends that both organizations adequately address the hazards that these sites present to members of the public through amendments to their existing codes or creation of additional guidance.
As a result of the investigation’s findings the CSB recommended that API warn of the explosion hazards presented by exploration and production sites, including requirements for security measures such as fencing gates and signs, recommendations for inherently safer storage tank design and acknowledgment of the public safety issue presented by these sites. Similarly the CSB recommended that NFPA amend NFPA 30 “Storage of Liquids in Tanks – Requirements for all Storage Tanks” to adequately describe unmanned extraction and production sites and include information in a relevant security standard that offers specifications on fencing and locks.
Chairman Moure-Eraso said, “As the demand for domestic energy resources continues to grow and the number of active extraction and production sites continues to rise steadily, it is important to ensure that these sites have the appropriate safeguards to save young people’s lives.”
On April 13, 2011, the CSB held a news conference and public meeting in Hattiesburg to release the safety video “No Place to Hang Out: The Danger of Oil Well Sites.”The video is aimed at educating young people about the hazards associated with oil storage tanks. In the video the CSB interviewed teenagers and adults who stated that it is a common practice in rural areas for young people to hang out and socialize at oil production sites.
To view the CSB’s final report click here.
To view the CSB’s safety video “No Place to Hang Out: The Danger of Oil Well Sites” click here.
The CSB is an independent federal agency charged with investigating serious chemical accidents. The agency’s board members are appointed by the president and confirmed by the Senate. CSB investigations look into all aspects of chemical accidents, including physical causes such as equipment failure as well as inadequacies in safety management systems. The Board does not issue citations or fines but does make safety recommendations to companies, industry organizations, labor groups, and regulatory agencies such as OSHA and EPA. Please visit our website, www.CSB.gov.
Contact: For more information, contact Communications Manager Hillary Cohen at 202-446-8094 or via email at Hillary.Cohen@csb.gov or Sandy Gilmour at 202-261-7614 or cell 202-251-5496 or via email at Sandy.Gilmour@csb.gov.
The UK Rail Accident Investigation Branch has published a report about a train derailment in Summit Tunnel, near Todmorden, West Yorkshire, UK.
Here’s the summary from their press release:
In the early hours of 28 December 2010, a passenger train was travelling from Manchester to Leeds when it struck a large amount of ice that had fallen onto the tracks from a ventilation shaft in Summit tunnel. All wheels of the front bogie were derailed to the left in the direction of travel causing the front driving cab of the train to strike the tunnel wall. The train remained upright and once it had stopped, the train crew took action to protect the train and raise the alarm. About three hours later, the passengers and train crew had been led out of the tunnel by the emergency services. No injuries were reported, while the train suffered damage to its cab windscreen, a coupler, bodywork and underframe. There was minor damage to the track.
The ice formed as water, seeping through the lining of a ventilation shaft, froze during a long period of freezing temperatures. This ice fell onto the track after a thaw which started on 27 December 2010. The train, which was the first to pass through the tunnel in over three days due to the Christmas holiday period, then collided with it. A combination of factors led to this accident:
- the risk of ice, particularly ice falls onto the track, was not identified before the train service resumed so the train was allowed to enter Summit tunnel while running at its maximum permitted speed; and
- the routine maintenance regime did not identify excessive ice in the tunnel and no additional inspections were carried out.
The RAIB has made five recommendations, all directed to Network Rail.
The first recommendation relates to how water in Summit tunnel is managed.
The second is about identifying those structures which are at risk from extreme weather and then checking they are safe to use after periods when no trains have been running.
The third calls for the potential hazards due to extreme weather and thaw conditions to be taken into account in Network Rail’s weather management processes.
The fourth calls for training and information to be given to staff who need to carry out the additional inspection of structures that are at risk in extreme cold weather.
The fifth relates to the management of safety related information (and details of actions taken) that is passed from Network Rail’s buildings and civils – asset management function to other parts of the company.
For the complete report and lessons learned, see:
Here’s a link to a summary on the NTSB web site:
Here’s the report:
Here is a CBS story about the report:
Here’s video of the NTSB press conference:
Circadian Technologies has published a white paper titled:
The Advantages & Disadvantages of 12-Hour Shifts:
A Balanced Perspective
Their press release states:
“12-hour shifts remain a much-debated topic in 24-hour operations. Do they cost more than 8-hour shifts? Are they safe? What impact do they have on alertness, health and productivity?
CIRCADIAN®, the global leader in providing 24/7 workforce performance and safety solutions for businesses that operate around the clock, has collected considerable data on the benefits and complications of 12-hour shifts. The goal of this white paper is to provide you with a balanced perspective of 12-hour shifts – one that will examine the pros and cons from both a management and shiftworker perspective.”
Here’s the link to register to receive this report:
If you are interested in preventing fatigue and developing a fatigue risk management program, sign up for the:
being held on February 27-28, 2012, in Las Vegas just prior to the TapRooT® Summit.
This training, being provided by Circadian Technologies, will help you:
• Design and implement a cost-effective Fatigue-Risk Management System
• Assess the risks and costs of fatigue in your business
• Determine safe staffing levels and optimal shift/duty patterns for your operation
• Train employees and supervisors to mitigate fatigue risk
• Improve employee health, safety, and quality of life
Also, the Summit (February 29 – March 2 in Las Vegas) has two sessions on fatigue as well as other sessions on improving human performance. Don’t miss it!
Here’s a link to the report:
Here’s a link to The Wall Street Journal article about the report:
I’ve just started reading the report so … no comments for now.
OSHA has an on-line calculator to help you find the total cost of an accident. See it at:
This calculator is very similar to the rough estimate provided in the TapRooT® Book.
For more information on the basis for this calculator, see:
Monday Accident & Lessons Learned: UK RAIB investigation report on the safety of automatic open level crossingsSeptember 5th, 2011 by Mark Paradies
From the report:
Following the fatal accident at Halkirk automatic open level crossing, Caithness, on 29 September 2009, the RAIB decided to carry out two separate investigations. The first of these was into the Halkirk accident1, while the second was to investigate the more general safety issues associated with automatic open level crossings installed on Network Rail’s managed infrastructure. This report addresses the more general safety issues.
The RAIB’s investigation confirmed that automatic open level crossings, which are protected only by road traffic light signals, and have no barriers, are the highest risk form of level crossing for vehicle drivers on public roads, and some of them have a significant history of incidents and accidents.
The investigation found that the lack of barriers at automatic open level crossings is the most significant factor contributing to vehicle drivers passing the road traffic light signals when they are operating, either deliberately or as a genuine error. The RAIB considers that the crossings with the highest risk of collision between trains and road vehicles should be upgraded, probably by fitting half barriers, but there may be other means which deliver an equivalent or better level of safety (eg closure).
The high cost of new level crossings is a reason why it can be difficult to justify upgrading existing crossings based on a cost benefit analysis. However, a system is being developed to retro-fit half barriers to existing automatic open crossings at a much lower cost than that of a new crossing. If this initiative is successful, it will be easier to justify the upgrade of existing crossings. The RAIB believes that this work should be prioritised accordingly.
The safety of level crossings can be improved by taking action against vehicle drivers who deliberately pass the flashing red lights. Where this behaviour is prevalent, red light enforcement equipment is a deterrent. The RAIB believes that the development of fixed digital cameras and their installation at selected level crossings, particularly in combination with greater penalties, would be beneficial in improving safety and should be prioritised.
The identification of factors at each crossing that lead to deliberate risk taking behaviour or genuine errors would enable appropriate risk reduction measures to be implemented. The RAIB believes that the existing risk assessments of automatic open level crossings should be reviewed to check whether all the relevant factors have been identified, and to determine whether additional mitigation measures are required.
Finally, the RAIB believes that Network Rail’s process covering the risk assessment of level crossings should include guidance to its staff on how to identify the relevant human factors, and take account of the associated risk, at specific level crossings in order to determine the adequacy of existing mitigation measures and the need for additional measures. This builds upon a similar recommendation the RAIB made following its investigation of the Halkirk accident.
For the complete report, see: