CSB Final Report on Xcel Energy Accident Finds
Company and its Contractor Failed to Adequately Prepare
for Hazardous Work Inside Confined Space of Hydroelectric Plant Tunnel;
Xcel Had No Technically Qualified Responders on Duty
Report Urges OSHA, Colorado Public Utilities Commission to
Strengthen Regulations; CSB Finds 45 Confined Space Fatalities
Have Occurred in 53 Incidents Nationwide Since 1993
Denver, Colorado, August 25, 2010—The tragic accident that took the lives of five industrial painting contractors deep inside an Xcel Energy hydroelectric plant tunnel in Georgetown, Colorado, was the result of several vital safety failures, the U.S. Chemical Safety Board (CSB) determined in a final investigation report issued today in Denver.
Nationally, the investigation identified 53 serious flammable atmosphere confined space accidents that occurred from 1993 to April 2010, causing 45 fatalities and 54 injuries, the majority since 2001.
The CSB also released a 15-minute safety video entitled “No Escape: Dangers of Confined Spaces,” which includes a detailed animation depicting the horrible tragedy that unfolded inside the mountain tunnel at Xcel’s Cabin Creek plant on October 2, 2007.
The accident occurred in the water tunnel, or penstock, of the hydroelectric plant, located 45 miles west of Denver. The penstock carries water from an upper reservoir to a lower one, driving power turbines. The painting contractors, from RPI Coating, Inc., were recoating a 1,530-foot steel portion of the 4,300-foot penstock when a flash fire suddenly erupted as the vapor from flammable solvent, used to clean the epoxy spraying wands, ignited, probably from a static spark in the vicinity of the spraying machine. The initial fire quickly grew, igniting additional buckets of the solvent, methyl ethyl ketone (MEK), and other combustible epoxy materials stored nearby.
The CSB concluded the causes of the accident included (1) a lack of planning and training for hazardous work by Xcel and its contractor, RPI Coating, Inc., (2) Xcel’s selection of RPI despite its h aving the lowest possible safety rating (zero) among competing contractors, and (3) allowing volatile flammable liquids to be introduced into a permit-required confined space without necessary special precautions.
The CSB report found that the permit-required confined space rule set by the U.S. Occupational Health and Safety Administration (OSHA) does not prohibit entry or work in confined spaces where the concentration of flammable vapor exceeds ten percent of the chemical’s lower explosive limit, or LEL. (The LEL is the concentration of vapor in air below which ignition will not occur.)
OSHA’s rule does state that an atmosphere exceeding ten percent of the LEL creates an atmosphere “immediately dangerous to life and health” and that steps should be taken to define safe entry conditions; however, the rule does not define what those safe entry conditions should be or specifically prohibit entry into such hazardous atmospheres, the report notes. The CSB recommended OSHA establish a fixed maximum percentage of the LEL for entry so that work in potentially flammable atmospheres would be prohibited.
Additionally, the Board made recommendations to the company, the governor of Colorado, the Colorado Public Utilities Commission, trade groups, and other organizations.
CSB Board Member William B. Wark said, “This tragedy should never have happened. The companies did not effectively plan for the dangers of bringing significant amounts of flammable liquids into the tunnel, which was a hazardous confined space. Doing so was an unacceptable deviation from good safety practices.”
There were ten workers in the tunnel and one at the entrance at the time of the fire. Five were unable to get around the fire on the painting platform to get to the only available exit – the improvised tunnel entrance. Five workers on the other side of the platform made it to safety, although three of those workers sustained injuries.
The CSB found that Xcel and RPI failed to have technically-qualified confined space rescue crews immediately standing by at the penstock in case of emergency, as required by regulations. Workers called 911 for help but responders entering the penstock had to retreat in the thick smoke, as did workers who had approached the fire with extinguishers.
The closest confined space technical rescue unit – equipped and trained to enter the smoke-filled tunnel – was approximately one hour and 15 minutes away. The trapped workers died about one hour before this response unit arrived, their escape blocked by a steep vertical section of the tunnel deep inside the mountain.
CSB Investigations Supervisor Don Holmstrom, who led the investigation, said, “The five trapped workers communicated with co-workers and emergency responders using handheld radios for approximately 45 minutes, desperately calling for help, before succumbing to smoke inhalation. Their lives likely could have been saved had qualified, company-provided rescuers been in a position to respond immediately to a fire or other emergency.”
Board Member Mark Griffon, joining Mr. Wark and Mr. Holmstrom at the news conference, said, “Even before the operation began, the stage was set for disaster. Xcel not only did not adequately plan for the operation, but it selected the painting contractor with the lowest possible safety rating among the bidders, and it did so mostly on the basis of cost – it was the lowest bid.”
The investigation found that Xcel hoped to compensate for RPI’s safety record by closely supervising the contract work, but did not do so even when the company learned of safety issues during the initial penstock work.
The CSB investigation found Xcel and RPI managers were aware of the plan to operate the epoxy sprayer in the tunnel and to use flammable solvent to clean the sprayer and other equipment.
Mr. Holmstrom said, “As a result of not performing a hazard evaluation of the work to be done, the companies failed to identify serious safety hazards involving use of flammable liquids within the confined space. Use of safer, nonflammable solvents was not evaluated, continuous air monitoring was not required, and key policies and permit forms did not establish a percentage limit for flammable vapor in the tunnel atmosphere.”
Board Member Wark noted the lack of planning for escape in an emergency. “The penstock had only one egress point – the tunnel entrance,” he said. “Xcel and RPI did actually identify this as a major concern in their planning. But despite this, no plans were made for prompt rescue in an emergency, and no rescuers qualified to enter this confined-space environment were standing by.”
The CSB investigation determined that while companies are required to perform a hazard analysis prior to issuing permits for work in confined spaces, regulatory standards pertaining to the use of flammables within confined spaces are inadequate.
Board Member Griffon stated, “Other OSHA regulations on confined and enclosed spaces – for example in the maritime industry and other sectors – prohibit work in such confined spaces above a specific percentage of the LEL, often ten percent. We are recommending that OSHA adopt such enforceable limits for all industry.”
The CSB recommended that OSHA amend its confined space rule to establish a maximum percentage substantially below the lower explosive limit for any given flammable for safe entry and occupancy while working.
The CSB made recommendations to nine other entities. These included that the governor implement an accredited firefighter certification program for technical rescue with specialty areas including confined space rescue; that the Colorado Public Utilities Commission (PUC) require regulated utilities to adopt provisions for selecting contractors based on safety performance measures and qualifications; and that the PUC require utilities to investigate all incidents resulting in death, serious injury or significant property damage and submit and make public written findings and recommendations within one year of the accident.
Numerous recommendations were made to RPI Coating, particularly aimed at revising its confined space entry program and guidance.
CSB investigators and board members cited difficulties encountered in the investigation resulting from efforts by Xcel Energy and RPI Coating to impede the investigation and prevent the release of the investigation report.
Citing a formal Letter of Admonishment sent to the Xcel chief executive officer earlier in the week, Board Member Wark said, “The lack of cooperation and efforts by Xcel to impede our investigation are unprecedented. Mr. Griffon and I join our chairman in criticizing these actions in the strongest terms.”
The letter, signed by CSB Chairperson Rafael Moure-Eraso, states Xcel Energy did not fully comply with CSB requests for documents or answers to questions in formal interrogatories. This required the CSB to seek assistance from the U.S. Attorney’s office in Denver, resulting in delays to the investigation and additional costs to taxpayers. In May, Xcel took the extraordinary and unprecedented step of going to federal court seeking to block release of the CSB report and the safety video. The court sided with the CSB in favor of release.
Xcel was given an advanced draft copy of the report last April for review for accuracy and for confidential business information in accordance with CSB review protocols. Xcel never responded, but in August 2010, contrary to the conditions of confidentiality attached to their receiving this preliminary copy, released it to a news organization.
The letter from Chairperson Moure to Xcel’s CEO concludes, “In light of this disappointing pattern of corporate conduct, I am writing you directly to ensure that you are personally aware of the actions taken by Xcel to delay the CSB investigation, block publication of the CSB final report, and distort the conclusions of the investigation by releasing an unauthorized draft copy of the CSB report. The CSB will issue a formal recommendation that Xcel shareholders be directly notified by management of the significant findings and recommendations of the CSB report, and of the actions Xcel management intends to take to implement needed safety improvements. In the wake of the corporate responsibility concerns raised by the Big Branch Mine accident in West Virginia and the disaster in the Gulf of Mexico, I strongly urge Xcel to renew its focus on safety and to swiftly implement the CSB’s recommendations.”
The BP Chairman told Sky News that comments by Mr Hayward have had detrimental effects as the company seeks to control the fallout from the disaster. BP Chairman Carl-Henric Svanberg said: “It is clear Tony has made remarks that have upset people.”
This morning on CNBC the commentators said that it was rumored that the BP Chairman was looking for new people qualified for the CEO job at BP.
The article says:
“Mr Svanberg admitted the rig blowout is turning from an industrial accident into a broader political concern with questions of the company’s ongoing viability being raised.”
“I still strongly believe in BP and that we will come through this,” the Chairman said.
As I said in an earlier blog posting, “You are either leading the solution or part of the problem.”
Here’s the whole interview …
and here is a previous report about BP CEO being “denounced.”
Went out to dinner at the Temple Bar Street area after day 1 of the 2-Day TapRooT® Incident Investigation and Root Cause Analysis Course in Dublin and there was great street entertainment. Here’s a sample …
When you mapped out your first SnapcharT® on your first Incident Investigation after your first class what “Ah ha” moment did you get?
Often people are surprised when they map out their first chart and then have a person not familiar with the Incident read it…. what did you learn?
I started this discussion so that our clients can share a learning process with our friends in this group.
“Ah ha” is used to make the”Aha” moment a greater moment of realization.
_____________________________________________________________________
From: Mike Rodriguez, Safety Specialist at ConocoPhillips
Chris,
I have to chuckle. I can’t remember my first chart and they were E&CF charts back then. However, to this day, the use of a chart proves to be the most effective method for “painting the picture” of what happened.
I have done hundreds of investigations with the tool and it reliably leads to understanding what happened. Of course you have to f”eed the machine” so if you’re not getting those “ah ha” moments it may be you haven’t gathered all the information.
How do you get “all” the information? Get to the field. Take those pictures, measurements, interviews…Review the SPAC. Put it all on the Spring SnapCharT(r). Let the incident talk to you. Review the 15 questions from the TapRooT(r) Tree(r). Put your findings in the Spring SnapCharT(r). Let the incident talk to you. And, the loudest, clearest talking tool is a SnapCharT(r).
Regards,
Mike
_____________________________________________________________________
From Me: Chris Vallee, Senior Associate and TapRooT® Instructor at System Improvements
Thanks Mike for kicking off the discussion. Many people get new posts at the end of the week so I am looking for more engagement as time goes.
It is difficult to remember my first SnapCharT(R) outside the classroom environment, but I can remember two very distinctive aha moments with two consulting jobs: one a thermoelectric that had a very bad accident, they were trying to blame the network. Nothing was checking out. After building a SnapCharT(R) with the results of the interviews, which lead to an audit of the automation system, we found out that the automation circuit had been modified in the field and was blocking all the self-protection sensor signals. They had three problems – the control system was modified, blocking all but one control computer; second, the control buttons were very near and an operator pushed the wrong button, and third, the generator field DISCONNECT button was not labelled according to the drawings, nor to the circuits or to the diagrams, so when time came to activate it it was not found. There were some issues in training too. They had been trained two years before and there was no refreshing of the concept once the plant finally started-up. Fortunately nobody died, but it certainly was a close call. When the client saw the SnapCharT(R) he could not believe it, that so many things were wrong and nobody had detected them before.
The second one was sea-borne fire in a transport ship. The captain could not believe that they were looking in the wrong place all the time. It was an eye-opener. _____________________________________________________________________
From: Mark Paradies, President at System Improvements
I remember my first two Aha! moments. They were in early 1986.
1. That there are multiple causal factors (not analyzing the cause of an accident – analyzing the causes of multiple causal factors).
2. That a guy who wrote an incident report was just making things up so he could get the report done because they couldn’t figure out what really happened and they had to meet a report deadline.
From: Marco Flores-Verdugo, Propietario, TECMEN SA de CV
I have to agree with Mark.
Many many times the reports are made up with little or none robust data .
You need to have something , once we had an explosion, w 3 injured in a plant in SEA, the report was made in Mexico ,without consulting with the plant!
To add insult to injury , the report was sent to SEA so that they know now what happened, and how to fix it. this was done again and again. The preffered analysis method were isle meetengs, at the home office.
_____________________________________________________________________
From Me: Chris Vallee, Senior Associate and TapRooT® Instructor at System Improvements
Thanks Mark and Marco for your comments.
The question often asked during an investigation is whether something is a fact, a judgment, or just plain made up. How do you get though this piece… pictures, videos, reports, and immediate interviews when possible… pre-planning is so vital to a good SnapCharT®.
_____________________________________________________________________
From: Mark Paradies, President at System Improvements
In that first case it was easy. The way they told the story … It just couldn’t physically happen. The system just didn’t work that way.
Went out to the field and talked to field folks. They showed us what they were doing to “troubleshoot” the problem. They had basic problems in the way they were trying to analyze the issue. When we fixed these, we realized that the problem only happened when the electronics heated up. Thus when tests were performed “cold” in the shop, everything worked great. The units would then be rotated back into “spares”. When they were used to replace another unit, it would take time (hours to days depending on how they were being used) to heat up. When they finally got hot … they would fail again, get replaced by another “spare”, and then go back to the shop for troubleshooting …
This had gone on for years! People said things like, “Those XXXXXX never do work right!” when about a dozen bad units were causing all the problems.
The SnapCharT® helps you see the logic (or lack thereof) behind what is happening. You have to understand what is happening before you can understand why it is happening.
Interesting Mark ,
A recent issue with a track detector in an automated transfer car at a switching furnace was traced to very much the same problem, the only extra problem is that the manufacturer in (Italy I believe) was aware of the problem and did not warn the users, this we found out while making the SnapChart(R).
Another case presented in one of my classes, that I remember, was a problem with a customer that wanted to charge a large fee (several million Dollars), for what he said was a forced shut-down due to lack of supply. The client, in this case, started making the SnapCharT(R). We started exchanging questions, rabbits started to jump everywhere. They called the Management. The Management called the General Director. Then the World General Director was called, he flew to Monterrey to be in the class. At the end they were not liable for anything. There were many things wrong in their chain of supply and in their customer’s communication network. Fortunately they did not have to pay anything and the funds were used to fix their own system. They really loved the SnapCharT(R) and the TapRooT(R) system. Now they are frequent attendees to our classes. A real “aha” moment, if any.
Watch the video. Do you think anything was learned?
I know, this video teaches us how good the “train avoidance training” was that we gave the worker. Now all we have to do is teach the rest of the employees how to leap out of the way at the last second!
At some conferences, 90% of what you get is the networking. You meet some great people and take home some new “best practices” that you can apply at your facility.
At some conferences, 90% of what you get comes from the great keynote speakers. They inspire you to go back to work and accomplish even more.
At some conference, 90% of what you get is from the breakout sessions. The small group interactions, lessons learned, and best practice sharing in these smaller sessions can really be helpful in developing an improvement initiative.
Why can’t you have all three? Who knows. But finding all three together is really uncommon.
I attend a lot of conferences. Seven to ten a year. More than most people. (Who can afford the time?) But I do it to stay on top of the latest improvement initiatives. And here is what I’ve found. There is one conference that has all three every time it is held … The TapRooT® Summit.
You get the 90% networking and the 90% great keynote speakers and the 90% amazing breakout sessions that are the highlights of most conferences all in one conference.
That’s 90% + 90% + 90% = 270%!!!
170% more than your average conference.
Ok … You may think that Mark is going nuts. After all, what conference has 170% more than other conferences? Well then, lets look at each of the percentages for proof.
Part of the purpose of the Summit is to send you back to work inspired to to your best. Who is better to inspire you than Jeff Skiles, the co-pilot of the Miracle on the Hudson flight. As the Summit’s closing keynote speaker, Jeff will tell his amazing story that includes lessons of teamwork, adaptability, training, and preparation that helped them make it through that day. Don’t miss this chance to learn from his experience and shake his hand.
What does it take to keep young workers safe? Do these same lessons apply to all employees? That’s what you will hear about and think about when our opening keynote speaker, Candace Carnahan, tells her story. She learned a difficult lesson about safety at an early age. She will share that lesson with everyone at the Summit. It’s a message that anyone with young employees needs to hear. Don’t miss it!
Finding the root causes of your problems and developing effective fixes is only half the battle. If you want to really improve performance, you need to get the improvements implemented. That’s why I chose an improvement implementation expert, Don Harrison, to be a Keynote Speaker. He is President of Implementation Management Associates and the developer of the Accelerating Implementation Methodology (AIM). And Don will speak about one of the most difficult aspects faced by people leading improvement efforts … Getting Sponsorship Right. So if you need senior management support for your improvement program and getting improvements implemented, you can’t afford to miss his talk.
Everyone knows that blame is counter productive when trying to improve performance. But what if you we a patient at a hospital and you were almost killed? The sentinel event required extensive rehab so that your life was disrupted for almost a year. Would you be empathetic? Or would you want to strike back? Would you sue over the error? What if the hospital didn’t seem to be telling you the truth? Would this make you mad? As an investigator of accidents, you often have to deal with people who are impacted by the accident – either those who were hurt or those who are being seen as “at fault.” That’s why you need to hear Linda Kenney’s story. You will hear see Linda eventually understood what happened, learned to work with those who “caused” the damage, and took this event and made it into something positive (for her and for the people who “caused” the damage she suffered). It’s an amazing story that will get you to think beyond just the root causes of an accident to the impact accidents can have on people.
I’ll be one of the Keynote Speakers at the 2010 TapRooT® Summit. And I’ll be talking about an important topic for all TapRooT® Users – Taking Improvement to the Next Level. Over the past couple of years, I’ve come to understand that many TapRooT® Users attend their initial TapRooT® Training and expect that that is all they need to do to have excellent root cause analysis and an outstanding performance improvement program. But those people are wrong. They have taken an important first step, but the journey is still at it’s beginning. To achieve excellence, they need to do more. And that’s what I’m going to share. The steps one must take to achieve excellence in their performance improvement journey using TapRooT®. If you are a TapRooT® User somewhere along your journey to excellence, don’t miss this keynote address. And if you don’t use TapRooT®? The lessons are just as applicable. Don’t miss this keynote address.
OK … I think this gets more than 90% … I think it gets 100%. But, being one of the speakers, I may be biased. What do you think?
NETWORKING
I’ve had many TapRooT® Summit participants tell me that the networking at the TapRooT® Summit is the best they have every experienced at any conference they have ever attended. And I know the reason why. We work hard to make it the best networking conference on the planet.
What do we do? You’ll have to be there to experience it, but I’ll give you some clues.
First, we start out with a networking exercise to get each participant to meet a minimum of three new people that they share something in common with. I personally assign these introductions for every Summit participant.
(Reception) (Golf)
Next, we sponsor two events – the Summit Reception and the Summit Golf – to help people get to meet even more folks on an informal basis. Both are great ways to find new friends.
Third, we even make lunch a networking event by letting you pick your favorite TapRooT® Instructor (or maybe a TapRooT® Instructor that you have never met) to sit with at lunch.
Fourth, several of the best practice sessions are organized to encourge even more networking.
Fifth, there really are great people to meet at the Summit. Speakers and participants alike! And because the Summit is not a “mega-conference” … you really can find people and meet them (and shake hands with and talk to speakers).
And finally, because we emphasize networking so much, everyone participates in it and makes it easy. Introductions happen spontaneously. If someone doesn’t know you, they ask who you are and what you do. And you will find meeting new people easier and more fun than you ever have before at any other conference.
How do I know that I’m right about the networking being so good? Listen to what just a few of the attendees had to say (and this is just a sample – double click each video to play):
(Quicktime Format .mov)
(.mp4 format)
(.mp4 format)
Do you agree that this deserves a 100% score (not just 90%)? I do. But read on…
TECHNICAL/BREAKOUT/BEST PRACTICE SESSIONS
When we first started planning Summits back in 1994, we worked to make each of the technical sessions full of new best practices, great ideas, and sharing of lessons learned. As the Summit developed over the years and additional tracks were added (there are 9 tracks this year), we have continued to make these sessions full of content that people interested in the topics of each track need.
Just look at these tracks and then see the Summit Schedule and look at the sessions in the track that you think you would like to attend the most and see how applicable the sessions are to your improvement efforts:
Safety & Risk Management
Behavior Change & Stopping Human Error
TapRooT® Certified Instructor
Investigation, Troubleshooting, & Root Cause Analysis
Wow! They are some great tracks aren’t they. But there’s more. You can mix and match to customize your Summit experience. You can pick from sessions from several tracks to develop Your Track – just like you want it.
But that’s not all. There are also “Special Topics” to choose from that are in addition to the sessions in the tracks. These Special Topics include:
Prioritizing Improvements (Mark Paradies)
What Does Management Need To Know About Root Cause Analysis (Kevin Palardy)
You can add these special topic sessions to your custom schedule to make the Summit even more significant to your improvement efforts.
To make this even more impressive, many of the speakers could have been Keynote Speakers. The problem is that we just have too many great speakers to fit them all into the five keynote slots. You might find dozens of the talks that are good enough to fill a Keynote slot, but here are some that I know would do the trick:
Bill Sirois: Measuring Fatigue Using FACT
Dr. John Grout: Using Mistake Proofing to Stop Human Error
Jennifer Mounce: Coaching Skills to Sustain Behavior Change
Brian Crawford: Lessons Learned about Human Factors & Generic Causes from Recent Airline Incidents
Karen West: Legal Aspects of Tap[RooT® Investigations
George Burk: Quality in Life & Work
Major General Doug Rob: High Performance Ideas for Leading Improvement
Brad Towe: Self Improvement for Your Future
Dr. Beverly Chiodo: Character Driven Success
Bill Nixon: Leading Successful Investigations: Drawing on UK Experience
Keith Recsky: Lifecycle of an Incident
And that’s just a sample. I can’t list them all or I’d list every session!
Again, don’t just take my word for it. Here is what people have to say about the quality of these sessions (in a .mp4 format video – double click to play):
Another 100%? I think so.
So maybe the formula should be:
100% + 100% + 100% = 300%
That makes the TapRooT® Summit 200% better than other conferences that you might attend.
That’s three times as good!
Even though you might not agree with my math … You get the idea.
So get the approvals you need and get signed up! You can’t afford to miss this Summit!