Root Cause Analysis Blog

I think Training NI is the cause…

The Department of Energy’s Nuclear Weapons Labs do important and sometimes dangerous research on many topics including nuclear weapons. A report released yesterday by the General Accounting Office claims that weak safety management and weak oversight has kept safety at the labs from improving.

Read the report at:

http://www.gao.gov/new.items/d0873.pdf

Having experienced a GAO audit, I know the feeling the day after a GAO report is released. The GAO tends to take your external and internal investigations and reports, take the most sensational findings and recommendations, and then use the critiques to make the picture look as bleak as possible. Sometimes the bleak outlook is justified. Seldom does the report mention any positive initiatives and improvements. The reports, requested by congressmen looking for controversy, tend to look at past history (rather than future performance). Managers - even if they are new to the job - must live with the whirlwind of criticism that follows the reports release. Often, the “culture change” programs that follow are similar to nuclear weapons … they destroy everything in their path. Often the massive improvement efforts throw out the baby with the bath water. Since these types of massive improvement programs have been tried since the mid-80s with only limited success, my guess is that the root causes of the problems either aren’t being fixed or … the problems get “unfixed” after a number of years. Certainly these types of problems would be more organizational (which INCLUDES congressional funding issues) rather than just managerial.

Management at Lawrence Livermore, Los Alamos, and Sandia National Labs and their oversight (the National Nuclear Security Administration) are, no doubt, scrambling to respond to the GAO critique and develop appropriate improvement initiatives. After living through such difficult times, my comment is … Good Luck!

The third strike for wrong site operation during brain surgery, could it have been prevented? What changes for those who did not die were made in these patients’ personalities and thought processes? “While the hospital has made improvements in the operating room, they have not extended these changes to the rest of the hospital,” stated the director of health for this facility. How often we as a company struggle to fix the system without involving all that touch it. If you would like to change the system look through the TapRooTç success stories, see how it is possible to operate with the right and left side of the brain using our human factors driven process.

Mark recently spoke at the 33rd ESReDA Conference (The Future of Accident Investigation Conference) at the European Commission’s Joint Research Centre in Ispra, Italy. Mark’s talk was “GETTING BEYOND SWISS CHEESE IN ACCIDENT INVESTIGATIONS - Modeling and Improving Human Performance.”

Here’s a picture of Mark and the conference attendees.

If you would like to learn to find human performance, system, and organizational root causes of accidents and incidents, consider attending the 5-Day TapRooT® Advanced Root Cause Analysis Team Leader Course. For more information, click on the Courses link above.

And the root cause is …

In America, today is a day to get together with family and friends and reflect on our blessings - which are many!
One of my ancestors, Peregrine White, was the first child born to the Pilgrims in the New World. During November of 1620, Peregrine’s mother Susanna, gave birth to him aboard the ship Mayflower anchored in Provincetown Harbor. His father, William, died that winter - a fate shared by about half of the Pilgrim settlers.

The Pilgrims faced death and the uncertainty of a new, little explored land so that they could establish a place where they could worship freely.

With the help of Native Americans that allied with and befriended them, they learned how to survive in this “New World”. Today, we can be thankful for our freedom because of the sacrifices that these pioneers made to worship god in a way that they chose without government control and persecution.

One other interesting history lesson about the Pilgrims was that they initially decided that all food and land should be shared communally. After almost starving to death, they decided that each family should be given a plot of land and be able to keep the fruits of their labors. Thus those that worked hardest could, in theory, reap the benefits of their extra labor.

The result? A much more bountiful harvest that everyone was thankful for. Thus, private property and keeping the fruits of one’s labor lead to increased productivity and a more bountiful harvest.

Perhaps this is the root cause of Thanksgiving!

This story of Thanksgiving bounty is passed down generation to generation in my family. But if you would like more proof, read the words of the first governor of the Plymouth Colony, William Bradford:

“And so assigned to every family a parcel of land, according to the proportion of their number, or that end, only for present use (but made no division for inheritance) and ranged all boys and youth under some family. This had very good success, for it made all hands very industrious, so as much more corn was planted than otherwise would have been by any means the Governor or any other could use, and saved him a great deal of trouble, and gave far better content. The women now went willingly into the field, and took their little ones with them to set corn; which before would allege weakness and inability; whom to have compelled wold have been thought great tyranny and oppression.”

William Bradford, Of Plymouth Plantation 1620-1647, ed.
Samuel Eliot Morison (New York : Knopf, 1991), p. 120.

Many of us have had experience working with fault trrees.  We have worked throught the painstaking process of determining how a particular material failure will get us to a specific observed failure.  It is an excellent tool to structure your thinking when you need to see exactly what could possibley have caused the failure we are investigating.

Too often, we seem to focus only on the material failures.  The fault tree will have items such as “Relief Valve Fails”, Vessel Wall Fatigue Failure”, “Overpressure”, “Power Supply Fails.”  This is very comfortable, because it is only a piece of “stuff” that failed, and I can easily lay the blame for the failure on an inanimate object.  The fault tree will use the basic event (or equivalent) symbol, signifying that we need look no deeper than this event.

The more realistic investigator may add in a set of events that include “Human Error” into the mix.  More often than not, however, these events are catagorized as Incomplete Events, or those that we just don’t have enough information to further develop.  It is easier to just leave these events on our fault trees as incomplete, and we just accept that the human error is just going to occur.  After all, I can’t replace the human like I can a part.  And parts don’t get their feelings hurt or complain about the result!

Yet, deep down we know (and studies have proven over and over again) that these “human errors” are at the root of over 90% of equipment failures.  Therefore, although our fault trees may eventually get you to the point of discovering what part or condition contributed to the final failure, we still never seem to get to the bottom of 90% of our failures.

It is important that, when conducting a fault tree analysis, we include and run down the human errors that lead to the failure conditions.  The failure conditions that we discover should be considered Causal Factors (“Relief Valve did not lift”), and then a proper root cause analysis can be conducted on the reasons why the relief valve did not lift.  Use Equifactor® to help populate your fault trees.  Then take the results of your fault tree, plug them into your SnapCharT®, and finish your investigation.  Using the rest of the TapRooT® system to assist in your equipment troubleshooting process, you will get much further beyond where the fault tree drops you.

AFP reports on a mining disaster in the Ukraine. 90 have been confirmed dead and 10 are still missing. For the complete story see:
http://afp.google.com/article/ALeqM5jnxMxWdxKy4XLFaGpSw6Z9LKOYZQ

Click on the links below to download the investigation reports:

Derailment of Freight Train at Washwood Heath

Incident at Wellesley Roaf on Croydon Tramlink

Don Perry, who represents 12 people injured in the Texas City Refinery explosion, has filed a motion in federal court objecting to the proposed settlement between the government and BP.

The hearing on the plea and settlement is set for the Tuesday after Thanksgiving.

For more info see:

http://www.chron.com/disp/story.mpl/business/5319044.html

Woking within a team in an aircraft assembly environment, the candidate will be accountable and responsible for the timely delivery of projects that directly contribute to 2007 cost reduction targets through the elimination of non conformance. The applicant will be focussed on the resolution of assembly problems, often of a complex nature. The candidate will be responsible for data gathering, analysis, and identification of root cause and embodiment of the corrective actions. The individual will be expected to lead/facilitate cross-functional problem solving teams. The candidate will also be responsible for supporting Continuous Improvement activities. The candidate will ensure the manufacturing cell compliance with Company procedures and work instructions, also to JAR 21G, JAR 145, and environmental compliance through ISO 14001 AND 18001.

Key Skills / Knowledge Required (MUST HAVE) · Problem resolution, Complex problem solving skills, Project management, Lean manufacturing fundamentals. Working within a 19001 quality environment Display a pro - active approach and demonstrate the ability to work unsupported. · Microsoft Office, effective communication, team working

For more info see:

http://www.thecareerengineer.com/cand-viewjob.php?jid=260161

Human Factor Series 2: Encourage rewarding of safe group processes and individual willingness to stop suspected unsafe conditions

Why not just focus on unsafe conditions and not suspected unsafe conditions too? Do your supervisors encourage or even empower your employees to stop a procedure if they think it could be detrimental to the product, themselves, or the customer? As senior leadership what is your reaction to an employee who stops the manufacturing line for suspected safety issues and it turns out to be a false alarm?

One example that comes to mind is “Seventy-three seconds after the countdown clock reached zero the Challenger exploded”. While group think is the leading theory behind this launch failure, the underlying root causes are the failure to enforce standards, the need for complex knowledge decisions, and accountability needs improvement. Had TapRooT® been used for the root cause analysis would the findings and culture be different today?

Are more missed safety findings more important to your company than stopping more false alarms? See the change of criterion based on employee focus. To put it simply, if you reward schedule only then the company rewards the hero employee willing to make sacrifices.

Sorry - no posting today - I’m on a plane flying back from an accident investigation conference in Italy.

Also no posting next week due to Thanksgiving.

Mark

The Failure Analysis Engineer will be responsible for conducting root cause failure analysis of products, both from the field and compliance testing. Analyzes any and all available information to determine when products are not performing to the customer’s satisfaction or product specifications. Incorporate these findings into the product review and revision. Relies on extensive experience and judgment to plan and accomplish goals with a degree of creativity and latitude expected.For more info see:http://www.gadball.com/jobs/details.aspx?query=10691999/failure-analysis-engineer.aspx&AspxAutoDetectCookieSupport=1

The Associated Press reports that an Oklahoma City federal jury awarded Juan and Veronica Lara $3.6 Million for an injury when a pipe fell, hitting Juan on the head and causing brain and vision damage.

Your realtor describes this feature as “well lighted driveway for extra home security” . . . 

You think … “Root Cause Analysis needed!”  

Dave Thompson and Malcoml Gresham recently taught a successful TapRooT® 2-Day Incident Investigation and Root Cause Analysis Course in Kuala Lumpur. Dave stayed on to teach the thitd day to complete the 3-Day Equifactor® / TapRooT® Equipment Troubleshooting and Root Cause Analysis Course.

Here’s the class photo:

If you are interested in Public TapRooT® Training see the Courses link above.

If you are interested in a course at your site, contact us at 865-539-2139 or use the Contact Us link above.

(System Improvements, Inc. is the company that provides TapRooT® Root Cause Analysis Training and Consulting Services.)

How can you make the nuclear accident at Chernobyl worse? Watch the video and find out…

The NTSB has developed two simulations of “very close” near-misses that happened when:

• a plane landing almost hit a plane taxing and

• a plane taking off almost hit a plane taxing.

These are available at CNN’s wen site (I couldn’t find them on the NTSB web site). See:

http://www.cnn.com/2007/US/11/09/runway.safety/index.html

People look on as ship is driven ashore.

Reports say that as many as ten ships sank or ran aground in the Strait of Kerch and in nearby areas of the Black Sea in a fierce storm. Up to 11 sailors are dead or missing. For more information see:
http://www.chron.com/disp/story.mpl/world/5293205.html

Picture of damage to ship

I love reading newspaper accounts of accidents. An Associated Press story had the following two quotes:

“Investigators were focusing on possible communication problems between the ship’s crew, the pilot guiding the vessel and the Vessel Traffic Service, the Coast Guard station that monitors the bay’s shipping traffic.”

“A language barrier between the vessel’s pilot, Capt. John Cota, and the ship’s all-Chinese crew was not likely a factor in the crash, since the ship’s captain and officers are required to speak English, officials said.”

I wonder what official said that a Chinese crew speaking English was not likely a factor because they are REQUIRED to speak English?

Here are some recent stories and press releases about OSHA fines:

OSHA Fines Kansas City Power & Light $21,000 After Accident that Kills 2 and Injures 4

OSHA Proposes $128,000 in Fines for Violations at Oshkosh Specialty Vehicles

OSHA Proposes $196,000 in Fines for Violations at Cintas Corp in Mobile, AL

OSHA Fines Gary Grading $136,200 After Accident

D’Alessandro Corp Fined $52,000 After Trench Cave-In

Does your safety program have the tools it needs to prevent these types of accidents and fines? Good root cause analysis - applied during the investigation of incidents and near-misses and during proactive audits and observations - can help you stop major accidents and avoid serious OSHA fines.

To learn about TapRooT® and how it can help you improve performance, we recommend attending either a 2-Day TapRooT® Incident Investigation and Root Cause Analysis Course or 5-Day TapRooT® Advanced Root Cause Analysis Team Leader Course.

Preventing even a single accident or OSHA fine can pay for your training and your improvement program for years to come.

CNN reports that 5 rail cars carrying coal were dumped into the Anancostia River when a CSX freight train struck another train that was parked on a bridge.

For the complete story from CNN, see:

http://www.cnn.com/2007/US/11/09/train.derailment/index.html

The AP reports that 30 people riding an escalator were hurt in a strange escalator accident. The escalator, that was going down, suddenly accelerated and then stopped - throwing people down the escalator.

Six people were taken away by paramedics.

For more info see:

http://www.chron.com/disp/story.mpl/ap/tx/5287999.html

It hurts to watch:

http://strange.blosker.com/link/the-most-unfortunate-soccer-accident-22812

Sr. Advisor – SHE (Corporate)

ENSCO International, a leading global drilling contractor and member of the S&P 500, is currently seeking a Safety, Health and Environment (SHE) Advisor for its corporate headquarters in Dallas, Texas.
This position will be responsible for supporting the integration and development of SHE systems and culture within ENSCO and identifying current and best practice “gaps” and recommending improvements.  This includes, but is not limited to:

• Supporting SHE Leadership Development program initiatives.
• Prepare monthly Corporate SHE reports.
• Analyzing SHE audit reports and incident data and making recommendations for improvement.
• Ensuring SHE standards and policies (including permit-to-work programs, STOP, and ENSCO’s Corporate Safety and Environmental Management Systems) are properly implemented in the Business Units and monitoring efforts to close gaps.
• Developing Corporate SHE standards, policies, and procedures.

Successful candidates will have a Bachelor’s degree plus 5 years experience supporting progressive SHE systems in an industrial working environment, preferably in the offshore drilling or E&P industries.  Position requires excellent analytical and communication skills (both written and verbal), as well as strong PC skills.  Experience with incident reporting software or databases such as Synergi is strongly preferred, as well as with accident investigation systems such as TapRooT®.  Position will require some international and domestic travel.

—

To apply go to the ENSCO web site and navigate to the employment section.

It was a good day until the ship hit “something.”

The water started coming in.

In the Intercoastal Waterway, you don’t have to worry about becoming the next Titanic. You just point your bow toward the shore, “go aground” in soft mud, and wait for the Coast Guard to arrive.

For more on this accident see coverage on CNN’s web site:

http://www.cnn.com/2007/US/11/08/ship.rescue/index.html

Holcim has an immediate opening for a results-oriented, experienced Reliability Engineer with root cause analysis skills. For more info see:

http://maintenancetalk.com/blog.php/viblog/reliability_engineer_bloomsdale_mo2/

Reuters reported that a boiler tube rupture at Dominion ’s Salem Harbor Power Plant in Massachusetts killed three workers. Dominion said it will shut down all four units at the plant while they investigate the accident.

Interested in Amusement Park accidents? See:

http://www.rideaccidents.com/

Responsible for implementing Total Maintenance Effectiveness across all the Schwan Food manufacturing facilities and leading facility teams in researching and developing a Total Maintenance Effectiveness program, Infrastructure Needs and Best Practices plan for our maintenance operations and then lead the process by working with plant teams and corporate to facilitate the implementation.

Needs experience in Root Cause Failure Analysis.

For more info see:

http://maintenancetalk.com/blog.php/viblog/maintenance_and_reliability_engineer_marshall_mn/

Volt Services Group is seeking an experienced Reliability and Maintainability Engineer with several years of experience in working with the Military to be part of a centralized Reliability/Maintainability group supporting products designed for the Military.

For more information, see this job posting:

http://www.hotgigs.com/logged_in/search/index.cfm?profileid=Ji5RWi9ZPk4gCg%3D%3D&userid=JS5BWixYXlwgCg%3D%3D&id=Ji5RWi9ZPk4gCg%3D%3D&page=viewsearchresults&Mode=G&override=true&jobview=yes

One of the elements of the revised standard that I thought would interest readers of this blog is  LD.4.260. It states that:

The organization implements an integrated resident safety program throughout the organization.

LD.4.260 includes 13 specific elements:

1. There is an organization-wide, integrated resident safety program.

2. One or more qualified individuals or an interdisciplinary group manages the organization-wide safety program.

3. The scope of the program includes the full range of safety issues, from potential or no-harm errors (sometimes referred to as near misses, close calls, or good catches) to hazardous conditions and sentinel events, which have serious adverse outcomes.

4. All departments, programs, and services within the organization participate in the safety program.

5. The organization creates procedures for responding to system or process failures, such as continuing  to provide care, treatment, and services to those affected, containing the risk to others, and preserving factual information for subsequent analysis.

6. The organization: Defines responses to various types of potential adverse events. 

7. The organization: Conducts proactive risk assessments.

8. The organization: Makes support systems6 available for staff members who have been involved in a sentinel event.

9. The organization: Analyzes and uses information about a system or process failure to improve safety.

10. The organization: Provides systems for the internal and external reporting of a system or process failure.

11. The organization: Provides governance at least once a year, with written reports on all system or process failures, on the number and type of sentinel events, on whether the residents and the families were informed of the adverse events, and on all actions taken to improve safety, both proactively and in response to actual occurrences.

12.  The organization: Disseminates lessons learned from root cause analyses to staff who provide services or are affected by the situation.

13. The organization: Encourages external reporting of significant adverse events, including voluntary reporting programs in addition to mandatory programs.
To review the revised program see:

http://www.jointcommission.org/NR/rdonlyres/CD9A619C-C364-4543-A840-50E9AD914E17/0/09_ld_ltc2_prepubstds.pdf

The Superior Daily Telgram reports thyat 4 workers were overcome by Hydrogen Sulfide in a confined space at the Lakehead Blacktop Material Landfill. federal OSHA will be conducting an investigation. For complete details see:

http://www.superiortelegram.com/articles/rss.cfm?id=23269&freebie_check&CFID=64903453&CFTOKEN=97764091&jsessionid=88306f07ed346e6f3d44

Agence French-Presse reports that the cause of a ship going aground on a Greek reef was an inaccurate chart that was issued by the Greek Hydrographic Service. The cruise company says that the reef was larger and further out to sea than the chart indicated.

Is this the result of an accident being caused by a single failed Safeguard (the chart) or should have other Safeguards been in place?

For more information see:

http://newsinfo.inquirer.net/breakingnews/world/view_article.php?article_id=98459

Human Factors Series 1: Taking Signal Detection Theory (SDT) out of the Laboratory and into the Workplace

Everyday we have to make decisions about uncertain events. Signal Detection Theory is a model of how people make uncertain decisions. To put it simply, can you single out one object (beep, light, defect…) from the surrounding workspace? How important is it not to miss the object you are searching for (correct hit) and how detrimental to you is it if you indicate the object is there when it isn’t (false alarm)? What does senior leadership reward?

An article published in http://www.latimes.com/ stated that 75% of the fake bombs and explosives sent through LAX Airport security screeners during an undercover terrorism drill passed through undetected.

Airport authorities were not surprised. The simulated devices were not “sticks of dynamite” but “more like caps on a pen….a piece of metal with a wire in it. TSA’s remedy is to send poorly performing screeners into remedial training until their screening scores get better or remove the employee. To see Signal Detection Theory used to assist training for security screens see this SDT Simulation.

A former security director at LAX cited in the article stated that authorities could: purchase more advanced screening equipment, fund the proper number of screeners at each airport and ensure that screeners who consistently fail covert tests are removed.

The question I ask is how does one get from the incident of missing the explosives in the screening process to corrective actions such as remedial training, possible firing, hiring more employees, and purchasing better equipment? This is where the human factors science such as Signal Detection Theory behind TapRooT® could have helped with the root cause analysis.

Use a structured root cause analysis like TapRooT® that asks questions tied to human engineering such as were errors (targets) detectable? Are choices made by the person subject to knowledge-based decisions? Are the safeguards or barriers dependent on human action?

While types of human error are subject to change, human factors science continues to be introduced into the workplace with increasing usability. Replacing “common sense” with a structured root cause analysis based on science will continue to improve corrective actions.

Human Factors Series 2 will discuss Management Systems influence on correct hits and false alarms.

The Associated Press reported that 14 railway officials, including the Deputy Chairman of the Railways Authority, were sentenced to one year in jail for their part in a 2006 train accident that killed 58 people. At least part of the cause of the accident was reported to be poorly maintained switching equipment.

Below is a sample of Mid-East TV coverage from the accident (some graphic accident footage included) from the web.

QuickTime Format (.mov)

QuickTime Format (.mov)

A new release from the Chemical Safety Board:

Des Moines, Iowa, November 1, 2007 -  Investigators for the U.S. Chemical Safety Board (CSB) said today progress is being made in the investigation of a large explosion and fire which occurred at the Barton Solvents facility in Des Moines, Iowa on October 29.  The company packages, stores, and delivers solvents and other chemical products, many of which are widely used in the paint and coatings industry.

Lead Investigator Randy McClure said the accident occurred when an as-yet unidentified source ignited the ethyl acetate solvent being loaded into a 330 gallon square tank, called a tote. Mr. McClure said the operator who was filling the tote had turned away momentarily when he heard what was described as a popping sound, which witnesses believe was a pressure relief device.

Mr. McClure said, ‘A fireball then erupted from the tote.  Pressure from the eruption knocked the filling nozzle out of the tank, spraying ethyl acetate into the room and onto the operator. His clothing ignited, but he quickly removed the ignited clothing and escaped serious injury. Another operator tried to use a hand held extinguisher to battle the blaze, but it emptied before the fire could be extinguished.  He then shut off all power to the area and the plant was successfully evacuated.  The flames spread quickly and eventually consumed and destroyed a large portion of the facility.’

Mr. McClure said, ‘We will conduct tests to determine if the solvent could have produced enough static electricity to be the ignition source.  We will also evaluate the design and maintenance of equipment in the area to determine its suitability for use in a flammable environment.’

(more…)

See the job posting at:

http://www.bayt.com/job/job.adp?xid=653596

Last Monday I published an article titled:

Monday Accidents & Lessons Learned: Third Accident in 2 Months - When is an Accident a Repeat Failure?

The Danish Accident Investigation Board has published a report that suggests that this third Dash-8 Q400 accident was NOT a Repeat Failure but rather a failure of the landing gear from a totally separate cause.

To see the Danish Accident Investigation Report go to:

http://www.hcl.dk/graphics/Synkron-Library/hcl/dokumenter/Redegorelser/2007/510-000449%20LN-RDI%20Preliminary%20Report%20UK_03112007.pdf

The investigation shows that this accident could have been caused by an o-ring failure and prior landing gear replacements (although the investigation is not complete).

Lesson Learned: You can’t assume that a failure is a Repeat Failure until the investigation is complete and you are sure that the cause is indeed the same.

One more note: The o-ring failure might be a Repeat Failure. The difference is that this time they moved a component from one system to another that allowed the failed o-ring to enter the landing gear system. (See the report for more info.)

Drawing from Report

Face Shield . . .

Hard Hat . . .

Welding Mask . . .

Secure work platform (not recommended if your co-workers are mad at you) . . .

Thanks for e-mailing us the jokes, Michele!

Tompson Financial reported that two people were killed and two others were injured at Gold Fields Ltd’s No. 4 Shaft of Kloof Gold Mine, near Westonaria in South Africa. For more info see:

http://money.cnn.com/news/newsfeeds/articles/newstex/AFX-0013-20623889.htm