How much can a service outage cost? It’s costing Rackspace between $2.5 to $3.5 million because they guarantee reliability and provide credits if their service is out for more than a specified period of time.
With millions on the line, let’s hope they perform an adequate root cause analysis. The information in their preliminary release points to a series of electrical equipment failures. (They should have an Equifactor® Troubleshooting Chart for these.)
Portland will be the location for the 5-Day TapRooT® Advanced Root Cause Analysis Team Leader Training September 28-October 2, 2009. You will learn all the TapRooT® tools, and also many advanced skills, that a Team Leader needs to collect information, analyze root causes, and develop effective fixes that will help your company improve performance.
Portland is a city located in the Northwestern United States, near the confluence of the Willamette and Columbia rivers. As of July 2008, it has an estimated population of 575,930, making it the thirtieth most populous city in the United States, and has been referred to as the greenest city in the United States. Portland is Oregon’s most populous city, and the third most populous city in the Pacific Northwest, after Vancouver, British Columbia, and Seattle, Washington.
(Source http://en.wikipedia.org/wiki/Portland,_Oregon)
Often called “the big city with the small town feel,” Portland is known for its mild weather, breathtaking scenery, exciting location, and friendly people. The city has made many “Best Of” lists, including Top 10 Least Expensive Cities, Top 10 Places to Vacation, Top 10 Best Places to Live, and Top 25 Arts Destinations. With so much to offer, here are just a few attractions and restaurants to check out on your visit to Portland:
The Pittock Mansion was home to Henry and Georgiana Pittock from 1914 to 1919. Located high in the West Hills of Portland, the mansion sits 1,000 feet above the city’s skyline.
Portland Japanese Garden is nestled in the city’s scenic West Hills. The garden is authentically Japanese and consists of five separate and distinct garden styles.
The Pearl District offers visitors outstanding shopping, dining, arts & crafts and much more in this historical area of Portland.
Page’s Northwest Guide Service will show you the best fishing in Oregon, specializing in sturgeon, salmon, steelhead, shad, and walleye. Enjoy a day fishing many of Portland’s beautiful rivers–the Columbia, Willamette, Tillamook Bay, Yaquina, Siletz, and the Nehalem.
The Grotto is set among 62 acres of botanical gardens. As an internationally renowned Catholic sanctuary, The Grotto offers a place of peace and quiet reflection. More than 100 beautifully sculpted statues and shrines are nestled among flower-lined pathways, and visitors can enjoy spectacular cliffside vistas.
Dining in Portland:
Portland is a diverse city, and she shows that off in many ways. But the city’s passion for food and drink is something to be reckoned with. New restaurants pop up as often as daily menu specials, and emerging neighborhoods, markets, microbreweries and wineries make Portland a culinary playground.
Henry’s 12th Street Tavern features fresh seafood, USDA choice steaks, pastas, pizzas, and even authentic Asian dishes. They also offer more than 100 varieties of beer.
Huber’s is Portland’s oldest restaurant and creates delicious entrées like Prosciutto-Wrapped Turkey Florentine, Hazelnut Encrusted Gorgonzola Filet, and Beef Stroganoff.
Lucy’s Table is located in the trendy Nob Hill district and offers such dishes as Wild Boar Ravioli, Seared Scallops, and Tea Smoked Crispy Skinned Duck Breast.
Meriwether’s Restaurant is located in the historic Vaughn building on the site of the famous 1905 World’s Fair. They offer beautiful garden dining with a Pacific Northwest menu.
Mint 820 is located in the historic Frederick Torgler Building and they offer a wide range of menu options like Tequila Marinated Pan Fried Snapper with a Quinoa Salad, Red Chili Sauce, and Snow Peas topped with Spicy Slaw.
The Equifactor® tables in the TapRooT® Software give your equipment experts the tools they need to find the root causes of equipment failures. The extensive tables included with the software are based on the experience of Heinz Bloch, a respected reliability expert. These tables cover a wide range of equipment, including pumps, compressors, engines, and blowers.
For those using the latest version (5.1.3) of the Enterprise web software, additional tables have been added, covering AC and DC motors, valves and valve actuators, motor and generator commutators, and conveyor belts.
But what if a table does not exist for your specialty equipment? Chances are, your machinery contains individual components such as bearings, mechanical seals, and gear couplings. We have included tables for these common components.
Still need more detailed tables to help troubleshoot your equipment? The TapRooT® Software gives you the opportunity to add your own custom tables to the software! Have your equipment troubleshooting experts codify their knowledge by developing these custom tables. Have a piece of finicky equipment that only one guy seems to be able to keep running? Develop a custom table for that machinery, before your expert retires. This gives your newer, less seasoned maintenance people the benefit of years of troubleshooting experience.
The network versions of the TapRooT® Software have these same custom table features. Using the extensive security features built into the web software, you can designate specific people to have access to the custom table development features. This gives you more control over who is entering the data, what information is added to the software, who has validated the data, and when it is actually available to your troubleshooting community.
If you would like to talk to Heinz Bloch, now’s your chance! Back by popular demand, Heinz and his son Ken will be giving another set of equipment reliability seminars at the TapRooT® Summit this October in Nashville. Don’t miss this opportunity to discuss your equipment reliability experiences with these two experts.
Here’s pictures from a course being done in Europe with simultaneous translation. Sanjay Gandhi (in the center in the picture below with the translators) is the instructor. This is the second course Sanjay has done for a nuclear plant in Europe this year.
If you would like to schedule an on-site course for your facility, contact us by clicking here.
Rackspace, a web hosting company, had a 45 minute outage on June 29th. An Article in Web Host Industry Reviews says that they are going to perform a root cause analysis of the event.
Analyzing the root causes of web outages can be very similar to other outage analyses that people do using TapRooT®.
For example, BellSouth used TapRooT® to review 911 outages, long distance network outages, and local service outages.
Another high reliability computer service provider, Tandem Computers who was later bought by HP, used tapRooT® to analyze network and computer reliability issues/outages.
It really is amazing how even with different technologies, the same proven techniques can be used to find the root causes of human error and equipment failure.
If you would like to learn advanced root cause analysis to analyze service problems, attend a TapRooT® Course. For more information, see:
It is pretty amazing that they were able to rescue a small child from the ocean; I’d like to hear more about that.
After the Air France crash and this, I can hear the kneejerkers now; Airbus can’t fly in bad weather! Hopefully they can find the black box on this one.
Sad story. What is troubling is that they had a previous fatality at the plant and appear not to have learned much from that experience as evidenced by the recent violations:
“Records from the most recent OSHA inspection, conducted earlier this year, show that the federal agency identified several serious violations at the plant, including violations concerning floor and wall openings and holes, industrial stairs, respiratory protection, medical services, and oxygen-fuel gas welding and cutting.”
How many fatalities does it take before people get the message?
Don’t get caught in this trap - attend a TapRooT® course and learn how to find the root causes of problems after they occur and proactively identify and address issues BEFORE they occur. See the course schedule HERE.
I posted a blog a few weeks ago with a similar story; a construction accident in a developing country where someone is “being held.” The common theme is that blame is the focus with no mention of actually fixing the problem. In this story, officials are “embarrassed.” Can you say SCAPEGOAT?
They should be embarrassed. They should be embarrassed that they don’t have good systems to prevent such incidents. Meanwhile people are being hurt and killed.
I’m not sure I would want to be a construction supervisor or safety manager in this environment; you might end up in jail!
TapRooT® Instructors, Chris Vallee and Dave Janney, teach a record attendance Equifactor® course in Houston last week. Chris is wearing the loud red shirt. So who attended this class you may ask?
Medical…. Power Generation…. Petro-Chemical equipment reliability leaders and safety leaders sat in the class this day. Wait, did you read safety? Just ask the safety leader sitting next you and ask how many times a piece of equipment DID NOT BREAK during a major incident. Now you as an EH&S person may not be able to answer the equipment questions asked in Equifactor® but if you were given a structured way to ask competent equipment questions to your equipment experts would you use it?
Of course aside from major Incidents why would Equifactor® be useful for the EH&S leader….. after all a 30 cent O-Ring that frequently gets replaced on a shutoff valve located in a confine space doesn’t need to be analyzed does it? See below for upcoming Equifactor® courses close to you.
Location Dates
Chicago, ILLINOIS - Sep 17
Calgary, CANADA - Oct 16
Halifax, CANADA - Nov 5
Dallas, TEXAS - Nov 6
Aberdeen, SCOTLAND - Nov 11
Salt Lake City, UTAH - Nov 12
Edmonton, CANADA - Nov 25 Read the rest of this entry »
Here’s an article from Abu Dhabi about a second fatal accident involving a child and a school bus. It’s interesting because the writer discusses the hierarchy of Barriers (we call them Safeguards in the TapRooT® System) and how they can be used when developing corrective actions. You usually don’t see this level of understanding of root cause analysis in a newspaper article or letter to the editor.
Client located in the state of Pennsylvania has an immediate need for a Steam Turbine Specialist with 5+ years experience inspecting, repairing, maintaining and diagnosing steam turbines, gas turbines, diesel engines, and generators. Needs troubleshooting and root cause analysis skills. See:
A Global Investment Bank located in Jersey City, NJ has an immediate Full Time opportunity for a Voluntary Corporate Actions Specialist with root cause analysis skills. See:
In my previous life, I have a great deal of experience dealing with regulators. While I have had limited dealings with the FDA, I think the kind of approach Nestle is taking (if the article is accurate) is not very smart.
My experience is that you must develop partnerships with your regulators so problems can be solved together. Making life difficult for them will only come back to haunt you later.
While we are far from reaching conclusions on what brought down Air France 447, there is at least some indication that air speed may have been a factor.
Air France is replacing the air speed indicators (pitot tubes) on its fleet of A330 aircraft. What we cannot tell from the article is whether the same exact type of pitot tubes are on the other aircraft mentioned, so let’s no jump to any conclusions.
One thing I would like to mention is that airlines do have processes for the crew to follow in the event of an airspeed discrepancy. Why the Air France crew was not able to respond is a key question that is still unanswered, although we do know weather like they encountered requires a great deal of extra work on the part of the crew.
There are still more questions than answers. Until we have those answers, I’ll be more than happy to step onto an Airbus. I have confidence that the NTSB will do all possible to determine if this is a generic issue.
This situation brings up an interesting question - how do different companies in your industry share information about events? In the case of the airline industry, the aircraft manufacturers share information with those that operate their aircraft, and entities like the NTSB do as well. Not every industry has this type of feedback mechanism however. Food for thought.
I reviewed the slide show and the Executive Summary and I couldn’t find anything that I would call “root causes.”
I did see a good failure scenario that would make a good SnapCharT® and then could be used to identify Causal Factors (which are similar to the “Failure Conditions” in the presentation pdf). Their failure conditions were:
Increased Loads Due to Higher Fill
Hydraulically Placed Loose Wet Ash
Fill Geometry & Setbacks
Inusually Weak Slimes Foundation
But they didn’t analyze these factors to find the root causes behind them and they certainly didn’t look for Generic Causes.
They won’t be reopening this site so this accident won’t be repeated here. But I didn’t come away with lessons that TVA’s Management should be learning to improve their performance.
Am I missing something? Review the materials and see what you think.
Nine are now confirmed dead after the Washington Metro train collision.
Two stories now speculate about the failure of circuits to detect the train that was approaching the other train stopped at the station. To read more, see:
The 2-Day TapRooT® Incident Investigation and Root Cause Analysis will be held September 15-16, 2009, and the 3-Day TapRooT®/Equifactor® Equipment Troubleshooting & Root Cause Failure Analysis will be held September 15-17, 2009.
In just a few short days you will learn the basics of the TapRooT® System for finding the root causes of problems PLUS, if you stay the extra day, you’ll learn the Equifactor® Equipment Troubleshooting Technique for Root Cause Failure Analysis of Equipment Problems. Once you find the real root causes using this systematic process, learn to develop effective fixes that will keep problems from happening again.
“The Windy City” borders Lake Michigan and has a population of 2.8 million, making it the largest city in Illinois. The city’s attractions, businesses, and commerce bring 44.2 million visitors annually. Making use of its abundant resources, Chicago has a heritage for hosting major international, national, regional, and local events that include commerce, culture, entertainment, politics, and sports. On June 4, 2008, Chicago was chosen as one of the final four world city candidates to host the 2016 Summer Olympics and is the only American city on the list.
Chicago has 29 miles of lakefront paths making it one of the world’s most walkable and beautiful cities. Not only dubbed “The Windy City” but Chicago is often called “The Friendly City.” Whether you’re shopping, dining, or sightseeing, Chicago has much to offer. Here are a few recommendations for your visit to “The Windy, Friendly City.”
Area Attractions:
The Sears Tower offers an incredible view of the city from the tower’s 99th floor Skydeck. Spanning up to 50 miles, visitors can see four states from the deck.
Lincoln Park Zoo is home to more than 1,000 mammals, reptiles, and birds. Located just minutes north of Chicago, the zoo is a plush oasis in the shadow of skyscrapers.
The Garfield Park Conservatory is Chicago’s botanical gem under glass. The conservatory has been home to rare tropical plants for more than 100 years.
Navy Pier is Chicago’s lakefront playground and the Midwest’s #1 attraction with more than 8.6 million visitors a year. On the shores of Lake Michigan, the boardwalk offers rides, restaurants, exhibitions, shopping, tour boats, dinner cruises and more!
Millennium Park is an award-winning center for art, music, architecture and landscape design. The park has concerts and events all year long and is located in the heart of downtown.
The Field Museum of Natural History boasts a comprehensive set of human cultural anthropology exhibits, including artifacts from ancient Egypt, the Pacific Northwest and Tibet.
The Magnificent Mile is Chicago’s version of the Champs-Elysées: a grand wide boulevard with exclusive shops, museums, restaurants and ritzy hotels.
The Shedd Aquarium is a must-see for anyone visiting Chicago. Entertainment, education, conservation—the aquarium is a definite crowd pleaser!
Dining in Chicago:
From picnics in the park to rooftop eateries to great jazz clubs, Chicago has many ways to enjoy food, fun, and libations.
Andy’s Jazz Club is one of Chicago’s most popular jazz clubs. Great music is accompanied by a full menu including steaks, chicken, seafood, pasta, pizza and more!
The Weber Grill Restaurant will delight even the pickiest of grill masters. Featuring all your favorites, the Grill offers steaks, burgers, barbeque and more.
Avec Restaurant has an extraordinary menu! Try their wood-fired flatbread with shaved beets, roasted olives, marjoram, arugula and manchego cheese or their whole roasted fish with braised cranberry beans, fresh basil, and breadcrumbs.
Lou Mitchell’s Restaurant has been a Chicago favorite for more than 85 years. Known primarily for their breakfast menu, (which they serve all day) they also offer an array of sandwiches. So whether you’re craving breakfast or lunch, Lou Mitchell’s has something for everyone.
Gene & Georgetti Steakhouse has been one of Chicago’s finest since 1941, but don’t let the name fool you. In addition to fantastic steaks, they also have a full Italian menu with all the traditional favorites.
Manny’s Cafeteria and Delicatessen has been a Chicago institution since 1942. Manny’s offers daily specials, and their regular menu is something to behold–from a corned beef sandwich to meatloaf or even salmon, it’s hard to make up your mind when you walk into Manny’s.
Time for a math lesson….. 60 students divided into two classes equals 14 final group exercises and presentations which equals 14 separate incidents analyzed using TapRooT®. Now that was a good day!
Someone asked me yesterday how they could justify the cost of the Summit to their boss. “After all”, they said, “We’re in a giant downturn and everybody is trying to cut, cut, cut. How can I justify the cost to come to the Summit?”
There are three reasons that it is important to attend the Summit even in the current recession (which is ending sooner than you think).
Reason 1: It’s a Great Investment Especially in Bad Times
When money is tight, every dollar counts.
We can’t waste money on a “boondoggle” conference.
Now the good news. The Summit isn’t a boondoggle (although you will have fun while you learn).
Many companies have designated their application of TapRooT® as business critical. And thus keeping their staff up to date on the latest root cause analysis best practices is also business critical. So they allow attending the Summit even in tough times.
So the Summit isn’t a waste of money. Instead, it is a guaranteed investment in the future of your company.
Reason 2: You Can’t Afford Two Years Without Improvement
Many companies put their improvement plans on hold this year because of budget constraints.
If all companies did this, then maybe a single year’s slip wouldn’t seem that bad. But here’s the bad news: They DIDN’T.
Some companies have invested in improvement this year and they are pulling ahead.
If you don’t have an aggressive improvement plan for next year - you’ll be left in the dust.
The Summit is an outstanding place to network, learn the most recent best practices, and talk to experts to develop your improvement plan for 2010.
Your goal will be to catch up with your 2009 goals and leapfrog ahead in 2010.
The improvements you achieve will more than pay for all your conference costs and more.
So you can’t afford to wait to attend the 2010 Summit because by then, you will be too far behind to hope to catch up.
Reason 3: Attending the Summit Can Help You Make Short Term Improvements That Contribute Immediately to the Company’s Bottom Line
The Summit isn’t just about long-term improvements. There are lots of quick fixes and great ways to save effort and money immediately.
Applying the best practices that you learn in a single investigation could produce dramatic savings that immediately add to your company’s bottom line.
One of the attendees at a past Summit told me that a single idea that he got on the first day was going to save his company over a million dollars on a project that they were working on.
Now I can’t guarantee a million dollar idea on the first day for everyone. But we do hand out sheets of paper with $10,000 bills printed on them for people to take notes. The idea is that every time you hear something that can save your company money, you write it down. At the end of the Summit you decide which of these ideas can save your company the most money and you develop a plan to implement the ideas when you return to work.
I’ve seen people with dozens of $10,000 Idea sheets. Their problem was trying to decide between all the great ideas they had. After all, there is only so much one person can accomplish (which is why you should bring a team to the Summit - but that’s a different article).
Therefore, if you really want to save money and help your company make it through this recession - the Summit should be an essential part of your plan.
WHAT ARE YOU WAITING FOR?
See your boss, get permission to attend, and get registered!
What happens when 60 people register for our 2-Day TapRooT® Incident Investigation and Root Cause Analysis Course in Houston with a capacity of 32 people? We bring in two more instructors and open up another class room. No is never the answer when it comes to developing cores skills that help people work safe and more efficient.
Instructors, Dave Janney and Hal Curry, working with their group of energetic students.
Instructor, Ken Turnbull, teaching students how to define Causal Factors.
Don’t let the economy prevent you from missing key training in root cause analysis. You sign up and we will make sure there is always a seat in class for you. Go here to see our up coming courses. If you need a month to get approval and funding, meet me in Boston. http://www.taproot.com/courses.php
Type of Activity: Lifting, crane, rigging, deck operations
Type of Injury: Struck by
Two workers were injured when a lift sling dislodged from a winch hook that did not have a properly operating safety latch.
A service rig crew was in the process of laying down the working floor when they realized that the floor would not be able to come down all the way because a pressure switch was in the way. The operator began to raise the floor so the pressure switch could be turned. As the floor was raised to approximately 45 degrees, the winch hook came unlatched and the floor fell. To save time and speed up the operation, the floorhand and the derrick hand had stepped underneath the floor to pass a 36″ pipe wrench. This placed both individuals directly in the fall path of the floor. Both workers were contacted by the falling floor.
What Went Wrong?
The rig was equipped with two winch lines, the first of which had a damaged Kuplex hook. At the time of the incident the second winch line, which was not used, was fully operational.
The rig crew identified “overhead loads, swing paths, and pinch points” in a safety meeting.
The rig manager was aware that the safety latch on the winch hook was damaged and could not be properly secured. This was verbally communicated to all crew members and documented the hook in a hazard report.
The winch hook was taken out of service for repair after it had been identified as damaged.
A winch hook repair kit had been ordered and was on location at the time of the incident.
After the hook repair kit had been received, the rig manager had attempted to fix the winch hook but was unable to install the new latch locking mechanism.
Believing it would be “OK” for the task of positioning the work floor, the winch hook was returned to service.
When the rig operator began running the winch controls, the floor hand and derrick hand were standing out of the fall path of the rig floor.
At the time of the incident, both workers had moved below the fall zone of the rig floor before the rig floor was in position.
The winch hook was attached to the centre pad and an engineered sling was attached to the work floor. As the floor was raised, the weight of the floor transferred from the center pad sling to the work floor sling. Without the latch lock mechanism, the D-ring pushed out of the winch hook allowing the rig floor to fall.
Corrective actions and Recommendations:
Important hazard identification and control recommendations include:
Never use damaged equipment: Although all members of the rig crew were aware that the winch hook was damaged, the hook was returned to service and used to raise the rig floor.
Always give careful thought about your proper position for the task: In this case, both the derrick hand and the floor hand moved below the rig floor while it was being hoisted by the winch line and were standing directly below an overhead hazard.
Consider the need for design changes before an incident happens: In this case, the company initiated a review the work floor design after the incident and made changes to eliminate the necessity of having the winch hook on the work floor centre pad making lifting operations much safer.
A final thought: Efficiency is important and something we all take a lot of pride in. Always take an extra moment to consider your actions whenever attempting to save time or effort. Make sure your decision is a safe one.
DISCLAIMER:
This Safety Alert is designed to prevent similar incidents by communicating the information at the earliest possible opportunity. Accordingly, the information may change over time. It may be necessary to obtain updates from the source before relying upon the accuracy of the information contained herein. This material is presented for information purposes only. Managers and supervisors should evaluate this information to determine if it can be applied to their own situations and practices.
Source Contact:
This alert is being distributed via a partnership between the International Association of Oil and Gas Producers (http://www.ogp.org.uk/) and Enform (http://www.enform.ca/).
Twitter is a free, simple messaging client which allows you to receive updates to your computer and your phone. All you have to do is follow us to stay on top of all the latest happenings here on the Root Cause Analysis blog. Each time we post a blog entry here, it automatically gets “tweeted” to Twitter.
Follow TapRooT_RCA on Twitter! (If you haven’t joined Twitter, membership is free.)
2. Already attended a TapRooT® Course but want to learn the latest improvements in the techniques? Attend the 2-Day TapRooT® Advanced Techniques Course that is being held on October 5-6 in Nashville, Tennessee. For more information and registration, see:
3. If you want to learn best practices from around-the-world, attend the Investigation and Root Cause Analysis Track at the TapRooT® Summit on October 7-9 in Nashville, Tennessee. In addition to the great Keynote Speakers, you will also attend the following Best Practices sessions:
Accident Analysis Presentations (Attack on the USS Stark, Crandall Canyon Mine, 230 Environmental Incidents)
Success Stories from the Field (common cause analysis of multiple aviation accident investigations and proactive use of Safeguard Analysis for manufacturing quality improvement)
DISCover How To Communicate After an Accident
The Good, The Bad, and The Ugly: Which Describes Your Investigations and Reports and What Can You Do To Improve?
TapRooT® User’s Best Practice Forum
How To Prove That Fatigue Was the Cause of an Incident
Advanced Ideas for Defining Causal Factors
Lessons from Tennessee OSHA Fatality Investigations
Portland will be the location for the 5-Day TapRooT® Advanced Root Cause Analysis Team Leader Training September 28-October 2, 2009. You will learn all the TapRooT® tools, and also many advanced skills, that a Team Leader needs to collect information, analyze root causes, and develop effective fixes that will help your company improve performance.
Portland is a diverse city, and she shows that off in many ways. But the city’s passion for food and drink is something to be reckoned with. New restaurants pop up as often as daily menu specials, and emerging neighborhoods, markets, microbreweries and wineries make Portland a culinary playground.
The 2-Day TapRooT® Incident Investigation and Root Cause Analysis will be held September 15-16, 2009, and the 3-Day TapRooT®/Equifactor® Equipment Troubleshooting & Root Cause Failure Analysis will be held September 15-17, 2009.
The Sears Tower
From picnics in the park to rooftop eateries to great jazz clubs, Chicago has many ways to enjoy food, fun, and libations.
