Measuring Safety/Quality Improvement
Why TRIR is a Poor Measure of Safety and Other Considerations When Measuring Safety and Quality
People often use the Total Recordable Injury Rate (TRIR) as a way to measure and compare safety performance. The problem with the TRIR is that it is not a very reliable measure and the better your safety program is, the less there is to measure. That problem was highlighted in an article published by the CSRA (Construction Safety Research Alliance) and in Process Safety (a journal of the American Society of Safety Professionals) titled: “The Statistical Invalidity of TRIR as a Measure of Safety Performance.”
My goal here isn’t to repeat the discussion in the article. You can read that at the link above. However, key points of the article are (just to name a couple):
- TRIR is not associated with fatalities. Why? Because many (or most?) TRIR injuries could never become a fatality and therefore aren’t indicative of future fatalities or serious injuries.
- TRIRs are almost totally random. They don’t occur at repeatable intervals.
However, we have seen companies significantly reduce both their fatalities/serious injuries rates and their TRIR and have seen other companies that have a bad record of fatalities/serious injuries and TRIR.
Thus, there must be measures and ways to trend infrequently occurring safety statistics (and other statistics, for example some types of quality statistics).
First Develop Your Metrics
At almost every company, trending fatalities is not a good measure. Hopefully, they hardly ever occur. Plus, having a fatality is a poor way to be alerted that safety has declined.
For any negative performance indicator (safety or quality) you may measure a negative outcomes indicator (fatalities, customer complaints, accidental environmental releases, …) but a better indicator would be leading (not lagging) and predictive (not results of past performance – thus, reactive).
Managing a business by using lagging, reactive measures is like driving by looking in the rear-view mirror. Yes, you should occasionally look in the rear-view mirror but, as they say in the financial industry:
Past performance is not indicative of future results.
So, you must know the system you are trying to improve (safety performance, quality results, environmental performance, …) and then find measurable indicators of future performance.
In the book, Performance Measures and Trending for Safety, Quality, & Business Performance, the following proactive performance measures (leading and proactive) are suggested as possible measures for safety and quality performance:
- Pre-job safety meetings conducted divided by the number of jobs completed in a month
- Percent of safe work behaviors observed per total behavior observations in a month
- Safety budget
- Number of hours spent performing safety audits/observations per month
- Number of safety improvements implemented per month
- Results (score) of an employee safety attitude survey
- Total hours spent in safety meetings (including toolbox safety briefs) and pre-job safety planning per 200,000 hours worked
- Number of precursor incidents investigated
- Number of quality corrective actions implemented per month
- Number of quality improvement projects started per month
- Number of quality improvement projects completed per month
- Quality improvement budget
- Number of management hours spent on quality improvement per month
- Number of quality audits performed per month
Note that these indicators may or may not work at your facility. It depends on the state of your program and the processes that you have in place.
Once again, in the book quoted above it says:
In some industries, there are no adequate measures. For example, in the late 1990s, the Institute of Medicine issued a report: To Err is Human: Building a Safer Health System (November 1999). In the report, they estimated that as many as 98,000 people per year may be killed by medical errors. Why did they have to make an estimate of the number of people killed by medical errors? Because there is no standard for or reporting of deaths by medical error.
You might ask … if there isn’t a reliable outcome measure, how do patient safety advocates know if patient safety is getting better or worse? That’s an excellent question.
An idea for developing hospital measures is proposed on page 19 of the book, Performance Measures and Trending for Safety, Quality, & Business Performance.
For more ideas about developing measures, watch this video:
And then try these references:
- Vital Signs, Scorecards, and Goals, Copyright © 2015 an e-book by Kevin McManus at http://greatsystems.com/vital-signs/.
- Measurement of Safety Performance by Dan Petersen, Copyright © 2005, American Society of Safety Engineers, Des Plaines, Illinois.
Once you have developed measures that will reliably predict future performance, you will need to trend them.
What type of graph/trending process do we recommend?
The Process Behavior Chart
What does a Process Behavior Chart look like? Here’s an example:
Would you like to learn more about this advanced trending technique that can be used for reactive or proactive measures (including infrequently occurring safety data)? Then get a copy of Book 8:
Or, attend the Trending Techniques for Incident Data Nest Practice Session at the 2021 Global TapRooT® Summit.
To see all the Best Practice Sessions, see the schedule for the Summit at THIS LINK.
To register for the 2021 Global TapRooT® Summit (either to attend virtually or in-person), click the button below…
PLEASE NOTE: You can SAVE by attending a pre-Summit Course and the Summit and by registering multiple attendees…
The location for the in-person courses and the Summit is the Knoxville Convention Center in Knoxville, TN. If you can’t travel to Knoxville, you can attend virtually. Indicate in-person or virtual when you register.