Meet The Experts
MCC Ambassador and Director
Dorricott Metrics and Process Improvement, LTD
Linda B. Sullivan
Metrics Champion Consortium
Q: I work for a MCC member company and we are ramping up our Risk Based Quality Management processes and have found the MCC RAMMT v2.0 as an extremely helpful tool. In review of the tool, I was hoping to receive clarification on how KRIs play into #2 [Critical Data and Risks] and #3 [Risk ID Eval & Control]. After reading the instructions, I understand you utilize #2 to define critical data and processes, creating your if/then statement. These are not always detectable/controllable and may not always lead to a KRI. If the risk is something that is detectable/controllable, it’s my understanding KRI(s) would then be developed setting thresholds and/or QTLs. Can you clarify where the KRIs are initially developed and thresholds are set?
A: Great to hear that you are finding the MCC Risk Assessment & Mitigation Management Tool (RAMMT) 2.0 useful. You are right that tab 3 “Critical Data & Risks” is designed to help develop your risks. You should consider the critical data, the associated processes to capture that critical data, and any other critical processes. The risks should be transferred onto tab 2 “Risk ID Eval & Control” where they can be evaluated. We also recommend you review reference tab A “Risk and Control Library” in case there are additional significant risks that should be added to your list of risks on
Once the risks have been scored for (L)ikelihood, (I)mpact and (D)etectability, you can order the risks by their risk scores and review the risks with high scores. With these risks, you will want to try to reduce them or eliminate them if at all possible – and this will involve taking actions to reduce one or more of the L, I or D scores. Example ways to reduce the likelihood score are improved instructions, training, checklists etc. It may be possible to reduce the impact score by developing a contingency plan that is ready to implement. And the detectability score can be reduced by improving or implementing a detection measure such as a Key Risk Indicator (KRI). The detection method should be one that provides a signal in plenty of time to act – see the MCC paper referenced below for more advice on developing leading metrics. Note that it is not necessary for all risks to be reduced – focus on the risks that matter, those with high risk scores. And not all risks need a detection method. In terms of Quality Tolerance Limit (QTL) parameters, these are really study level KRIs that are focused on key risks. They are focused on risks that if they are realized will lead to study failure e.g. patient dropout too high. The QTLs are the thresholds for those parameters.
Further MCC resources available:
- Paper: “Measure the Right Things at the Right Time: Design KRIs and KPIs that provide timely insights during study conduct” – available here.
- RAMMT Implementation Q&A meeting on 17-Jun-2020 at 10am EDT – Log into the Member portal to register for the meeting
- MCC eLearning Course 301: “Retooling Risk-Based Quality Management Approaches in the Era of ICH E6 (R2): Fundamentals of Clinical Trial Risk Management” – available at a discount to MCC member organizations. Details here.