Metrics Champion Consortium

Twitter  LinkedIn

December 2018

News icon

  Important News

MCC’s Risk and Performance Management Summit a Big Success
More than 70 North American and European-based individuals from large and small biopharma, CROs, risk-based monitoring and consulting organizations participated in the 2018 MCC Risk and Performance Management Summit in Princeton, NJ November 14-15.

The Summit began with keynote speaker Ken Getz from the Tufts Center for the Study of Drug Development. Ken’s session, The Evolving Role of Consensus Metrics in Monitoring, Informing and Predicting Drug Development Performance shared insights about the current state of the industry and outlined a path for moving from company-centric to patient-centric drug development. Next, FDA Senior Policy Advisor Dr. Jean Mulinde, Office of Compliance, CDER shared insights about risk-based quality management, risk-based monitoring and how the agency is reviewing data to identify where to conduct inspections in her session entitled, Maintaining Quality in Clinical Research – Optimizing Planning and Oversight Activities. The remainder of the day’s presentations, panel sessions and breakout group discussions focused on defining what to measure to manage risk and improve performance.

On Day 2 of the Summit, the theme shifted from what to measure to how to use measurements effectively. The panel session and presentations emphasized the need for staff with “critical thinking” skills and how to train staff to analyze data and take effective action to mitigate risk and improve performance.

The Summit concluded with an opportunity for participants to apply their critical thinking skills to uncover the root cause(s) of issues depicted in a case study through a team exercise. Participants stated that the exercise was an “eye opening experience” about the challenges of conducting root cause analysis.

Please contact MCC Executive Director Linda Sullivan or MCC Ambassador Keith Dorricott if you have questions or would like to learn more about the MCC Summit. The MCC will host another summit in 2019.

Imaging Work Group Reconvenes in February
The MCC Imaging Work Group’s meetings will start up again on February 12, 2019 when they begin the process of reviewing the MCC Imaging Performance Metrics.

The work group will start the review process from the perspective of quality risk management in terms of the process steps and potential site Key Risk Indicators.

MCC members interested in joining the Imaging Work Group can update their profile on the MCC Member website and select Imaging Work Group to gain access to the workgroup’s webpage and receive email updates.

Measuring Compliance with CRO Oversight Plans and Staff Selection
The MCC Vendor Oversight Work Group is looking at an approach to selecting CROs and appropriate oversight to ensure qualified and trained staff work on studies.

The work group is discussing CRO oversight plans and different ways to assess whether the plans are being followed and regularly reviewed. Inspections looking at oversight will start with these plans and then look for evidence of compliance.

The work group is also discussing the challenging topic of sub-contractors and their oversight. This is new to ICH E6(R2) and different organizations have different approaches depending on their “risk appetite.” Not all sub-contractors pose the same risk so it is likely there would be a different approach for different sub-contractor types.

The work group will now develop a set of draft metrics for oversight of quality at CROs. A public survey will be launched in the first quarter of 2019 and the feedback will be reviewed before the metrics are finalized. Additionally, the work group will look to develop an At-a-Glance document, webinar and Implementation Guide.

MCC members interested in learning more about the Vendor Oversight Work Group can contact the work group facilitator and MCC Ambassador Keith Dorricott.

Member news icon

  Member News

MCC Membership Increasing as Consortium Welcomes New Members
MCC welcomes the following new members and welcomes back our returning members.

MCC’s newest member organizations are:

  • Myokardia
  • Symbio, LLC

MCC welcomes back the following member organizations:

  • Roche / Genentech  - F. Hoffman-LaRoche
  • Celgene

Working group icon

  Featured Work Group

ICH E6(R2) and Critical Data and Processes for Central Labs
The MCC Central Lab Work Group has been reviewing the metric set from the perspective of quality risk management. Central Lab data is critical data and the process is therefore critical. What does this mean for the Central Lab metrics?

The work group has defined a high-level process map starting with the Technical Specification Document right through to the final data transfer. Some steps are more critical for success than others and the work group highlighted six of the 20 process steps as the most critical to succeed. The metrics were then reviewed in relation to those steps to see if there were gaps. One gap identified was measuring the quality and compliance of lab samples received by the lab. Some of these can be forwarded on to specialty labs and without measuring the samples that arrive there is the potential of passing poor quality samples on to those specialty labs. The work group decided to develop a metric to monitor this.

Section 5.18.3 of ICH E6(R2) is another area in which the work group is focusing. The central lab can be an early detection of issues at a site. Samples should arrive from the subject visit within 1-2 days and the quality, timeliness and responsiveness to queries by the site are all indicators of site performance and of potential site issues. These could be used for detection of site risks along with other indicators, for example through centralized monitoring. The work group reviewed metrics that might be used this way (Key Risk Indicators) and added two additional metrics. Of particular interest to the work group was a discussion on expedited lab supplies. From the perspective of the central lab, this might be seen as no more than an inconvenience but if a site is suddenly requesting urgent lab supply shipments when they weren’t in the past that could be an indicator that something has changed. It could be an indicator of risk that could lead to an investigation that could stop a risk turning into a significant issue leading a site and associated subjects dropping out of the trial.

This approach of reviewing the process from the perspective of quality risk management, both in terms of the process steps themselves and possible site Key Risk Indicators has been very productive. The MCC plans to use the same approach as we open the review of the MCC Imaging Performance Metrics in Q1 2019.

All MCC member organizations are invited to participate in the MCC Central Lab Work Group and the MCC Imaging Performance Work Group. The work group meets the first Tuesday of every month. Contact group facilitator and MCC Ambassador Keith Dorricott for more information.

Calendar icon

  Upcoming Meetings

MCC Monthly Work Group Meetings
Meetings are open to participants from MCC member organizations. To register for a work group meeting or for a detailed list of meeting dates and times, please log into the member portal or contact Customer Service for assistance. Not a member? Join today!

 Work Group Meeting
Central Lab
1st Tuesday of the Month at 10AM EDT
Jan 8, 2019
Feb 5, 2019
Mar 5, 2019
Central Monitoring
3rd Wednesday of the Month at 10AM EDT
Jan 16, 2019
Feb 20, 2019
Imaging Performance
3rd Thursday of the Month
at 10AM EDT
Feb 12, 2019*
Site Selection & Start-Up
2nd Thursday of the Month
at 10AM EDT
Study Quality Trailblazer 
2nd Wednesday of the Month
at 10AM EDT
Vendor Oversight
1st Wednesday of the Month 
at 10AM EDT 
Jan 9, 2019
Feb 6, 2o19
Mar 6, 2019
*Meeting deviates from regularly scheduled frequency.

MCC Metric Sets and Tools Implementation Support Group Meetings
These newly created meetings will be held on a quarterly basis to provide MCC members the opportunity to ask questions and share experiences related to implementing specific metric sets or tools.

 Metric Set/Toolkit Implementation Q&A
Date & Time
Data Management, Biostatistics & Medical Writing Toolkit 2.0
Mar 12, 2019 10 AM EDT
Site Contracting Metrics Toolkit 1.0
Jun 13, 2019 10 AM EDT
TMF Metrics Toolkit 2.0
Jun 11, 2019 11 AM EDT


Connect with MCC
MCC staff looks forward to having good conversations with MCC Members in 2019.

SCOPE 2019
February 19, 2019
Vendor Performance Metrics and KPIs
Moderator - Linda Sullivan, MCC Co-Founder & Executive Director

February 20, 2019
Ensuring Harmony Between all Stakeholders – Sponsors, CROs and Site – When Site Sourcing
Panelists – Linda Sullivan, MCC Co-Founder & Executive Director

Learn more about SCOPE 2019.

Ask icon

  Ask the Experts

MCC Answers Your Metric Questions
Q: We've been using a measure of cycle time for lab queries to monitor site responsiveness. Recently, we had a few nasty surprises - the cycle time for specific queries was very long and caused missing critical data in our database but the median cycle time metric did not change very much. Why were we unable to recognize that there were problems? Are we using the right measure?

Meet The Experts

  Keith Dorricott
  MCC Ambassador and Director
  Dorricott Metrics and Process Improvement, LTD

  Linda B. Sullivan
  Executive Director
  Metrics Champion Consortium

A: What you are describing occurs when there are occasional extreme values in the distribution of results - cycle times that are very long. When measuring the average using a median, a few extreme values will not alter the average much, if at all. The figure below illustrates a scenario like the one you described – the median in Q2 is less than Q1 but there are more long cycle time results reported in Q2. If you don’t view the results in a graph and/or review interquartile range (IQR) results - which allow you to see how closely measurements are grouped around the median and the spread of outlier values – you may not know that you have a problem.

Are you using the right measurement? To avoid encountering this problem in the future, you need to review the IQR values and/or review an additional metric to see the complete picture. For example, you could look at the proportion of queries that are answered within set number of calendar days (e.g. 7 or 14 days). This type of metric allows you to recognize when you have queries with long cycle times. But it might drive the wrong behavior - taking seven calendar days for all queries rather than trying to get all of them closed as soon as possible.

The MCC Central Lab Work Group has recently been having a discussion along these lines and has decided to keep two metrics for query cycle time - both a median and percent closed in seven calendar days. With MCC metrics, we clearly define the purpose of each metric, what it measures and also what it does not measure.

Education icon


Is Your Organization Keeping Up with Industry Changes?
Changes in the industry are forcing organizations to do more than just collect and enter data – they have to understand what the data is saying.

To be ICH E6(R2) compliant, organizations need to change the way their trials are conducted and reported, making the need for metric knowledge more important than ever. MCC’s eLearning Education Program is the tool every organization needs to give employees that needed knowledge.

The unique eLearning education program allows professionals to gain a solid metric knowledge foundation with a flexible format that can be customized to meet your needs. The MCC eLearning Education Program:

  • Teaches industry-leading metric understanding
  • Provides a self-study, on-demand learning program to accommodate your busy schedule
  • Accommodates several learning styles
  • Allows you to pause and pick up at a later time
  • Can be easily customized and deployed on your organization’s Learning Management System

Learn more about the entire MCC Education Program and view demos on the following eLearning courses:

MCCe201: The Elements that Make Metrics Effective

MCCe202: Developing Effective Metrics Using a Structured Process

MCCe203: Effective Metric Visualizations to Gain Valuable Performance Insights 

Join icon

  MCC Membership and User Accounts

Access to MCC metric sets, tools and other assets are limited to employees of MCC member organizations. Please visit the MCC website or email MCC Membership Director Terry Holland for membership information.

Setting Up User Accounts to Log onto Member Portal
If you work for a MCC member organization, you are welcome to set up an individual user account to take advantage of the benefits MCC Membership offers. Click HERE to start the process.


Join the MCC LinkedIn Group to learn about industry performance metrics and stay informed of new developments.


Metrics Champion Consortium,  300 N. Washington St.,  Falls Church, VA 22046,  United States 
Phone (617) 948-5100 | Toll free (866) 219-3440

© 2020 Metrics Champion Consortium

Untitled Document