Metrics Champion Consortium

Twitter  LinkedIn

January 2020


Letter from the Executive Director

Dear Colleagues and Friends:

As we begin a new year and decade, I am excited about what we accomplished in 2019 and the work we are planning to engage in over the next year. MCC continues to grow and provide a unique place where clinical research stakeholders can be part of a community working together to share ideas, define performance measures, develop risk and quality scoring tools and discuss how to use metrics as the foundation to drive process improvement. During 2019, MCC hosted 44 one-hour virtual meetings for 11 work groups and community of practice groups. More than 700 individuals participated in work group meetings – many in multiple groups. Some of the topics discussed and work completed by the work groups include:

  • Development and release of the Vendor Oversight Quality Metric toolkit v1.0
  • Development of the Vendor Oversight Finance Metric toolkit v1.0 (to be released Q1 2020)
  • Release of updated versions of the Central Lab Metric and Site Contracting Metric toolkits
  • Review of the Imaging Metric toolkit (to be released Q1 2020)
  • Presentations and discussions about various centralized monitoring topics – data normalization and quality tolerance limit chapters will be added to the Centralized Monitoring Guidance document in Q1 and Q2 2020
  • Review of possible updates to the Protocol Operational Complexity Scoring Tool and the Risk Assessment and Mitigation Management Tool in 2020
  • Implementation of TMF metrics in small sponsor programs

The 2nd Annual MCC Clinical Trial Risk and Performance Management Summit held on September 4-5 in Philadelphia received rave reviews. The 2019 Summit had a 40% increase in participants, many who returned for a second year. The Summit focused on five themes:

  1. Tracking the Patient Engagement Movement and its Impact on Clinical Research Execution: Keynote speaker Kenneth Getz, Director of Sponsored Programs and Associate Professor, Tufts University; Founder and Board Chair, CISCRP shared insights about the state of the drug development industry and the need for and impact of patient engagement approaches.
  2. Metric Program Implementation: Practical approaches and lessons learned from industry veterans.
  3. Risk-based Quality Management: Emerging practices in the adoption of risk-based quality management, including the impact of ICH-E6(R2) updates, risk management for small sponsors, quality tolerance limits, developing and using key risk indicator libraries and the application of artificial intelligence and machine learning.
  4. Risk-based Vendor Oversight: The impact of ICH E6(R2) on vendor oversight, key considerations to developing healthy sponsor/vendor engagement, and using metrics to manage vendor risk and performance.
  5. Applying Critical Thinking Skills to Utilize Metrics Effectively: Participants worked in teams to uncover the root cause(s) of issues described in a case study.

The Summit format of presentations, panel discussions, audience polling, facilitated breakout group discussions and small group exercises encouraged active participation of all attendees, and fostered the sharing of ideas and collaborative discussions. Participant feedback showed that we continued to provide a “pre-competitive”, collaborative environment that enables participants to engage in meaningful dialogue with their peers. As mentioned by several participants, “It gave me time to network and THINK.” Please visit to see how our plans are shaping up for the 2020 Summit.

Looking ahead to 2020, MCC will support the development of several new metric toolkits, including the site selection and start-up, and centralized/onsite monitoring process metric toolkits. Additionally, some existing metric toolkits will be reviewed and updated including the Imaging and eCOA metric sets. Finally, MCC will expand the number of Community of Practice (aka Knowledge Sharing) groups from three to five groups with the addition of the Artificial Intelligence/Machine Learning and Compliance – CAPA groups. Additional information about all of the MCC Communities of Practice and Metric Work Groups is available on the MCC website at

On behalf of the entire MCC team, thank you for your support and participation. We look forward to working with you in 2020 and beyond.

Linda B. Sullivan

News icon

  Important News

2020 MCC Clinical Trial Risk and Performance Management Summit – Call for Abstracts
Is your organization interested in sharing a case study about using MCC metrics and/or quality tools? Contact Linda Sullivan, MCC Executive Director, or Keith Dorricott, MCC Ambassador, to learn how your organization can be a part of the 2020 MCC Clinical Trial Risk and Performance Management Summit.

The 2020 Summit will be September 9-10 in Philadelphia. The 2019 Summit was a huge success with themes exploring: Tracking the Patient Engagement Movement and its Impact on Clinical Research Execution, Metric Program Implementation, Risk-based Quality Management, Risk-based Vendor Oversight and Applying Critical Thinking Skills to Utilize Metrics Effectively. Contact Linda or Keith if there are topics you would like to see included for the 2020 Summit and if you are interested in participating as a presenter or panellist. Click here to learn more.

Dr. Frank M. Berger to Present Protocol Complexity Case Study at January 23 Study Quality Trailblazer Meeting
There is still time to register to hear Dr. Frank M. Berger, Head of Analytics, Global Clinical Operations at Boehringer Ingelheim, present “Case Study – Protocol Complexity” during the January 23, 2020 Study Quality Trailblazer meeting at 10AM EST.

Dr. Berger will cover the following topics:

  • What is complexity?
  • Trial complexity and protocol complexity
  • Common beliefs around trial complexity
  • Different perspectives
  • How to measure trial/protocol complexity
  • Call to action: Let’s collaborate!

MCC members can register for the January 23 meeting by visiting the Study Quality Trailblazer Work Group page on the MCC member website or contacting Lara Knitter, MCC Customer Service & Product Support, at for assistance.

Announcing the New Artificial Intelligence and Machine Learning Group
MCC is pleased to announce the launch of its new Artificial Intelligence (AI) and Machine Learning (ML) Knowledge Sharing Group. This MCC best practice community group will explore ways to leverage AI and ML to improve quality, reduce cycle time in clinical trials and help deliver the right drugs to patients.

Discussions for the group will include:


  • What are the challenges in transforming current SMEs into people who can leverage and or design data science solutions?
  • Change management, what works, where are the challenges?


  • Use-cases for adopting AI/ML. What works, where are the challenges?
  • Developing an AI/ML Roadmap.


  • What is AI/ML and how can it apply to clinical trials?
  • Where are the opportunities for adapting current technologies to be more data science friendly (adding in coding and other process data into systems)?

The first AI/ML meeting will be February 11, 2020 with additional meetings taking place on the second Tuesday of every other month with industry participation. The presentations will be spread between the People, Process and Technology aspects of implementing an AI/ML and how they apply to process improvement in clinical trials. These presentations will be heavily interactive but lead by industry experts.

There will also be a LinkedIn group where MCC members can have off-line discussions and share resources and articles.

MCC members can register for the February 11 meeting by logging into the member portal on the MCC website and clicking on the Upcoming MCC Work Group Meetings link. Members can also visit the AI/ML homepage for meetings recordings, white papers and other resources.

Click here to learn more about the AI/ML Knowledge Sharing Group.

MCC Proposes New Compliance & CAPA Community of Practice Group; Take the Survey!
The MCC is discussing creating a new Community of Practice (CoP) group for Compliance & CAPA to continue discussions on the processes and differences between member companies’ implementation.

This CoP would provide members the opportunity to:

  • Explore the processes of clinical quality compliance & CAPA
  • Review relevant MCC metrics, definition and tools, and revise and develop new ones as needed
  • Share learning and challenges of clinical quality compliance & CAPA

Virtual meetings for this group would take place every other month and anyone from an MCC member organization is welcome to register for any meeting. Each meeting will focus on a different topic and consist of an approximate 20-minute presentation followed by an open discussion and a Q&A session.

Interested MCC members are encouraged to take a four-minute survey that will gather input about your organization’s level of interest in participating in this group and topics to explore. The survey closes on January 31, 2020.


Introducing the Newly Designed MCC Member Portal
The redesign of the MCC Member Portal makes it easier to find what you are looking for. Users can enter the website by selecting their role and/or areas of interest.

After logging into the member portal, users will now see the below homepage.

Role/Area of Interest Image

Once an area is selected, users will then be directed to that functional area page. For example, the Clinical Operations (including monitoring) page is shown below.

Clinical Operations Image

This new design brings you everything need with just one click!

Going to SCOPE? Connect with the MCC
Linda Sullivan, MCC Executive Director and Keith Dorricott, MCC Ambassador, look forward to connecting with you during the 11th Annual SCOPE meeting in Orlando February 18-21, 2020. Linda and Keith will be leading track and breakout sessions on Risk-Based Monitoring, Vendor Oversight, and Vendor Performance Metrics and KPIs.

Linda will give the chairperson’s remarks on Risk-Based Quality Management Strategy on February 19 before co-presenting Bridging the Gap Between Issue Management and Risk-based Quality Management with Ann McCabe, Director, Process Excellence & Risk Management, Daiichi Sankyo, Inc. Both sessions fall under the Implementing Risk-Based Monitoring Track (Part 1).

Keith will lead an interactive breakout discussion with Caro Unger, Senior Clinical Operations Standards and Innovation Manager, Clovis Oncology, on February 19. He will be at Table 22 discussing Vendor Performance Metrics and KPIs.

Keith will also lead a presentation on Vendor Oversight: Metrics, KPIs and ICH E6(R2) on February 21, This session falls under the Managing Outsourced Clinical Trials track.

If you would like to schedule a time to meet with Linda and Keith during SCOPE, please contact MCC Membership Director Terry Holland at who will assist you in setting up a meeting.

Learn more about Linda and Keith’s sessions.

Member news icon

  Member News

MCC welcomes the following new members and thanks those for their continued membership.

MCC’s newest members are:

  • H3 Biomedicine
  • LMK Clinical Research Consulting
  • United Therapeutics

MCC thanks the following members for their renewal:

  • Bill & Melinda Gates Medical Research Institute
  • Debiopharm International
  • Eli Lilly & Co.
  • GW Pharmaceuticals
  • Halloran Consulting Group
  • Harper CRC
  • Jazz Pharmaceuticals
  • Just in Time GCP
  • MedQIA 
  • Myokardia
  • Praxis Precision Medicines
  • Sarepta Therapeutics
  • University Health Network

Working group icon

  Work Group

Small Sponsor Community of Practice Group to Discuss Vendor Oversight for Small Sponsors
MeetingTableMaria Makarovskaya and Jacob Ehrlich from Pure Health Consulting will focus on the topic of vendor oversight when they lead the discussion “Working with CROs – Small Sponsor Edition: How do you do vendor oversight?” at the next meeting on Tuesday, February 11, 2020.


Meeting discussions will include:

  • How to execute effective oversight
  • Lessons learned for successful engagements
  • Small company considerations
  • Importance of relationship dynamics
  • Planning and execution for successful engagements

MCC members can register for the February 11 meeting by logging into the member portal and visiting the Small Sponsor Community of Practice webpage and knowledge portal.

Calendar icon

  Upcoming Meetings

MCC Metric Development Work Groups
Members work collaboratively, through structured discussions, to develop and refine metric sets and tools.

Work group members are encouraged to regularly participate in monthly, 60-minute web-based meetings. On occasion members will be asked to complete structured reviews and provide feedback outside of meetings (assignments typically require 15-30 minutes to complete). To register or for more information please log into the member portal or contact Customer Service for assistance. Not a member? Join today!

 Work Group
Meeting Frequency
Centralized and Site Monitoring Metrics
3rd Thursday of the Month 10AM EST
Feb 27, 2020* 11AM EST**
Mar 19, 2020
Imaging Performance
3rd Tuesday of the Month 10AM EST
Jan 21, 2020
Feb 25. 2020* 11AM EST**
Site Selection and Start-Up Metrics
1st Tuesday of the Month 10AM EST
Feb 4, 2020
Mar 3, 2020 11AM EST**
Vendor Oversight
1st Wednesday of the Month 10AM EST 
Feb 5, 2020
*Meeting date deviates from regularly scheduled frequency.
**Meeting time deviates from regularly scheduled frequency.

MCC Best Practices Communities
Members share ideas and practical approaches to address challenges unique to these communities.

To register or for more information please log into the member portal or contact Customer Service for assistance. Not a member? Join today!

Best Practice Community
Meeting Frequency
Centralized Monitoring
3rd Wednesday of the Month 10AM EST
Jan 22, 2020*
Feb 12, 2020*
Mar 18, 2020
Small Sponsor Community
of Practice
Every Other Month 11AM EST
Feb 11, 2020
Study Quality Trailblazer
2nd Wednesday of the Month 10AM EST
Jan 23, 2020*
Feb 12, 2020 11AM EST**
*Meeting date deviates from regularly scheduled frequency.
**Meeting time deviates from regularly scheduled frequency.

MCC Implementation Support Group Meetings
Members are provided the opportunity to ask questions and share experiences related to implementing specific metric sets or tools at quarterly web-based meetings.

 Metric Set/Toolkit Implementation Q&A
Date & Time
Cardiopulmonary Metric Toolkit v2.1 (ABPM, ECG, Spirometry)
Mar 4, 2020 11 AM EST
Central Lab Metrics v3.0
Mar 11, 2020 12 PM EDT**
Compliance & CAPA Process Metrics
Data Management, Biostatistics & Medical Writing Toolkit v2.0
Mar 10, 2020 10 AM EDT
eCOA Metric Toolkit v2.0
Risk Assessment & Mitigation Management Tool v2.0
Mar 12, 2020 10 AM EDT
Site Contracting Metrics Toolkit v1.0
Mar 11, 2020 10 AM EDT
TMF Metrics Toolkit v2.0
Mar 10, 2020 11 AM EDT**
**Meeting time deviates from regularly scheduled frequency.

To register or for more information please log into the member portal or contact Customer Service for assistance. Not a member? Join today!

Connect with MCC at Industry Events 
MCC staff looks forward to having good conversations with you in 2020. Please let us know if you are interested in scheduling a one-on-one meeting at any of the following events.

February 18-21, 2020
Orlando FL

Track: Implementing Risk-Based Monitoring – Part 1
Date: February 19, 2020
Session Title: “Risk-based Quality Management Strategy”
Session Participants: Linda Sullivan, Co-Founder & Executive Director, WCG-Metrics Champion Consortium and Ann McCabe, Director, Process Excellence & Risk Management, Daiichi Sankyo, Inc.

Interactive Breakout Discussions
Date: February 19, 4:20-5:10 PM
TABLE 22: Vendor Performance Metrics and KPIs

  • How effective are your KPIs for measuring vendor performance and quality?
  • What is your strategy for establishing KPIs and metrics?
  • What are the key areas that should be evaluated for vendor performance and quality?

Moderators: Keith Dorricott, Ambassador, Metrics Champion Consortium; Director, DMPI Ltd., and Caro Unger, Senior Clinical Operations Standards and Innovation Manager, Clovis Oncology

Track: Managing Outsourced Clinical Trials
Date: February 21, 2020
Session Title: Vendor Oversight: Metrics, KPIs and ICH E6 R2
Session Participant: Keith Dorricott, Ambassador, WCG-Metrics Champion Consortium

Learn More

Clinical Trial Innovation Summit
April 6-8, 2020
Cambridge, MA

Track: Risk-based Monitoring
Presentation: “Is Your Centralized/Onsite Site Monitoring Program Being Implemented per ICH E6(R2)? How Do You Know?”
Session Participant: Linda Sullivan, Co-Founder & Executive Director, WCG-Metrics Champion Consortium

Roundtable Discussion: “Implementation of Clinical Development Risk Management”
Session Participants: Linda Sullivan, Co-Founder & Executive Director, WCG-Metrics Champion Consortium and Jonathan Rowe, Pfizer

Learn More

DIA 2020 Global Annual Meeting
June 14-18, 2020
Washington D.C. 

Track: 02 Clinical Trials and Clinical Operations
Presentation: “Management of Centralized, Remote, and Site Monitoring Models Using Metrics to Improve Process and Oversight”
Session Participant: Linda Sullivan, Co-Founder & Executive Director, WCG-Metrics Champion Consortium

Learn More

MCC Clinical Trial Risk and Performance Management Summit
Sept 9-10, 2020
Philadelphia, PA

Please contact MCC Executive Director Linda Sullivan or MCC Ambassador Keith Dorricott if there are topics you would like to see included for the 2020 Summit and if you are interested in participating as a presenter or panelist.

Learn More

Ask icon

  Ask the Experts — MCC Answers Your Metric Questions

Meet The Experts

  Keith Dorricott
  MCC Ambassador and Director
  Dorricott Metrics and Process Improvement, LTD

  Linda B. Sullivan
  Executive Director
  Metrics Champion Consortium

Q: I have recently changed jobs and my new company wants me to help them implement CRO oversight metrics. I am familiar with the MCC metrics and my new company is an MCC member, so I have access to the Vendor Oversight metrics. However, I am having difficulties getting the vendor oversight team to adopt the MCC metrics. One person commented that they once looked at an MCC metric workbook and were overwhelmed. How can I work with the team to support adoption of MCC metrics?

A: MCC metric workbooks contain detailed metric definitions, including information summarizing the discussion and decisions made by metric development work group members. This information provides important context about how, why and when to use each metric. However, it can be difficult to use the metric workbook to select which metrics to adopt – people get overwhelmed with the details. To address this challenge, the newer MCC metric toolkits include implementation guides and metric selection tools that allow you to select metrics based on the following criteria:

  • Performance questions you are seeking to answer
  • Issues you want to manage
  • Metric level: high level KPIs vs detailed drill down metrics
  • Availability of data to generate metrics 

The diagram below shows the four-step process we recommend organizations follow to select and implement the right MCC metrics for their programs. First, the team needs to establish an understanding about why they want metrics. What will team members do with the information? Are there specific key performance questions they are seeking to answer? Or issues that they want to focus on managing better? Next, the team should select an MCC metric toolkit and review the brief introduction video and Implementation Guide to ensure that the metric set covers the right area.

After completing steps 1 and 2, the team is ready to select metrics. We don’t recommend implementing all metrics in a metric set, they should use the MCC selection tools to help them identify the metrics that meet their specific needs. The tools will show the team which metrics in the set meet their selection criteria and the data elements needed to generate the metrics. Now the team can use the metric workbook as a reference guide to learn more about the metrics identified by the selection tools. The detailed metric definition in the workbook includes the rationale for use of the metric – why it is important but also what it doesn’t measure. There is detailed information about how to calculate the metric, performance targets as well as suggested analysis to perform when the results do meet expectations.

The final step is to use Tool E, the metric reporting planning worksheet, to document which metrics are selected and plan how the metrics are going to be reported and used. This final step should be completed with your CROs. Your CROs will be able to bring additional knowledge to help ensure the metrics program is a success. Tool E is designed as a collaborative tool can be used to discuss why the metrics are selected and who will be responsible to tracking and reporting the metrics.

Flow Chart Image

Education icon


Make a Resolution to Gain Metrics Knowledge with the MCC eLearning Institute
Organizations have to do more than just collect data – they have to understand what the data is saying. MCC’s eLearning Institute is the tool every organization needs to give employees that needed knowledge.

The unique eLearning education program allows professionals to gain a solid metric knowledge foundation with a flexible format that can be customized to meet your needs. The MCC eLearning Institute:

  • Teaches industry-leading metric understanding and risk management concepts
  • Provides a self-study, on-demand learning program to accommodate your busy schedule
  • Accommodates several learning styles
  • Allows you to pause and pick up at a later time
  • Can be easily customized and deployed on your organization’s Learning Management System

Learn more about the entire MCC Education Program on the following eLearning courses:

MCC e201: The Elements that Make Metrics Effective

MCC e202: Developing Effective Metrics Using a Structured Process

MCC e203: Effective Metric Visualizations to Gain Valuable Performance Insights

MCC e204: MCC Metric Master Certification Independent Project

MCC e301: Retooling Risk-Based Quality Management Approaches in the Era of ICH E6(R2): Fundamentals of Clinical Trial Risk Management

Join icon

  MCC Membership and User Accounts

Access to MCC metric sets, tools and other assets are limited to employees of MCC member organizations. Please visit the MCC website or email MCC Membership Director Terry Holland for membership information.

Setting Up User Accounts to Log onto Member Portal
If you work for a MCC member organization, you are welcome to set up an individual user account to take advantage of the benefits MCC Membership offers. Click HERE to start the process.


Join the MCC LinkedIn Group to learn about industry performance metrics and stay informed of new developments.


Metrics Champion Consortium,  300 N. Washington St.,,  Falls Church, VA 22046,  USA 
Phone (617) 948-5100 | Toll free (866) 219-3440

© 2021 Metrics Champion Consortium

Untitled Document