Metrics Champion Consortium
MetricInsights

Twitter  LinkedIn

December 2020

News icon

  Important News

MCC Clinical Trial Optimization Podcast – New Episode Available!
PhoneTestImageThe MCC Clinical Trial Optimization Podcast brings together clinical research stakeholders to exchange ideas, share knowledge, and think creatively about how to oversee, manage and optimize clinical trial planning and execution in a twice-monthly podcast. For more information about the podcast and to view a full listing of show episodes please click here.

EPISODE 10
How the Right Metrics Enable You to Develop World-Class Sponsor-CRO Relationships with Linda Sullivan, Steve Crow, and Keith Dorricott Listen Now

EPISODE 11
TMF Accredited Training Program is Key To Building Your World-Class TMF with Linda Sullivan and Sholeh Ehdaivand Listen Now

Subscribe today! Apple PodcastsSpotifyStitcherAndroid

Effectiveness Checks for CAPAs – why are they so difficult to get right?
Does your organization struggle with checking the effectiveness of CAPAs? The MCC Audit, Compliance & CAPA Community of Practice will meet on 12-Jan-2021 to discuss the topic: “Effectiveness Checks for CAPAs – why are they so difficult to get right?” MCC members are invited to join us for a panel discussion followed by a Q&A session.

Session Moderator: Keith Dorricott, MCC Ambassador

Expert Panelists:

  • Elizabeth Atkin, Senior Manager, CPS Quality Assurance, Covance
  • Dawn Niccum, Senior Director, QA and Compliance, InSeption Group
  • Oleg Shevaldyshev, Associate Director, Quality Assurance, PRA Health Sciences

The panelists will explore the following topics:

  • If the check relies on measuring repeated issues, how do you determine whether an issue is truly a repeat of a past issue?
  • Should a CAPA be kept open until the effectiveness check is complete?
  • Do all CAPAs need effectiveness checks?

For more information about this community of practice group, please visit
https://metricschampion.org/compliance-capa-process-metrics-work-group/

How Does Your Organization Track and Analyze False Positive or False Negative Risk Signals in Centralized Monitoring?
The MCC Centralized and Site Monitoring Process Metrics Group will meet on 21-Jan-2021 to review the details for the remaining proposed metrics for the risk & issue management portion of the monitoring process – cycle time for managing a risk and % “false positives”.

For more information about this metric development work group, please visit
https://metricschampion.org/centralized-and-site-monitoring-metrics-work-group/

MCC Work Group Developing Translation Metrics to Supplement eCOA Process Metric Toolkit
The MCC eCOA Work Group released an updated version of the eCOA performance metrics in July 2018. However, eCOA is a rapidly developing area so MCC initiated a metric review process earlier this year. The work group will meet on 19-Jan-2021 to discuss the eCOA translation process reviewing a process map, discussing the challenges and deciding on the Key Performance Questions (KPQs) members want metrics to help answer. Metrics will be defined to help answer these KPQs at a follow-up meeting.

For more information about this metric development work group, please visit
https://metricschampion.org/ecoa-epro-metrics/

MCC Members Have Opportunities to Share Knowledge at Upcoming Online Meetings

 Work Group
Topic
Date
Audit, Compliance & CAPA CoP
“Effectiveness Checks for CAPAs – why are they so difficult to get right?”
12-Jan-2021
11am-12pm EST
Centralized Monitoring CoP
“You’ve detected a risk – now what?”
20-Jan-2021
11am-12pm EST
Centralized & Site Monitoring Process Metrics
Review remaining proposed metrics for the risk & issue management portion of the process – cycle time for managing a risk and % “false positives” detail. Begin discussion on the set-up portion of the process.
21-Jan-2021
11am-12pm EST
eCOA Process Metrics
Focus on the translation process for eCOA by reviewing a process map, discussing the challenges and determining Key Performance Questions (KPQs) we want our metrics to answer. Metrics will be defined to help answer these KPQs.
19-Jan-2021
11am-12pm EST
Inspection Readiness/TMF CoP
“Can a TMF Completeness Measure Tell the Whole Story?”
09-Feb-2021
11am-12pm EST
Site Selection & Start-up Metrics
Continued focus on the ethics/IRB submission and approval process – process map, Key Performance Questions (KPQs) we want our metrics to answer, and possible metrics.
13-Jan-2021
11am-12pm EST

Member news icon

  Member News

MCC welcomes the following new members and thanks those for their continued membership.

MCC newest members are:

  • Centrexion Therapeutics
  • Lokavant, Inc.
  • Novartis and subsidiaries
  • TB Alliance

MCC thanks the following members for their renewal:

  • Debiopharm International SA
  • Eli Lilly & Co.
  • Halloran Consulting Group
  • Harper Clinical Research Consulting
  • Praxis Precision Medicines
  • Triumph Research Intelligence

Working group icon

  Featured Work Group

Site Selection & Start-Up Metrics Work Group
SiteSelectionImageHow Can We Measure Ethics/Reg Submission & Approval?

Tim Schuckman, Chief Commercial Officer, Ethical Review WCG joined the December work group meeting to provide insights about the ethical review process and suggested metrics to include in the MCC metric set. During the session, he discussed:

  • How the process works currently in the USA (local IRB vs central IRB)
  • Measurements currently used – sponsor and IRB cycle times and quality measures (e.g. application right first time)
  • Opportunities to improve the process.

The session was recorded and available for members to view by logging into the member portal and visiting the Site Selection & Start-Up Metrics Work Group webpage.

For more information about this metric development work group, please visit
https://metricschampion.org/site-selection-working-group/


Calendar icon

  Upcoming Meetings

MCC Metric Development Work Groups
Members work collaboratively, through structured discussions, to develop and refine metric sets and tools. Work group members are encouraged to regularly participate in monthly, 60-minute web-based meetings. On occasion members will be asked to complete structured reviews and provide feedback outside of meetings (assignments typically require 15-30 minutes to complete).

To register or for more information please log into the member portal or contact Customer Service for assistance. Not a member? Join today!

 Work Group
Meeting Frequency
Date
Centralized and Site Monitoring Process Metrics
3rd Thursday of the Month at 11AM ET
Jan 21, 2021
Feb 18, 2021
Mar 18, 2021
eCOA Process Metrics
3rd Tuesday of the Month at 11AM ET
Jan 19, 2021
Feb 16, 2021
Apr 20, 2021
Site Selection and Study Start-Up Metrics
1st Tuesday of the Month at 11AM ET
Jan 13, 2021*
Feb 2, 2021
Mar 2, 2021
*Deviates from regularly scheduled day or time.

MCC Communities of Practice
Members share ideas and practical approaches to address challenges unique to these communities.

To register or for more information please log into the member portal or contact Customer Service for assistance. Not a member? Join today!

Communities of Practice
Meeting Frequency
Date
Artificial Intelligence (AI)/
Machine Learning (ML)
2nd Tuesday every other Month at 11AM ET
TBD
Audit, Compliance & CAPA
2nd Tuesday every other Month at 11AM ET
Jan 12, 2021
Mar 9, 2021
May 11, 2021
Centralized Monitoring
3rd Wednesday of the Month at 11AM ET
Jan 20, 2021
Feb 17, 2021
Mar 17, 2021
Consultancy
Quarterly at 11AM ET
Feb 26, 2021 
Inspection Readiness/TMF
Quarterly at 11 AM ET
Feb 9, 2021
Apr 13, 2021
Small Sponsor Community
of Practice
Every other Month at 11AM ET
TBD
Study Quality Trailblazer
2nd Wednesday of the Month at 11AM ET
TBD
*Deviates from regularly scheduled day or time.
 

MCC Implementation Support Group Meetings
Members are provided the opportunity to ask questions and share experiences related to implementing specific metric sets or tools at quarterly web-based meetings.

To register or for more information please log into the member portal or contact Customer Service for assistance. Not a member? Join today!

Work Group
Date & Time
Cardiopulmonary Metrics
Jan 26, 2021 11AM EST
Central Lab Metrics
Jan 27, 2021 11AM EST
Data Management & Biostatistics Metrics
Jan 26, 2021 10AM EST
Imaging Process Metrics
Jan 27, 2021 10AM EST
Risk Assessment & Mitigation Management Tool
Jan 28, 2021 10AM EST
Site Contracting Metrics
Jan 28, 2021 11AM EST
Vendor Oversight
Jan 20, 2021 11AM EST
 

Connect with MCC at Industry Events 
MCC staff looks forward to having good conversations with you in 2021.

WCG-MCC Clinical Trial Risk and Performance Management Virtual Summit
See you September 2021


Ask icon

  Ask the Experts — MCC Answers Your Metric Questions

Meet The Experts

  Keith Dorricott
  MCC Ambassador and Director
  Dorricott Metrics and Process Improvement, LTD




  Linda B. Sullivan
  Executive Director
  Metrics Champion Consortium
 
 
 

Q: We are looking at how to set levels on vendor oversight metrics where penalties should be imposed. I couldn’t see anything related to penalties in the MCC metric sets. What levels would you recommend?

A: We are all used to the phrase “what gets measured, gets done” so why not add an extra incentive to what gets measured to make sure it gets done? There are several reasons why this can be detrimental:

  1. It will encourage “gaming the metrics” – where the focus is on achieving the metric goal rather than the actual outcome that is desired. For example, high-pressure sales quotas at Wells Fargo bank led to the opening of fake bank accounts to meet the target. In clinical trials, we have come across a situation where site monitors received a penalty for not submitting Monitoring Visit Reports within a set time. This led to site monitors submitting blank reports in the system if they were unable to meet the timeline.
  2. Success or failure of processes are rarely down to one party or individual meaning the incentive or penalty is perceived as unfair. This can be detrimental to an open, trusting relationship which most sponsors want with their vendors. For example, the timeliness of central lab tests is not just dependent on the central lab but also on the quality of the samples they receive from the sites, the couriers who shipped the samples, and the sponsor/CRO who provided training to the sites on the requirements. By penalizing the central lab for not meeting the timeliness requirements, you may actually be taking action against the wrong party.
  3. The targets are unlikely to take account of changing circumstances. This, again, can be detrimental to an open, trusting relationship. For example, the COVID-19 pandemic led to difficulties in sites entered data within agreed timelines. It is surely unfair to penalize them for this – but making the judgement of an acceptable reason for not meeting a target is subjective. It is likely to be confrontational and so detrimental to an open, trusting relationship.
  4. Setting the target level is no longer about the best we can achieve. There are likely to be heated debates about target levels if there are associated incentives or penalties. The target will likely be a compromise rather than focused on what the process can potentially achieve.
  5. The focus becomes very narrow rather than on the overall purpose of the process. It drives behavior of having to meet targets rather than improving the process to make it more effective / efficient. Effort is wasted on the narrow focus rather than achieving the overall purpose.

We recommend using metrics to understand processes – what works well, what needs improvement? Along with targets, they are a way of ensuring a clear understanding between parties of what matters. Parties can then work together with the data to understand causes of performance that does not meet (or exceeds) expected levels and then implement actions to improve (or repeat) the performance. MCC metrics for vendors include an indication of which parties’ performance is being measured and this should help with selection of the right metrics and also in metrics use.

We would urge caution in using metrics and associated targets for applying penalties or incentives.


Education icon

  Education

WCG Metrics Champion Consortium Expands Its Learning Institute with New eLearning Courses
Addition of Clinical Pathway’s On-Demand, Performance Management and Good Clinical Practice Training Courses Enriches Resources Available to MCC Members

MCC now offers 13 new eLearning courses on performance management and good clinical practice (GCP) developed by Clinical Pathways, LLC to provide stakeholders across the clinical research community with high-quality, effective training resources.
Learn More


Join icon

  MCC Membership and User Accounts

Access to MCC metric sets, tools and other assets are limited to employees of MCC member organizations. Please visit the MCC website or email MCC Membership Director Terry Holland for membership information.

Setting Up User Accounts to Log onto Member Portal
If you work for a MCC member organization, you are welcome to set up an individual user account to take advantage of the benefits MCC Membership offers. Click HERE to start the process.

1f6751a2-5be3-491c-856c-cf2d51c4a008.gif

Join the MCC LinkedIn Group to learn about industry performance metrics and stay informed of new developments.

 
WCG MCC Logo

Metrics Champion Consortium,  300 N. Washington St.,,  Falls Church, VA 22046,  USA 
Phone (617) 948-5100 | Toll free (866) 219-3440

© 2021 Metrics Champion Consortium

 
Untitled Document