Faculty Development to Improve Mini-CEX Ratings

Overview

We will conduct a faculty development workshop with intent to improve interrater reliability and rater accuracy. Outcomes will include interrater reliability, accuracy, and workshop evaluations.

Full Title of Study: “Effect of Faculty Development on Mini-CEX Score Reliability and Accuracy: A Randomized, Controlled Trial”

Study Type

  • Study Type: Interventional
  • Study Design
    • Allocation: Randomized
    • Intervention Model: Parallel Assignment
    • Primary Purpose: Treatment
    • Masking: None (Open Label)
  • Study Primary Completion Date: June 2006

Detailed Description

Hypothesis: An intervention for training faculty preceptors on the use of the mini-CEX will improve inter-rater reliability and classification accuracy. Methods. We will develop a faculty development workshop aimed at improving the inter-rater reliability and accuracy of mini-CEX scores. To evaluate the workshop we will conduct a randomized, controlled trial. Preceptors will be randomly assigned to immediate or delayed workshop. The primary outcome will be inter-rater reliability of mini-CEX scores as preceptors rate videotaped resident-patient encounters before and four weeks after the workshop. We will also evaluate the accuracy of performance classifications. We will also collect mini-CEX ratings of actual resident-patient interactions one year preceding and one year following the workshop, and calculate interrater reliability. Statistical methods. Inter-rater reliability will be determined using the intraclass correlation coefficient (ICC) and corresponding confidence intervals. ICC will be used to determine chance-corrected agreement (accuracy). Discrimination (differences in mean scores among scripted levels of performance) will also be evaluated as a measure of accuracy. Scientific basis. Clinical performance assessments constitute a key element of resident education. The effect of rater training on the inter-rater reliability of mini-CEX scores is unknown. This workshop will utilize methods (rater error and field of reference training) shown to be effective in improving rater accuracy in non-medical fields. Inclusion criteria. All internal medicine residency continuity clinic preceptors are eligible.

Interventions

  • Behavioral: Rater training workshop
    • Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion

Arms, Groups and Cohorts

  • Experimental: Workshop
    • Rater training workshop: rater error training, performance dimension training, behavioral observation training, and frame of reference training using lecture, video, and facilitated discussion
  • No Intervention: Delayed workshop
    • No specific intervention until after follow-up assessment, then receives same workshop.

Clinical Trial Outcome Measures

Primary Measures

  • Interrater reliability (videotaped resident-patient encounters)
    • Time Frame: April-June 2006

Secondary Measures

  • Rater accuracy (agreement with scripted levels of performance on videotaped resident-patient encounters): percent agreement, chance-corrected agreement, mean scores
    • Time Frame: April-June 2006
  • Interrater reliability (actual resident-patient encounters)
    • Time Frame: March 2005 (historical) to June 2007

Participating in This Clinical Trial

Inclusion Criteria

  • Internal medicine residency continuity clinic preceptor at Mayo Clinic Rochester Minnesota Exclusion Criteria:

  • None

Gender Eligibility: All

Minimum Age: N/A

Maximum Age: N/A

Are Healthy Volunteers Accepted: Accepts Healthy Volunteers

Investigator Details

  • Lead Sponsor
    • Mayo Clinic
  • Provider of Information About this Clinical Study
    • David A. Cook, M.D., Mayo Clinic
  • Overall Official(s)
    • David A Cook, MD, MHPE, Principal Investigator, Mayo Clinic

Clinical trials entries are delivered from the US National Institutes of Health and are not reviewed separately by this site. Please see the identifier information above for retrieving further details from the government database.

At TrialBulletin.com, we keep tabs on over 200,000 clinical trials in the US and abroad, using medical data supplied directly by the US National Institutes of Health. Please see the About and Contact page for details.