Clinical Study

Development And Validation Of A Machine Learning Model For Automated Workplace-Based Assessment Of Resident Clinical Reasoning Documentation

Posted Date: Aug 23, 2022

  • Investigator: Danielle Weber
  • Type of Study: Observational/Survey

Clinical documentation is a core component of medical training. In fact, ACGME milestone ICS3 is “appropriate utilization and completion of medical records including effective communication of clinical reasoning." However, since the advent of Electronic Health Records (EHRs) there has been a decline in documentation quality including lack of demonstration of clinical reasoning and infrequent feedback on notes. Barriers to giving feedback include time constraints of supervising faculty and lack of a shared mental model of high-quality clinical reasoning documentation. Several note rating instruments have been validated to assess documentation quality such as QNOTE, PDQI-9, the RED checklist, ANAT, and the IDEA Assessment Tool. While these rating instruments can be effective methods of giving feedback to learners on their documentation, they can be time consuming. Additionally, these note rating instruments possess varying degrees of detailed evaluation of clinical reasoning and of anchors for what constitutes high- vs low-quality clinical reasoning documentation. The purpose of this research is to develop and validate a machine learning (ML) model for automated workplace-based assessment (WBA) of clinical reasoning documentation at two-sites: University of Cincinnati and New York University (NYU). Our specific aims include: 1) Develop a ML-based WBA to assess the presence of clinical reasoning in resident admission notes and to provide specific feedback on areas of deficiency; 2) Gather validity evidence using Messick's framework to support use of the ML-based WBA in both formative and summative assessment; 3) Create a process for implementation to widely disseminate this novel WBA




Documentation, Clinical Reasoning, Natural Language Processing

For More Information:

Danielle Weber

  • This contact page is monitored Monday – Friday during business hours, 8 am to 5 pm EST. Do not rely on communication through this website for medical treatment or care. This website is not designed for such purposes.
    For questions regarding clinical studies at UC Health, contact us at or (513) 245-3417.