Improving Organisational Learning from Mefication-related Incident Reports

Abstract

Medication-related incidents remain a significant threat to patient safety, yet the learning potential of incident reports is often underutilised due to challenges in data quality, analysis, and feedback mechanisms. This thesis investigates how organisational learning from medication-related incident report data can be enhanced through the use of data science methods; in particular, natural language processing (NLP). The research comprises four empirical studies. Study 1 investigates how Medication Safety Officers (MSOs) in the English National Health Service (NHS) interpret and use incident report data. Using qualitative interviews with 14 MSOs from diverse healthcare settings, it identifies barriers to learning— such as inconsistent data practices and limited analytical capacity—and highlights the role of MSOs in shaping local safety responses. Study 2 describes the development of the Medication-Related Incident Report Annotation (MRIRA) scheme, which is a standardised method for annotating medication incident reports in preparation for NLP analysis. The scheme was iteratively developed using 55 reports drawn from the NHS England Controlled Drug Reporting database. Study 3 evaluates the reliability of MRIRA by measuring inter-annotator agreement using strict and relaxed F1 scores across two medication safety datasets—the NHS England Controlled Drug (CD) Reporting database and the NHS England Learning from Patient Safety Events (LFPSE)/National Reporting and Learning System (NRLS) databases. This study found strong consistency between annotators in annotating core components, based on dual annotation of 15 reports from each dataset (CD reporting and LFPSE/NRLS). Study 4 assesses the content validity of MRIRA through review by subject matter experts, using both quantitative measures (Content Validity Ratio, Content Validity Index, and modified kappa) and qualitative feedback to confirm the scheme’s relevance, importance, clarity, and comprehensiveness. This study involved 12 experts in medication safety and 11 NLP specialists. Collectively, the four studies offer an original contribution to the field of medication safety by identifying persistent challenges in organisational learning and developing a novel, validated annotation scheme (MRIRA) to support structured analysis of narrative incident reports. The MRIRA framework enables the transformation of free-text data into structured representations, laying the groundwork for future integration with analytical tools and automation. While real-world implementation lies beyond the scope of this thesis, the research establishes a methodological foundation for enhancing feedback processes, strengthening analytical capacity, and enabling more consistent learning from medication-related incidents.