Case study: Evaluation of GE Healthcare Critical Care Suite

How did RAIQC support the evaluation of an AI-assisted image interpretation tool in the detection of PTXs?

Project overview

Validation of GEHC AI assisted image interpretation tool in the detection of pneumothoraces

By making use of RAIQC's capabilities, GE Healthcare was able to undertake a large diagnostic accuracy study across 6 NHS sites with 18 different readers, exploring the effectiveness of the GE Healthcare Critical Care Suite (GEHC CCS) PTX algorithm in the detection and reporting of a pneumothorax.

Partners

GE Healthcare

GE Healthcare is a leading global medical technology and digital solutions innovator, aiming to enable clinicians to make quicker and more informed decisions

Oxford University Hospitals NHS Foundation Trust

Oxford University Hospitals (OUH) is a world renowned centre of clinical excellence and one of the largest NHS teaching trusts in the UK.

Details

GE Healthcare wished to undertake a Multi Reader Multi Case (MRMC) study evaluating the efficacy of their chest radiograph AI assistive technology detecting pneumothoraces. Readers/Participants of the study included practitioners with different specialities within the clinical setting that could benefit from use of the tool e.g. ICU, ED clinicians and radiographers. Readers also included a range of differing levels of experience i.e. junior, mid-level and senior.

Delivery

RAIQC supported the study throughout, facilitating many aspects including:

  • Provision of a secure web based platform allowing the transfer of study images from NHS Trust
  • Accessibility of the web based DICOM viewer to allow ground truthing by experts, including the annotation of pneumothoraces on study images, through placement of a Region of Interest (ROI)
  • Development of a training module created by experts within RAIQC, to allow readers/participants to familiarise themselves with the platform before study commencement
  • Curation of a study package, with 395 chest x-rays split into bite sized modules over the study period to reduce fatigue in readers
  • Project management of readers alongside study team to ensure timescales were met
  • Data collection from RAIQC in addition to the study metrics also included metrics surrounding reader time taken to review each image

Summary

This research study found increased reader sensitivity after use of the GE AI-assisted pneumothorax detection tool, especially in junior readers. RAIQC provided a streamlined web-based DICOM viewer and a team of dedicated experts to support this large-scale research study compatible with the NHS and industry needs.

  • Tailored specifications to fit GE Healthcare’s needs i.e. ROI
  • Person centred, as a spin-out from the NHS, we place value on ensuring the RAIQC platform is user-friendly for all participants.
  • Focussed on meeting timescales within our project management capabilities, we were able to aid in the completion of the study within

We congratulate Professor Alex Novak and the entire study team in completing and publishing this important study. You can access the published version of the article in the BMJ's Emergency Medicine Journal.

Schematic of reader study process