Multi-Stage and Multi-Target Data-Centric Approaches to Object Detection, Localization, and Segmentation in Medical Imaging

Abstract

Object detection, localization, and segmentation in medical images are essential in several medical procedures. Identifying abnormalities and anatomical structures of interest within these images remains challenging due to the variability in patient anatomy, imaging conditions, and the inherent complexities of biological structures. To address these challenges, we propose a set of frameworks for real-time object detection and tracking in ultrasound scans and two frameworks for liver lesion detection and segmentation in single and multi-phase computed tomography (CT) scans. The first framework for ultrasound object detection and tracking uses a segmentation model weakly trained on bounding box labels as the backbone architecture. The framework outperformed state-of-the-art object detection models in detecting the Vagus nerve within scans of the neck. To improve the detection and localization accuracy of the backbone network, we propose a multi-path decoder UNet. Its detection performance is on par with, or slightly better than, the more computationally expensive UNet++, which has 20% more parameters and requires twice the inference time. For liver lesion segmentation and detection in multi-phase CT scans, we propose an approach to first align the liver using liver segmentation masks followed by deformable registration with the VoxelMorph model. We also propose a learning-free framework to estimate and correct abnormal deformations in deformable image registration models. The first framework for liver lesion segmentation is a multi-stage framework designed to incorporate models trained on each of the phases individually in addition to the model trained on all the phases together. The framework uses a segmentation refinement and correction model that combines these models' predictions with the CT image to improve the overall lesion segmentation. The framework improves the subject-wise segmentation performance by 1.6% while reducing performance variability across subjects by 8% and the instances of segmentation failure by 50%. In the second framework, we propose a liver lesion mask selection algorithm that compares the separation of intensity features between the lesion and surrounding tissue from multi-specialized model predictions and selects the mask that maximizes this separation. The selection approach improves the detection rates for small lesions by 15.5% and by 4.3% for lesions overall.