HIL: Interactive Segmentation & Validation

About the Project

Medical AI often suffers from a 'cold start' problem where high-quality annotations are too slow to produce manually. I developed an end-to-end Human-in-the-Loop (HIL) pipeline designed to bridge this gap. The core of the system is a hybrid segmentation engine that uses a smart Watershed algorithm to identify rough tissue centroids which then serve as a set of instructions for the Segment Anything Model (SAM). This automated pass provides pathologists with near-perfect starting masks. The true value of the system lies in the validation loop: experts quickly verify or adjust these masks, and the system automatically formats this high-fidelity data to retrain and improve the underlying model. This transforms the pathologist's role from manual creator to high-level validator, reducing annotation time by orders of magnitude.

Engineering Challenges

01Segmenting Paired Instances

Oysters are typically processed in pairs per slide. Fragmented or touching tissue often confuses standard models, leading to 'merged' or incomplete animal instances.

Solution: Engineered a pre-segmentation 'Instruction' layer. I used a customized Watershed algorithm to perform a first pass, identifying the distinct centroids of each oyster. These coordinates were then passed to SAM as specific point-prompts, ensuring individual, high-fidelity isolation for both instances.

02The Manual Annotation Bottleneck

Standard pixel-level annotation of 1TB+ medical datasets is prohibitively expensive and slow.

Solution: Implemented a validation-centric workflow. By automating the bulk of the segmentation work and providing a clear path for expert verification, the system uses 'corrected' masks as a fresh training signal, creating an iterative improvement cycle that saves months of manual work.