Observer-Reliability Studies for the Evaluation of Automated Boundary Detection Algorithms in Ultrasound Images

Stephen L. Breen, PhD

Our research group is developing a medical robot to assist surgeons with a number of urological procedures, including interstitial laser coagulation and laser ablation for benign prostatic hyperplasia, and interstitial brachytherapy for prostatic carcinoma. During these procedures, ultrasound images of the prostate and adjacent anatomy are acquired to guide the surgeon. As the images are recorded, the boundaries of the prostate must be outlined, not only for the physician’s information, but also to limit the range of motion of the active end of the robot.

It is a goal of our group to develop a reliable technique for the automatic identification of the prostate boundary on ultrasound images. However, the automated analysis of ultrasound images is fraught with difficulty because of several characteristics of the images, notably: poor spatial resolution; noise due to speckle; and limited soft tissue contrast. Any automatic boundary detection method must be evaluated by comparison with a "gold standard". For our application, a gold standard can be obtained by averaging the responses of several expert observers (experienced radiologists) as they manually outline the prostate boundary on computer images.

We are developing software that will send a series of ultrasound images of the prostate over the Internet to a group of expert observers. The expert observers will then outline the prostate boundary, and this outline will be returned to our lab via the Internet. Their responses will be averaged for each image. Subsequently, the automated boundary detection algorithm will analyze the same set of images. The boundaries that have been identified by the automated boundary detection algorithm will be compared to the averaged responses of the radiologists. Before the automated boundary detection algorithm can be integrated with the robotic surgery system, its ability to identify the prostate boundary must be as reliable and accurate as the group of radiologists.

The data gained during the establishment of the gold standard may be useful in the design of other boundary detection algorithms that use certain characteristics of the radiologists’ responses to identify segments of the prostate boundary. We also intend to use this software to develop a similar standard for the automated detection of kidney boundaries.

Related links :