Assessing the performance of ultrasound imaging systems using images from relatively high‐density random spherical void phantoms: A simulation study

Abstract

Background The development of clinically meaningful, objective, and quantitative methods for assessing the performance of ultrasound imaging systems represents a continuing area of interest. One approach has been to image phantoms with randomly distributed spherical voids. Purpose The objectives of this study were: (1) to explore the potential of using relatively high-volume fraction random spherical void (RSV) phantoms as an approach for quantitatively assessing the performance of ultrasound imaging systems; (2) to identify potential metrics that can be used to provide quantitative assessments of images obtained from relatively high-volume fraction RSV phantoms; and (3) to demonstrate changes in the quantitative metrics that can occur as image features are degraded. Methods A series (10 each) of computer-simulated RSV phantoms with a range of RSV volume fractions (0.05, 0.15, and 0.25) were generated. To determine the number of image planes necessary to provide robust measurements, a series of consecutive planes (ranging from 1 to 150) within each type of simulated phantom were analyzed. The observed circular cross-section radii histogram distributions (representing the intersection of each plane with the local distribution of spherical voids) were compared with the theoretical histogram distribution. Simulated phantom images were produced by adding speckle and degradation of imaging system performance was modeled by averaging 1 to 9 neighboring planes to represent increasing elevation plane thicknesses. Quantification of the performance of the imaging system was determined by measuring the: (1) mean number of circular cross-sections detected per image frame; (2) mean fractional area of circular cross-sections detected per image frame; (3) agreement of observed circular cross-section radii histogram distribution with the theoretical distribution (Chi-square statistic); and (4) contrast and contrast-to-noise ratio as a function of observed circular cross-section radius. Results Results suggest that analyses of a sufficient number of image planes (providing over approximately 3000 total circular cross-sectional areas) provides excellent agreement between the observed and theoretical histogram distributions (mean Chi-square < 0.004). For the 0.15 volume fraction series of simulated RSV phantoms, using 150 image plane analyses, phantom images show decreasing mean number of circle cross-sections detected per frame (31.5 ± 0.3, 28.4 ± 0.3, 28.2 ± 0.3, 26.3 ± 0.3, and 25.3 ± 0.3); decreasing mean fractional area of circle cross-sections per frame (0.157 ± 0.002, 0.133 ± 0.001, 0.133 ± 0.001, 0.111 ± 0.001, and 0.108 ± 0.001); and a decreasing agreement with the theoretical histogram distribution of radii (Chi-square values: 0.070 ± 0.004, 0.140 ± 0.005, 0.149 ± 0.007, 0.379 ± 0.011, and 0.518 ± 0.010) for 1, 3, 5, 7, and 9 plane averages, respectively. Contrast and contrast-to-noise measurements as a function of observed circular cross-section radius also demonstrate marked changes with simulated image degradation. Conclusions Results of this simulation study suggest that analyses of images obtained from relatively high-density RSV phantoms may offer a promising approach for assessing ultrasound imaging systems. The proposed measurements appear to provide reproducible, robust, quantitative metrics that can be compared with corresponding theoretical values to provide quantifiable, objective metrics of imaging system performance.