On the usage of average Hausdorff distance for segmentation performance assessment: hidden error when used for ranking

European Radiology Experimental

Table 1 Example ranking of a set of segmentations with increasing number of errors by average Hausdorff distance (AHD) and balanced average Hausdorff distance (bAHD)

Segmentations	Count of errors	AHD values	AHD rank	bAHD values	bAHD rank
Ground truth	0	0	1	0	1
E1	1	0.308	2	0.314	2
N1_E1	2	0.455	3	0.467	3
N1_K3_E1	3	9.836	5	23.487	4
N1_H1_K3_E1	4	10.138	7	24.925	5
N1_H1_K3_E1_P991	5	10.111	6	24.928	6
N1_H1_K3_G1_E1_P991	6	10.213	8	25.394	7
N1_H1_K3_G1_E1_P991_M0	7	10.345	9	25.435	8
N1_H1_K3_G1_E1_V2_P991_M0	8	10.638	11	25.613	9
N1_H1_K3_G1_E1_R1_V2_P991_M0	9	10.628	10	25.690	10
N1_H1_K3_G1_E1_R1_V2_P991_C992_M0	10	9.768	4	25.843	11

Values of performance measures (in voxels) are shown with the resulting rankings for one example set. The segmentation ranking of bAHD perfectly correlates with the count of errors in the simulated segmentations. The traditional AHD, however, fails to properly rank the segmentations in line with the number of errors they contain. Error abbreviations are given in the Segmentations column. Letters stand for error types and numbers 1, 2, 3 state the intensity levels subtle, moderate and severe, respectively. K3 False positive errors in the skull area, C992 Increased radius of the carotid artery (false positive voxels), M0 Missing M1 segment of the middle cerebral artery (false-negative voxels), E1 False-positive segmentation of the optical nerve and adjacent fat tissue, N1 False-positive segmentation of the middle meningeal artery (false-positive voxels), G1 False-positive segmentation of the sigmoid sinus, V2 False-negative segmentation of small vessels, R1 Random voxels added throughout the image (false positives), H1 False-positive segmentation of the meninges, P991 Increased radius of the posterior communicating artery (false positives)