|
Many computer-aided diagnostic problems involve three-class classification (e.g., abnormal (malignant), normal, and benign classes). A general treatment of the performance of a 3-class classifier results in a complex sixdimensional (6D) ROC space for which no analysis tools are available at present. A practical paradigm for performance analysis and a figure of merit (FOM) are yet to be developed. The area Az under the ROC curve plays an important role in the evaluation of two-class classifiers. An analogous FOM for three-class classification is highly desirable. We have been investigating conditions for reducing the 6D ROC space to 3D by constraining the utilities of some of the decisions in the classification task. These assumptions lead to a 3D ROC space in which the true-positive fraction (TPF) of one class can be expressed as a function of the two types of false-positive fractions (FPFs) from the other two classes. In this study, we investigated the properties of the 3D ROC surface for a maximum-likelihood classifier under the condition that the three class distributions in the feature space are multivariate normal with equal covariance matrices. Under the same conditions, we studied the properties of the normalized volume under the surface (NVUS) that was previously proposed as an FOM for a 3-class classifier. Under the equal covariance Gaussian assumption, it can be shown that the probability density functions of the decision variables follow a bivariate log-normal distribution. By considering these probability density functions, we can express the TPF in terms of the FPFs, and numerically integrate over the domain of the TPF to obtain the NVUS. We have compared the NVUS value obtained by directly integrating the probability density functions to that obtained from a Monte Carlo simulation study in which the 3D ROC surface was generated by empirical "optimal" classification of case samples in the multi-dimensional feature space following the assumed distributions. The NVUS values obtained from the direct integration method and the Monte Carlo simulation were found to be in good agreement. Our results indicate that the NVUS exhibits many of the intuitive properties that a proper FOM should satisfy and may be used as a performance index under the conditions that we imposed on the utilities.
|
