We are using Bayesian artificial neural networks (BANNs) to eliminate false-positive detections in our computer-aided diagnosis schemes. In the present work, we investigated whether BANNs can be used to estimate likelihood ratio, or ideal observer, decision functions for distinguishing observations which are drawn from three classes. Three univariate normal distributions were chosen representing three classes. We sampled 3,000 values of x for each of 10 training datasets, and 3,000 values of x for a single testing dataset. A BANN was trained on each training dataset, and the two outputs from each trained BANN, which estimate p(class 1x) and p(class 2x), were recorded for each value of x in the testing dataset. The mean BANN output and its standard error were calculated using the ten sets of BANN output. We repeated the above procedure to estimate the means and standard errors of the two likelihood ratio decision functions p(xclass 1)/p(xclass 3)/p(xclass 2)/p(xclass 3). We found that the BANN can estimate the a posteriori class probabilities quite accurately, except in regions of data space where outcomes are unlikely. Estimation of the likelihood ratios is more problematic, which we attribute to error amplification caused by taking the ratio of two imprecise estimates. We hope to improve these estimates by constraining the BANN training procedure.

Previous studies have shown that model observers can be used for automated evaluation and optimization of image compression with respect to human visual performance in a task where the signal does not vary and is known a priori by the observer (signal known exactly, SKE). Here, we extend previous work to two tasks that are intended to more realistically represent the day-to-day visual diagnostic decision in the clinical setting. In the signal known exactly but variable task (SKEV), the signal varies from trial to trial (e.g., size, shape, etc) but is known to the observer. In the signal known statistically task (SKS) the signal varies from trial to trial and the observer does not have knowledge of which signal is present in that trial. We compare SKEV/SKS human and model observer performance detecting simulated arterial filling defects embedded in real coronary angiographic backgrounds in images that have undergone different amounts of JPEG and JPEG 2000 image compression. Our results show that both human and model performance at low compression ratios is better for the JPEG algorithm than the JPEG 2000 algorithm. Metrics of image quality such as the root mean square error (or the related peak signal to noise ratio) incorrectly predict a JPEG 2000 superiority. Results also show that although model and to a lesser extent human performance improves with the trial to trial knowledge of the signal present (SKEV vs. SKS task), conclusions about which compression algorithm is better (JPEG vs. JPEG 2000) for the current task would not change whether one used an SKEV or SKS task. These findings might suggest that the computationally more tractable SKEV models could be used as a good first approximation for automated evaluation of the more clinically realistic SKS task.

We describe a probit regression approach for maximum-likelihood (ML) estimation of a linear observer template from human-observer data in two-alternative forced-choice experiments. Like a previous approach to ML estimation in this problem [Abbey & Eckstein, Proc. SPIE, Vol. 4324, 2001], our approach does not make any assumptions about the distribution of the images. The previous approach utilized a regularizing prior distribution to control the degrees of freedom in the problem. In this work, we constrain the observer template to be represented by a limited number of linear features. Standard methods of probit regression are described for estimating the feature weights, and hence the observer templates. We have used this probit regression method to estimate human-observer templates for the detection of a small (5mm diameter) round simulated mass embedded in digitized mammograms. Our estimated templates for detecting the mass contain a band of heavily weighted spatial frequencies from 0.08 to 0.3 cycles/mm. We show comparisons between the human-observer template data, and the templates of a number of linear model observers that have been investigated as perceptual models of the human.

We advocate a task-based approach to measuring and optimizing image quality; that is, optimize imaging systems based on the performance of a particular observer performing a specific task. This type of analysis can require numerous images and is, thus, infeasible with real patients. Researchers are forced to employ statistical models from which they can produce as many images as required. We have developed methods to accurately fit statistical models of continuous objects to real images. The fitted models can be used for hardware optimizations as well as image-processing optimizations. We have employed a continuous lumpy object model in this research and found that our method can accurately determine model parameters in simulation.

Kundel et al. Suggested the use of circle cues to assist human observers during signal-known-exactly (SKE) detection experiments. The circles were bipolar (with concentric black and white rings) and centered on potential locations of simulated masses added to mammographic backgrounds. They used a large circle cue (diameter 6.4 cm) and a background size of 7.7 cm (referred to the initial mammogram). They found significant detection performance improvement compared to the no cue conditions. In our previous experiments, we use mammographic background sizes of 6.1 cm and smaller circles with sizes dependent on lesion size. Our circle sizes were selected to subjectively optimize utility but choices may not have been the best. Also, detectability may also depend on background size. In this work, we present human observer results for detecting a realist mass added to mammographic backgrounds using 30 conditions (all combinations of the mass scaled to 3 sizes, 2 background sizes and 5 circle sizes). Performance did not depend on background size. For the smallest mass size (1 mm, 8 pixels), detectability decreased as circle size increased. There may be an optimum near a circle/mass size ratio of 4. The optimum size ratio for the 4 mm mass was 3. For the 16 mm mass, detectability decreased as steadily as circle size increased. The smallest size ratio used was 1.2.

One way to increase the efficiency of large-scale observer performance studies that utilize receiver-operator- characteristic (ROC) analysis would be to identify a group of observers whose ROC results consistently reflect a strong sensitivity to imaging mode differences. We reviewed the data of four large ROC studies performed by our group. The primary index used to classify observers was defined as the absolute difference between readers' A_zs for two modes divided by the standard deviation, appropriately corrected for correlation. Using this index, observers were ranked as to their ability to detect differences between modes. We performed four different analyses on data that included 119,550 observations in four studies and 14 different modes performed by 20 readers. With the exception of one data set, there was no correlation between the level of performance of observers, as measured by A_z, and their ability to discriminate between modes. Analyses designed to identify observers who were consistent in their rankings as to their ability to discriminated between modes was also unsuccessful in showing differences. This study suggests that without the use of special procedures or training, it is not likely that most research facilities will be able to select a group of highly experienced observers who consistently detect small differences among modes.

Small, efficient observer performance study methods, such as multi-point rank order and forced choice, can be used to either determine the necessity to perform a large scale, receiver operator characteristic (ROC) study or to set the boundary conditions at which it makes sense to perform an ROC study. Because these studies often require observers to discriminate between (among) modes having small visual differences, we decided to address the issue of observers' abilities to make distinctions between modes in these types of non-ROC studies. In this project we reviewed the data of six different non-ROC studies that included a total of 24 observers to determine whether some behave as better mode discriminators. Because of the actual small differences, most observers performed poorly in identifying differences between or among modes. At the same time, at least one and sometimes more observers could identify extremely small differences between modes. These differences were statistically significant. Our results indicate that a good mode discriminator in one study may not perform as well in another such study. Non-ROC studies can be highly sensitive to differences between modes. However, large differences in observer performance combined with observer inconsistency across studies necessitates that these studies include multiple observers.

This paper investigated the use of the Breast Imaging Reporting And Data System (BIRADS) Lexicon in ROC mammography experiments. Analysis was based on data from parallel ROC experiments performed at two Institutions with different readers and databases to compare film to digitized mammography. Seven readers participated in the studies and read approximately 200 cases each in two formats: film and digital or softcopy. Reporting was done using BIRADS categories 1 through 5. Training was done with a separate set of cases and included detailed review of the relationship between BIRADS and a standard ROC discrete 5-point rating scale. The results from both sites showed equivalency between film and softcopy mammography. Decisions using the BIRADS categories showed no unsampled ROC regions and no degenerate data. Fits yielded smooth ROC curves that correlated to clinical practice. In a qualitative evaluation, all observers indicated preference in using the BIRADS classes instead of a discrete or continuous rating scheme. Familiarity with the rating process seems to relieve some of the bias associated with the interpretation of digitized mammograms from computer monitors (softcopy reading). Our results suggested that BIRADS categories can be used in comparative ROC studies because they represent a scale familiar to the reader that can be followed consistently and they provide a rating approach that accounts for both positive and negative cases to be evaluated and categorized.

With low dose multi-slice CT for screening of lung cancer, physicians are now finding and examining increasingly smaller nodules. However as the size of detectable nodules becomes smaller, there may be greater differences among physicians as to what is detected and what constitutes a nodule. In this study, 10 CT screening studies of smokers were individually evaluated by three thoracic radiologists. After consensus to determine a gold standard, the number of nodules detected by individual radiologists ranged from 1.4 to 2.1 detections per patient. Each radiologist detected nodules missed by the other two. Although a total of 26 true nodules were detected by one or more radiologists, only 8 (31%) were detected by all three radiologists. The number of true nodules detected by an integrated automatic detection algorithm was 3.2 per patient after radiologist validation. Including these nodules in the gold standard set reduced the sensitivity of nodule detection by each radiologist to less than half. The sensitivity of nodule detection by the computer was better at 64%, proving especially efficacious for detecting smaller and more central nodules. Use of the automatic detection module would allow individual radiologists to increase the number of detected nodules by 114% to 207%.

Mammography is a widely used technique to screen for breast cancer. However, due to the complexity of the breast tissue and to the low prevalence of cancer in the screening population, between 10-30% of retrospectively visible cancers are not reported. Faulty visual search, that is, not examining the area where the cancer is located, is responsible for a third of these misses, but all other unreported cancers attract some amount of visual attention, as indicated by the duration of visual gaze in the location of the lesion. Thus, perceptual and decision making mechanisms must be understood, in order to aid radiologists to detect cancer at earlier stages. We have been working on modeling these mechanisms by using spatial frequency analysis, in a process that is inspired by the rules and complexity of the eye-brain system. In this paper we analyze the different decision outcomes of experienced mammographers and less experienced radiology residents, undergoing a mammography rotation, when examining a case set of 40 two-view mammogram cases. We also characterize the interplay between local factors, which are related to the area of the image that attracts visual attention, and global factors, which are related to breast sampling, as they affect decision outcome for each group.

To assess the performance levels of a radiologist in detecting non-cued masses and microcalcification clusters depicted on digitized mammograms, 120 mammograms depicting 57 verified masses and 38 microcalcification clusters were selected. During an observer performance study, the images were displayed on a computer monitor. Except for the first mode where no regions were cued, the images were cued in the other four modes using a combination of two cueing sensitivities (90% and 50%) and two false-positive rates (0.5 and 2 per image). One reader ignored all cued regions and identified suspicious regions only in non-cued areas. We examined how the performance of this observer was affected using the different cueing modes. Detection sensitivities of non-cued mammographic abnormalities ranged from 43% to 60%, which were lower (P<EQ0.05) than the 76% sensitivity achieved in the non-cued mode. Increasing the false-positive cueing rate from 0.5 to 2 per image reduced (<0.05) the detection sensitivity in the non-cued areas. When using a low performing cueing system, the performance reduction in non-cued areas might offset performance gains in cued areas, resulting in a negative impact on overall performance of the radiologists.

The purpose of the study was to determine if there were differences between the interpretations of radiographic images resulting from digitizing films using a recently developed CCD unit, and the readings of the original films. The general hypothesis to be tested was that there were no significant differences in the measures of accuracy, sensitivity, specificity and ROC analyses when the interpretations related to the two modes were compared. The authors selected 120 radiographic examinations for the study from departmental teaching files, which included chest, abdomen, extremity and other cases that were considered difficult to interpret. The authors also selected six specific abnormalities visualized on 60 of the cases as true positives to classify the reports. After anonymizing the patient identification, the films were digitized and independently interpreted by four board-certified radiologists. Each reader read all of the examinations, half on film alternators and the other half on a high-resolution soft-copy workstation. No reader interpreted the same examination more than once. As of this date, the preliminary results indicate that the hypothesis will be accepted, but more analyses of the data must be performed to confirm the early findings. The additional work will include complete verification of data entry and classification of interpretations, a detailed review of perceived image quality and completion of the ROC analysis by pairs of readers. If the results are confirmed, radiologists, other physicians and administrators will have another reliable option to conventional film practice through increased access to remote primary diagnosis and consultation using high-speed telecommunication media.

Rates of agreement for diagnosing ARVC/D using MRI are not known. 45 cardiac MRI cases were sent to 13 expert radiologists. Only 1 of 45 cases had complete agreement among readers. ARVC/D was more likely to be reported present if fat in the myocardium was reported, if the RV chamber size was enlarged, and if RV configuration was abnormal. A normal LV chamber size was more likely with a negative report and an enlarged LV chamber size was just as likely to be called positive as negative. The radiologists were more likely to correctly call a no ARVC/D case negative (71%) than positive, but they were below chance (47%) in calling an ARVC/D case positive. 13% of the cases were rated as having poor image quality, 42% fair, 38% good and 7% excellent. There was no relationship between image quality and percent readers agreeing on presence/absence of ARVC/D. Interreader variability for ARVC/D using MRI cardiac film images is quite high. Image quality does not seem to be a major contributing factor. Inclusion of MRI cine loop images of the heart, standardized protocol, and utilization of the most current MRI equipment may improve reader agreement as well as diagnostic accuracy.

In this study, we investigated how changing the kVp and mAs used to acquire digital mammograms affects detection of mammographic lesions. A Lorad Full Field Digital Mammography system was used to expose an anthropomorphic breast phantom at x-ray tube voltages ranging from 24 to 32 kVp and output factors ranging from 20 to 120 mAs. Lesions were added at various intensities to digital mammograms, and lesion visibility was assessed using a subjective probability of the lesion being present, with the image contrast varying from visible to invisible. Four observers ranked the visibility of a large mass lesion (2 cm x 1.3 cm) and a calcification lesion with a diameter of ~1mm. Visibility of both lesions was constant between 40 mAs and 120 mAs (constant 28 kVp), but the visibility of both lesions was significantly lower at 20 mAs. For clinically relevant radiographic techniques, quantum noise does not appear to affect observer performance for detection of lesions in the size range of 1mm to 2cm. At a constant mAs, there was a trend showing a reduction in calcification visibility with increasing kVp, but this was not statistically significant (p=0.057).

The purpose of this study was to evaluate the frequency and reasons of disagreement between film and full-field digital mammography (FFDM) interpretations observed in a prospective clinical trial performed with the GE Senographe 2000D system. The data from 643 mammography examinations comprising both digital and film mammograms were analyzed for this purpose. Reports indicated that 455 findings were identified on the digital softcopy reading and 457 findings on the standard film mammography with 408 discrepancies. Findings with discrepancies were matched and analyzed. A reason was identified and a relative conspicuity score of 0 to 10 was assigned to each finding at the time of resolution; 0 corresponded to a finding highly conspicuous on digital, 10 to a finding highly conspicuous on film, and 5 denoted equal visibility on both. After review, agreement was established between the two modalities in 73.3% of the findings; 13.5% of findings were seen better on digital and 13.2% of the findings were seen better on film. Approximately 63% of the discrepancies occurred due to variability in the reporting style of the radiologists and/or unavailability of prior films for comparison. Three cancer cases were identified in this study; two were seen on both modalities and one only on film. In conclusion, no statistically significant differences were observed between digital and film mammography, a result that despite the small size of our dataset is in agreement with previous reports. Inter-observer variability, display differences, and presentation disagreements are the main reasons for interpretation differences that are primarily identified in the classification and BIRADS assignment.

Radiologists resort to image magnification in x-ray fluoroscopy to better visualize small interventional devices. Image intensifier (II) systems use analog magnification with x-ray exposure inversely proportional to the area of field of view (FOV) so as to maintain light output for the camera. Flat-panel (FP) detector images can only be magnified using digital interpolation. We quantitatively investigated image quality of digital and analog magnification using a task of detecting a partially deployed stent in simulated x-ray fluoroscopy image sequences. Exposure for FP magnification was varied in an adaptive forced choice experiment so as to match performance with II magnification. II exposure was set at a nominal value of 5.0 (mu) R per frame at 23 cm FOV and using the standard exposure strategy, increased to 9.15 and 13.49 (mu) R per frame at 17 and 14 cm FOV, respectively. Image quality improved 1.44 +/- 0.065 and 1.64 +/- 0.073 times by going to the 17 and 14 cm FOV, respectively. For equivalent image quality, FP required 4.00 +/- 0.30, 9.50 +/- 0.62, and 9.85 +/- 0.64 (mu) R per frame at normal, mag-1 and mag-2, respectively. As compared to II, FP gives significant dose savings of 20 +/- 6% and 27 +/- 5% at the normal and highest magnification modes, respectively. A human-observer model describes results and can be used to predict results for similar acquisition and processing methods.

A physical and clinical evaluation of four different display systems intended for the presentation of medical x-ray images was performed. State-of-the-art b/w display devices were studied as well as a low-cost color display. The systems tested were: a) CRT 21' b/w 1K, b) CRT 21' b/w, 2K, c) CRT 21' color, 1K, d) LCD 20.8' b/w, 1.5K. All displays were, as far as possible, adjusted to conform to DICOM 3.0 part 14. The physical evaluation included quantities such as resolution, flickering and uniformity. The clinical evaluation was performed by 15 radiologists using visual grading analysis of one phantom chest image and 12 clinical images of the chest and small bones. One subtle pathological structure and one anatomical structure were rated. The CRT b/w 1K display was used as the reference display system. All displays were evaluated at two different luminance levels (160 and 320 cd/m²) and viewed under two different ambient light conditions (10 and 40 lux). The LCD 1.5K display was rated best and the color CRT display was rated worst for both luminance levels. The result for the color CRT display - unable to produce more than 120 cd/m² - at the high ambient light setting was very poor.

The conventional approach to performance evaluation for image compression in telemedicine is simply to measure compression ratio, signal-to-noise ratio and computational load. Evaluation of performance is however a much more complex and many sided issue. It is necessary to consider more deeply the requirements of the applications. In telemedicine, the preservation of clinical information must be taken into account when assessing the suitability of any particular compression algorithm. In telemedicine the metrication of this characteristic is subjective because human judgement must be brought in to identify what is of clinical importance. The assessment must therefore take into account subjective user evaluation criteria as well as objective criteria. This paper develops the concept of user based assessment techniques for image compression used in telepathology. A novel visualization approach has been developed to show and explore the highly complex performance space taking into account both types of measure. The application considered is within a general histopathology image management system; the particular component is a store-and-forward facility for second opinion elicitation. Images of histopathology slides are transmitted to the workstations of consultants working remotely to enable them to provide second opinions.

Each year, approximately 60% of all US women over the age of 40 utilize mammography. Through the matrix of an imaging technology, this Population of Patients (POP) interacts with a population of approximately 20,000 physicians who interpret mammograms in the US. This latter Population of Diagnosticians (POD) operationally serves as the interface between an image-centric healthcare technology system and patient. Methods: using data collected from a large POD and POP based study, I evaluate the distribution of several ROC curve-related parameters in the POD and explore the health policy implications of a population ROC curve for mammography. Results and Conclusions: Principal Components Analysis suggests that two Binormal parameters are sufficient to explain variation in the POD and implies that the Binormal model is foundational to Health Policy Research in Mammography. A population ROC curve based on percentiles of the POD can be used to set targets to achieve national health policy goals. Medical Image Perception science provides the framework. Alternatively, a restrictive policy can be envisioned using performance criteria based on area. However, the data suggests this sort of policy would be too costly in terms of reduced healthcare service capacity in the US in the face of burgeoning demands.

Based on observations made at our institution that diagnostic images could be read on cathode ray tube (CRT) displays controlled with 8-bit hardware, a reconsideration of the bit depth for primary interpretation of radiological images seemed in order. Using actual CRT performance and human visual system (HVS) models with target size, surround luminance and external noise parameters (detector, display and image), CRT luminance modulation as a function of bit depth is compared with the HVS detection threshold modulation. While best case HVS performance requires, at least, 10-bit control to avoid creating luminance artifacts, probable HVS performance is estimated when targets are small, surround luminance is not equal to target luminance and external noise is included as a mask. In this light, the HVS threshold modulation is elevated to such an extent that 8-bit hardware is sufficient. It is shown that when implemented in 8-bit space at the display, the DICOM display function standard creates additional noise and potentially, artifacts. Acceptable image display in an 8-bit space will be discussed with respect to display data representation alternatives such gamma space, which is based on the intrinsic (uncorrected) CRT display function.

A model of human retinal cones was studied as a function of the adapting luminance to predict the intrascene luminance dynamic range (LDR). It has shown that the human retinal cones do not have a unique perceptual characteristic because the adapting luminance is dependent on the data visualization (tone scale) and ambient lighting. Using the minimum retinal response and maximum luminance of the display, a relationship is derived that specifies the intrascene LDR as a function of the adapting luminance. It is concluded that an intrascene LDR of about 100 is acceptable for primary interpretation (viewing to generate the radiology report) provided the adapting luminance is less than half of the display maximum luminance. However, due to excessively high ambient lighting and lower maximum luminance of displays typically used for secondary interpretation (viewing after radiology report is available), an intrascene LDR of about 50 is recommended for this setting. As the retinal cones provide high spatial frequency response, the minimum display luminance should be greater than 1 cd/m² to ensure fully operational retinal cones. Finally, it is noted that fixing LDR and minimum luminance provides an opportunity to present images with true consistency for image distribution throughout an enterprise.

An interactive, easy-to-use computer program has been developed to assess the quality of softcopy display by measuring the contrast sensitivity, spatial resolution and spatial uniformity at different backgrounds and object types. The program features random variation of the test object location, which minimizes the guessing error often associated with psychophysical measurements. It operates on a Microsoft Window/NT platform and is intended for routine quality assurance (QA) as well as for acceptance testing of PACS. The QA data obtained with this program can be plotted chronologically and centrally managed so as to detect trends in monitor deterioration. The principal motivation for developing this program was to provide an indirect yet sensitive and accurate measure of monitor characteristics with a minimum of specialized equipment.

This paper provides results of a statistical analysis of two methods for arranging the temporal sequencing of the unaided vs computer-assisted reading in multiple-reader, multiple- case (MRMC) receiver operating characteristic (ROC) studies of computer-aided detection of solitary pulmonary nodules (SPNs) on chest radiographs. The modes are the Independent mode, in which the readings are separated by a time on the order of one month, and the Sequential mode, in which the CAD-assisted reading immediately follows the unassisted reading. The method of Beiden, Wagner, Campbell (BWC) was used to decompose the variance of the ROC area summary accuracy measure into the components that are correlated across unaided and aided reading conditions and the components that are uncorrelated across these reading conditions. The latter are the only components of variability that contribute to the uncertainty in a measurement of the difference in reader performance between reading conditions. These uncorrelated components were dramatically reduced in the Sequential reading mode compared to the Independent reading mode-while the total reader variability remained almost constant. The results were remarkably similar across two independent studies analyzed. This may have important practical consequences because the Sequential mode is the least demanding on reader logistics and time.

Both student's t-test for paired data and the Dorfman- Berbaum-Metz (DBM) method report a P value in comparing ROC curves of competing diagnostic modalities. We empirically compared the P values from the t-test and the DBM method using data of two observer studies involving the lung-nodule detection (15 readers 240 cases) and breast-lesion classification (10 readers 104 cases). We made 596,637 comparisons based on data drawn from different combinations and subsets of the readers and cases. The average difference in the P values was 0.11 and 0.058 in the lung nodule study (of two separate analyses) and 0.0061 in the breast lesion study. The lung nodule study showed, in the analysis that demonstrated statistical significance with the original full dataset, both P<0.05 or both p>0.05 in 83% of the comparisons. The t-test alone reported P<0.05 in 17%, and the DBM method alone reported P<0.05 in 1% of the comparisons. A second analysis of the part of the lung nodule study that did not show statistical significance with the original full dataset found both P<0.05 or both P>0.05 in 99% of the comparisons. The t-test alone reported P<0.05 in 1%, and the DBM method alone reported P<0.05 in less than 1% of the comparisons. The breast lesion study showed both P<0.05 or both P>0.05 in 91% of the comparisons. The t-test alone reported P<0.05 in 5%, and the DBM method alone reported P<0.05 in 4% of the comparisons. These results indicate that the t-test and the DBM method generally report similar P values, but their conclusions regarding statistical significance often differ and the DBM method should be used because it accounts for both reader and case variances.

It is possible that neglect of location information in lesion detection studies analyzed by the Receiver Operating Characteristic (ROC) method can compromise power. The Alternative Free-response ROC (AFROC) analysis considers the location information but its usage has been discouraged, since it neglects intra-image correlations. This study compared the statistical power of ROC and AFROC methodologies using simulations. A model including intra-image correlations was developed to describe the decision variable sampling and was used to simulate data for ROC and AFROC analysis. Five readers and 200 cases, half of which contained one signal, were simulated for each trial. Two hundred trials were run, equally split between the Null Hypothesis (NH) and the Alternative Hypothesis (AH). The ratings were analyzed by the Dorfman-Berbaum-Metz (DBM) method and the separation of the NH/AH distributions was calculated. It was found that the AFROC method yielded higher power than ROC. The separation of the NH and AH distributions were larger by a factor of 1.6, irrespective of the presence or absence of intra-image correlations. The effect of the incorrect localizations occurring in ROC analysis of localization data is believed to be the major reason for the enhanced power of the AFROC method, and greater use of AFROC methodology is warranted.

The assessment of image quality in mammography often relies on the subjective evaluation of films produced with test-objects containing structures such as masses, micro-calcifications and filaments. If the methodology is adequate to control the stability of a mammography unit, its use in a context of system optimization (from the x-ray spectrum to the detector response) is limited. Thus, a test-object which allows measurements of the detectability index d' from the non-prewhitening matched filter (NPWE) observer, was developed and tested. The test-object is 45 mm thick and allows the assessment of d' in areas of different glandular/fat compositions (i.e image quality evaluation taking into account the dynamic range parameter). To simulate the absorption of the skin, a 100% fat equivalent tissue, with a thickness of 5 mm, is placed on each side of the test-object. On a conventional unit, it is possible to assess the image parameters at three optical density levels (i.e. 0.5 - 0.8 ; 1.5 - 1.6 and 2.3 - 2.6) in one exposure. The imaging of this test object on digital units has also been tested satisfactorily.

The rapid development of digital imaging and computers networks, using Picture Archiving and Communication Systems (PACS) and DICOM compatible devices increase requirements to the quality control process in medical imaging departments, but provide new opportunities for evaluation of image quality. New StatPhantom software simplifies statistical techniques based on modern detection theory and ROC analysis improving the accuracy and reliability of known methods and allowing to implement statistical analysis with phantoms of any design. In contrast to manual statistical methods, all calculation, analysis of results, and test elements positions changes in the image of phantom are implemented by computer. This paper describes the user interface and functionality of StatPhantom software, its opportunities and advantages in the assessment of various imaging modalities, and the diagnostic preference of an observer. The results obtained by the conventional ROC analysis, manual, and computerized statistical methods are analyzed. Different designs of phantoms are considered.

The Sarnoff JNDmetrix visual discrimination model (VDM) was applied to predict human psychophysical performance in the detection of simulated mammographic lesions. Contrast thresholds for the detection of synthetic Gaussian masses on mean backgrounds and simulated mammographic backgrounds were measured in two-alternative, forced-choice (2AFC) trials. Experimental thresholds for 2-D Gaussian signal detection decreased with increasing signal size on mean backgrounds and on 1/f³ filtered noise images presented with identical (paired) backgrounds. For 2AFC presentations of different (unpaired) filtered noise backgrounds, detection thresholds increased with increasing signal diameter, consistent with a decreasing signal-to-noise ratio. Thresholds for mean and paired filtered noise backgrounds were used to calibrate a new low-pass, spatial-frequency channel in the VDM. The calibrated VDM was able to predict accurate detection thresholds for Gaussian signals on mean and paired 1/f³ filtered noise backgrounds. To simulate noise-limited detection thresholds for unpaired backgrounds, an approach is outlined for the development of a VDM-based model observer based on statistical decision theory.

The Sarnoff JNDmetrix visual discrimination model (VDM) was applied to predict the visibility of compression artifacts in mammographic images. Sections of digitized mammograms were subjected to irreversible (lossy) JPEG and JPEG 2000 compression. The detectability of compressed images was measured experimentally and compared with VDM metrics and PSNR for the same images. Artifacts produced by JPEG 2000 compression were generally easier for observers to detect than those produced by JPEG encoding at the same compression ratio. Detection thresholds occurred at JPEG 2000 compression ratios from 6:1 to 10:1, significantly higher than the average 2:1 ratio obtained for reversible (lossless) compression. VDM predictions of artifact visibility were highly correlated with observer performance for both encoding techniques. Performance was less correlated with encoder bit rate and PSNR, which was a relatively poor predictor of threshold bit rate across images. Our results indicate that the VDM can be used to predict the visibility of compression artifacts and guide the selection of encoder bit rate for individual images to maintain artifact visibility below a specified threshold.

The purpose of this study was to evaluate a method of creating synthetic normal and abnormal mammograms. Images consisting of 1024 x 1024 regions were extracted from digitized mammograms. Twenty-five regions included a single microcalcification cluster. A second set of twenty-five regions without calcifications was also selected. Calcifications were digitally removed by application of a median filter to form a third set of images. Finally, extracted calcifications were superposed on normal images to create a fourth set. Three mammographers evaluated the quality of the simulations. Their task was to classify the images according to real or simulated status using a 10-point rating scale. The classification accuracy was calculated by Receiver Operating Characteristic (ROC) analysis. Two other radiologists performed a paired image task on a subset of the images. They attempted to discriminate between real and simulated images that were simultaneously displayed, which was analyzed by a forced-choice method. In either case it was found that the probability of correct classification was insignificantly different from the chance level. We conclude that the simulation methodology employed was satisfactory. The ability to create synthetic images, that are indistinguishable from real images, is expected to facilitate modality evaluation studies in mammography.

A fully optimized computer-aided diagnosis (CAD) system using system-oriented optimization algorithm has been developed for mass detection in digital mammography for the improvement of CAD performance on sensitivity and specificity. Based on a series of developed adaptive modules, simulated annealing (SA) algorithm is employed on CAD system optimization that is a typical combinatorial mixed-discrete optimization problem. The CAD system is optimized on a training database and evaluated through a test database, the cases in both databases are biopsy proven and selected by experienced radiologists. Both optimized and corresponding un-optimized CAD system have been evaluated using the same test database to compare the system performance. Obvious performance improvement has been obtained on optimized system. The results express the effectiveness of the method developed in this paper.

Many objects in our visual field compete for neural representation. Both bottom-up, sensory-driven processes (luminance detection) as well as top-down mechanisms (attention and familiarity) can affect the result of this competition. In this study, visual evoked potentials were used to measure the changes induced by both stimulus variables and attention processes. The stimulus set consisted of a grayscale sine wave grating pattern with different degrees of spatially random noise. This stimulus set was generated using the ALOPEX optimization algorithm. This algorithm generated a series of sequential images while converging from a completely random noise pattern to the sine wave grating pattern template. All of the patterns in the stimulus set were normalized for average luminance during the ALOPEX convergence process. Additionally, the stimulus content of each pattern was quantified using a number of image processing algorithms including space-averaged global contrast, image entropy, central moments, 2D Fourier transform, and 2D wavelet transform. The visual evoked potentials were recorded using the same pattern set for different attention states of the subjects. The results presented demonstrate the contrasting affects of noise and attention on both the time and frequency components of the visual evoked potential recorded from different lobes of the brain.

Current methods of radiological displays provide only grayscale images of mammograms. The limitation of the image space to grayscale provides only luminance differences and textures as cues for object recognition within the image. However, color can be an important and significant cue in the detection of shapes and objects. Increasing detection ability allows the radiologist to interpret the images in more detail, improving object recognition and diagnostic accuracy. Color detection experiments using our stimulus system, have demonstrated that an observer can only detect an average of 140 levels of grayscale. An optimally colorized image can allow a user to distinguish 250 - 1000 different levels, hence increasing potential image feature detection by 2-7 times. By implementing a colorization map, which follows the luminance map of the original grayscale images, the luminance profile is preserved and color is isolated as the enhancement mechanism. The effect of this enhancement mechanism on the shape, frequency composition and statistical characteristics of the Visual Evoked Potential (VEP) are analyzed and presented. Thus, the effectiveness of the image colorization is measured quantitatively using the Visual Evoked Potential (VEP).

A test bank of verified chest radiographs was compiled for visual search experiments. The purpose was to investigate human performance in the detection of significant pulmonary nodules. Synthesized nodules supplemented native lesions. The distribution of all the lesions in the lung fields was consistent with the naturally occurring locations of these features. A measure of the physical characteristics of the lesions was derived in order to approximate the conspicuity of the synthetic to the natural nodules. The measure of conspicuity was given as (chi) =Tan(theta-1)S/N where (theta) is the maximum slope angle to the edge of the lesion profile, S is the mean pixel value of the lesion profile taken in four orientations, and N is the mean background pixel value taken in four orientations over one lesion dimension adjacent to the lesion. The variation in (chi) for each of the 81 lesions (46 natural and 35 synthetic) was plotted against SNR and edge angle. The influence of edge angle on the resulting (chi) values was more powerful than SNR for all the lesions in this experiment. Although there was an overall significant difference in (chi) values (p=0.015), observers were unable to distinguish synthetic from native lesions. Observer performance in nodule detection was measured by AFROC and supplemented with visual search recording. Correlation of AFROC scores and the (chi) values has shown no overall relationship (R²=0.0452) and this surprising result may be partly explained through inspection of the visual search recordings.

In this investigation we studied the imaging characteristics of a mammographic screen-film (MinR-2000, Eastman Kodak Co.) and an amorphous-silicon flat-panel digital mammography system (Senographe 2000D, GE Medical Systems) based on information perception by human observers. The focus of the study was to utilize an effective means to estimate the contrast-detail characteristics of x-ray imaging systems at various threshold levels to evaluate system performance with reduced observer subjectivity. We obtained three images of a contrast-detail phantom (CDMAM, Nuclear Associates) with screen-film and three images with digital mammography under identical exposure conditions. The digital images were printed using dry film printer (DryView 8600, Eastman Kodak Co.) after being windowed/leveled appropriately by two experienced radiologists. Seven observers reviewed the images and 'proportion correct' detection data were computed for each observer. A psychophysical signal detection model that hypothesizes a continuous decision variable internal to the observer with Gaussian probability density functions was used to fit the experimental observer data. Projection data from the detection curves at 50%, 62.5%, and 75% threshold levels were used to generate contrast-detail diagrams. Digital mammography, on average, exhibited lower (better) threshold contrast-detail characteristics compared to screen-film mammography.

We have performed a retrospective evaluation to assess the effects of temporal subtraction in chest screening. Nineteen abnormal and 30 normal examples of chest CR screening images were selected, and a data set comprising current, previous, and temporal subtraction (T-sub) images were generated. Abnormal examples were chosen from cases with abnormal findings that were confirmed by subsequent CT scanning to need close examination. In the evaluation experiments, only current and previous chest images were displayed at first. After a radiograph observer judged the existence of any abnormal findings by the 5-step rating method, the same observer judged the case again using the T-sub image. Physicians of three different categories, namely diagnostic radiologists with clinical experience of two years or more, less experienced radiologists, and non-radiologists participated in the experiments. As a result of ROC analysis, it was confirmed that the use of temporal subtraction improves the Az value by about 10% overall. The Az value increase was more significant in the group of less experienced doctors. The ROC curves using T-sub image of these physicians approached those of the well-experienced radiologists. These results strongly indicate the clinical efficacy of the assessed system in chest screening.

The goal of the project was to develop an efficient method of optimizing CRT performance for digital mammography. The paradigm measures radiologist performance for various display characteristics and uses these results to validate a model of human visual performance. The Sarnoff JNDmetrix vision model is based on psychophysical just-noticeable difference measurement and frequency-channel vision-modeling principles. Given 2 images as input the model returns accurate, robust estimates of their discriminability. Model predictions are compared with human performance. Mammographic images with microcalcifications were viewed by radiologists. Results were analyzed using ROC techniques. The images were viewed once on a monitor with P45 and once on a monitor with P104 phosphor. Results were compared with output of the model that was used to predict differences in perceptibility of calcifications using luminance data measured with a high-resolution CCD camera. Early results suggest that human performance is best with high contrast clusters and progressively gets worse with each decrease in contrast. Performance so far is better with the P45 than the P104 for targets at all contrast levels. The JNDmetrix model should predict the same pattern of results. The type of phosphor in a CRT monitor seems to influence observer performance.

Purpose: To investigate the relative importance of spatial resolution and noise on the image quality of clinical radiographs. Methods: The spatial resolution and noise of fifteen digitized lumbar spine radiographs were altered with image processing. Three different MTF curves and three different Wiener spectra were combined into seven different combinations of spatial resolution and noise. These seven combinations were applied to the original data set, and the resulting images were printed on film. Seven expert radiologists evaluated the clinical image quality of the resulting images with visual grading analysis (VGA) of structures based on the European Image Criteria. Results: The results show that added noise is more deteriorating than reduced spatial resolution for the clinical image quality. For a given MTF and noise level, the worst was the one with increased noise followed by the one with both reduced MTF and added noise (mimicking a faster screen-film combination). Reduced MTF only gave the highest rating. Conclusions: It is more important to find methods for removing noise than to try to improve the MTF of a radiographic system. A noisy image can sometimes be improved by reducing the spatial resolution.

The purpose is to verify the clinical usefulness of newly developed visual test-objects including chest images and contrast test images for a safe usage of CRT monitor. Various kinds of test images including the contrast-detail method were created. Here, the 11 CRT monitor display conditions that simulated the CRT monitors degraded by the long-term usage were tested. The soft-copy test-images under 11 kinds of monitor-luminance conditions were observed. From the results of the threshold contrast values at which the target was just visible, we determined the border zone of CRT monitor luminance below which the diagnostic performance was inferior to that of the normal CRT monitors. In the darker display conditions in which the maximum luminance was 0.607 or less of the normal CRT luminance (480 cd/m²), the correct detection rates of targets were significantly inferior to that in the normal CRT display condition (480 cd/m²)(p<0.05). In the perception study of the contrast test images luminance where the monitor became apparently too dark for the image interpretation was 0.52 to 0.59 of the normal CRT luminance. The method with proposed visual test-objects is practical for daily check of diagnostic CRT monitors and convenient for diagnostic radiologists.

This paper presents a statistical method to explore and assess variability among diagnosticians in their accuracy and the association between accuracy and characteristics of diagnosticians and patients. The method assumes random sampling from a population of patients. It is assumed the diagnosticians provide ordinal diagnostic ratings to all patients. In stage I, the Binormal Model is used to summarize the data into diagnostician-specific accuracy parameters at each patient covariate level. In stage II, the reduced data is then regressed on characteristics of the diagnosticians. Statistical inference is driven by bootstrapping. An application of the method to a national study of mammogram interpretation variability is presented. Empirical and theoretical evaluations are presented which substantiate the method. It will be shown that the model belongs to the well-known class of General Linear Models. The primary strength of the method is that it facilitates familiar and graphical approaches to the analysis of complex diagnostic ratings data arising from the simultaneous sampling of the population of diagnosticians as well as of the population of patients.