Introduction
============

Due to the fact that the diagnostic 3-megapixel (MP) liquid crystal displays (LCDs) currently used in our radiology department have degraded in quality, they are scheduled to be replaced. The intention is to replace them with either a pair of 3-MP LCDs or a 6-MP single LCD. Also, due to ongoing developments in radiology, color images are becoming more important;[@b1-mder-7-371] therefore, we opted to upgrade to color LCDs.

Rapid developments in the field of LCDs have led to more stable, consistent image quality. This has resulted in a broad choice of medical grade monitors with similar technical specifications, but widely variable prices. For example, in the present study, we found a three-fold difference between the retail prices of the most and least expensive LCDs.

Several studies have compared imaging monitors with different specifications, like cathode ray tubes (CRTs) versus LCDs,[@b2-mder-7-371]--[@b5-mder-7-371] diagnostic versus office grade or clinical review grade monitors,[@b6-mder-7-371],[@b7-mder-7-371] and monochrome versus color monitors.[@b6-mder-7-371],[@b8-mder-7-371]--[@b10-mder-7-371] Also, comparisons have been made between monitors that varied in number of pixels and pixel size.[@b11-mder-7-371]--[@b16-mder-7-371] Often, the monitors compared have differed in more than one variable (eg, monochrome CRT versus color LCD).

A number of studies have found that, under the conditions considered, medical grade LCDs were comparable with CRT monitors and hard-copy radiograph films for diagnostic readings in radiology.[@b2-mder-7-371],[@b3-mder-7-371],[@b11-mder-7-371],[@b12-mder-7-371],[@b14-mder-7-371],[@b17-mder-7-371] However, the use of office-grade monitors for diagnosis has been discouraged, despite some studies that demonstrated acceptable quality.[@b6-mder-7-371],[@b7-mder-7-371] In most studies, no significant differences were found between color and monochrome monitors.[@b8-mder-7-371],[@b9-mder-7-371],[@b10-mder-7-371],[@b18-mder-7-371] Finally, the number of pixels and pixel size should match the size of the image to minimize the time spent on zooming and magnification.[@b1-mder-7-371],[@b15-mder-7-371],[@b16-mder-7-371] Overall, no diagnostic differences have been found between monitors with different specifications; thus, it is not clear whether equally calibrated, diagnostic LCDs made by different manufacturers vary in image quality. To our knowledge, no studies have compared equally calibrated 3-MP or 6-MP color LCDs made by different manufacturers, based on the technical and clinical guidelines of the American Association of Physicists in Medicine (AAPM).[@b19-mder-7-371]

Due to the current global economic situation, we were motivated to perform a study on differently priced LCDs. This study aimed to perform a technical and radiological comparison of 3-MP and 6-MP color LCDs made by three major manufacturers, and establish whether they differed significantly in image quality. This information provided the basis for cost-effective decisions in purchasing new diagnostic LCDs.

Materials and methods
=====================

This prospective study was conducted in June and July, 2011. We investigated four different models of color LCDs made by three different manufacturers. We compared an NEC^®^ MD213MC 3-MP LCD (NEC, München, Germany), an Eizo^®^ RX320 3-MP LCD (Eizo Nanao Corporation, Hakusan, Japan), a Barco^®^ MDNC-3121 3-MP LCD (Barco, Kortrijk, Belgium), and a Barco MDCC-6130 6-MP LCD. The NEC, Eizo, and Barco 3-MP LCDs had a diagonal dimension of 21 inches and a resolution of 1,536×2,048 pixels, with a pixel size of 0.21 mm. The Barco 6-MP LCD had a diagonal dimension of 30 inches and a resolution of 3,280×2,048 pixels with a pixel size of 0.20 mm. Both types of Barco LCDs had an extra protective glass shield. The manufacturers were asked to supply optimally calibrated LCDs. All LCDs were calibrated according to the Digital Imaging and Communications in Medicine grayscale standard display function to obtain optimal black and white images at a brightness of 400 cd/m^2^. In a darkened, film-reading room, the displays were placed next to each other. Ambient light was 0.5 lux for technical measurements and 7.4 lux for radiological assessments. The names of the manufacturers were shielded to minimize bias. The 3-MP LCDs were set up in pairs in portrait mode, as commonly used in radiology. The 6-MP single LCD was assessed as if it were two 3-MP displays.

Technical assessment
--------------------

Technical assessment of LCD performance was performed according to standard guidelines provided by the AAPM.[@b19-mder-7-371] Digital test patterns from the AAPM Task Group 18 (TG-18) were evaluated with a calibrated spot luminance meter (Konica Minolta type LS-100, Osaka, Japan) and a calibrated colorimeter (Philips type PM 5639, Best, The Netherlands). The following parameters were measured: geometric distortion, reflection, luminance response, luminance dependencies, resolution, noise, veiling glare, and chromaticity ([Table 1](#t1-mder-7-371){ref-type="table"}).

Radiological assessment
-----------------------

For radiological assessments, 12 radiologists (7--42 years of experience) and seven residents (3 months to 3 years of experience) independently performed a side-by-side comparison of the four different types of LCDs. The observers were blinded to the brand of the LCD because we shielded the names of the manufacturers. The order of evaluating the different LCDs was randomized. The AAPM TG-18 anatomical test patterns included a chest X-ray image and a knee X-ray image.[@b19-mder-7-371] For a more complete evaluation, we included two AAPM TG-18 mammography images, although these were not commonly examined on 3-MP LCDs. Furthermore, we included anonymized thorax computed tomographic and brain magnetic resonance images from the hospital database. For all images, the evaluators responded to AAPM questions concerning the difficulty in interpretation, overall contrast, and overall sharpness of the images ([Table 2](#t2-mder-7-371){ref-type="table"}). The AAPM TG-18 images also included questions about specific anatomical and structural details, according to the AAPM TG-18 criteria.[@b19-mder-7-371] All images were scored on a visual analog scale (range 0--10).[@b20-mder-7-371] The visual analog scale consisted of a horizontal line, with anchors at 0 and 10. The evaluators marked their scores on the line.

Statistical analysis
--------------------

Cronbach's alpha was used to determine the internal consistency (validation) of the questions used to evaluate the images. LCD quality was compared with a global score of image quality for each LCD; the global score was the average score for all questions on image quality, based on the mean scores (from 19 evaluators) for each question, and each question was weighted identically.

Because the questions might not be equally important, we also performed statistical testing on individual questions to compare individual qualities of the LCDs. Some images were associated with additional questions; therefore, we examined a total of 18 scores. All five images were evaluated for difficulty in interpretation, overall contrast, and overall sharpness (five images × three scores =15 scores). Only three images were evaluated with the detailed anatomical questions. The scores for all the anatomical questions were averaged before performing statistical analyses; this resulted in three anatomical scores (three images × one anatomical score). Thus, we analyzed 18 scores (15+3 individual scores), and determined differences among LCDs with the repeated-measures analysis of variance method. We determined the sphericity of the data with Mauchly's test. When no sphericity was found, the Huynh-Feldt statistic was reported. After applying the Bonferroni adjustment, *P*-values ≤0.0028 (0.05/18) were considered to be statistically significant. All statistical analyses were performed with Statistical Package for the Social Sciences for Windows version 18 (PASW Statistics, Armonk, NY, USA).

Results
=======

Technical assessment
--------------------

[Table 3](#t3-mder-7-371){ref-type="table"} shows the results of the technical evaluations. The NEC and Eizo LCDs passed the AAPM acceptance criteria, except for the darkest pixel value in the grayscale standard display function. The Barco 3-MP LCD passed all AAPM acceptance criteria, but the Barco 6-MP LCD only passed the tests for geometric distortion, reflection, resolution, noise, and chromaticity.

Radiological assessment
-----------------------

For all images evaluated, Cronbach's alpha indicated that the questions used for the assessment had high internal consistency ([Table 4](#t4-mder-7-371){ref-type="table"}).

Overall scores of 8.1±1.0 (NEC), 7.8±1.2 (Eizo), 7.8±1.1 (Barco 3 MP), and 8.1±1.0 (Barco 6 MP) were obtained by averaging the scores of all questions. [Figure 1](#f1-mder-7-371){ref-type="fig"} shows the scores for individual questions for each image evaluated on each type of LCD. The corresponding *P*-values are shown in [Table 5](#t5-mder-7-371){ref-type="table"}. Although the NEC LCD and the Barco 6-MP LCD showed a tendency to higher scores than the other LCDs on most questions, there were no statistically significant differences found between the four different LCDs. Also, scores for the five different test patterns were similar.

On average, the radiologists' scores were higher than the residents' scores. The radiologists reported overall scores of 8.5±0.9 (NEC), 8.2±1.1 (Eizo), 7.9±1.2 (Barco 3 MP), and 8.4±0.9 (Barco 6 MP). The residents reported scores of 7.4±1.0 (NEC), 7.2±1.0 (Eizo), 7.4±0.9 (Barco 3 MP), and 7.5±0.9 (Barco 6 MP).

Discussion
==========

To our knowledge, this study represents the first technical and radiological assessment of medical digital LCDs with similar specifications, made by different manufacturers, based on international criteria. We established some technical differences, but like Samei et al, we could not find any statistically significant differences in radiological assessments made with different LCDs.[@b5-mder-7-371] This was consistent with most reported studies, where no diagnostic differences were found between LCDs with different specifications.[@b5-mder-7-371],[@b6-mder-7-371],[@b8-mder-7-371],[@b9-mder-7-371]--[@b12-mder-7-371],[@b14-mder-7-371]--[@b16-mder-7-371]

Overall, on technical assessments, all the included LCDs scored well on the AAPM acceptance criteria. However, we measured some technical differences. In contrast with the two Barco displays, the NEC and Eizo LCDs did not pass the criterion for the darkest grayscale in the grayscale standard display function. This may be due to the larger variations in measurements at lower grayscales, as reported by Fetterly et al.[@b21-mder-7-371] Nonetheless, this may be cause for concern, because it may lead to suboptimal contrast for dark structures.

The Barco 3-MP LCD passed all the AAPM acceptance criteria, but scored less well on the evaluations of reflections in ambient lighting and veiling glare. This might be due to its glass protective cover. This can be removed optionally by the manufacturers when ordering new LCDs. The American College of Radiology recommends against the use of protective shields, because they may contribute to reflections.[@b1-mder-7-371] On the other hand, the protective cover may prolong the lifespan of the LCD.

The maximum luminance response of the Barco 6-MP LCD deviated substantially from the calibrated value. However, when we evaluated the 6-MP LCD, we considered it equivalent to two 3-MP LCDs; therefore, we performed the measurements in the center of one half of the 6-MP LCD, instead of in the center of the whole LCD. Possibly, the LCD was not optimally calibrated to ensure the uniformity desired for a 2±3-MP set up. This problem only became apparent when we analyzed the technical results. This emphasizes the importance of carefully calibrating luminance uniformity and verifying the calibration.

Radiological image assessments showed no significant differences among LCDs, although there was a tendency for the Barco 6-MP and the NEC 3-MP LCDs to receive slightly higher scores than the other LCDs. Thus, the Barco 6-MP exhibited radiological results that were slightly in contrast with the technical results. Despite the precautions taken to avoid bias by obscuring the names of the manufacturers, the observers might have recognized the Barco 6-MP LCD, because it was the only wide screen LCD currently available. Therefore, we could not fully exclude observer bias.

The results of this study are consistent with a report by Crespi et al, who stated that calibrated medical monitors possessed good quality.[@b17-mder-7-371] To maintain this quality, they strongly recommended periodic quality assurance tests.

There are several limitations to our study. First, we did not measure stability over time. Deterioration of the color filters in color LCDs might lead to rapid decreases in luminance.[@b8-mder-7-371] Kuroki et al suggested that the luminance of LCDs affected observer evaluations.[@b6-mder-7-371] Second, our radiological assessments were based on AAPM TG-18 criteria, but we did not investigate specific diagnostic criteria. Further study is required to rule out possible diagnostic differences. Third, the number of readers who participated in the radiological assessment is limited, which explains the width of the uncertainty around the means. However, it is possible that with a larger sample size the standard deviations would be smaller. Last, in considering the price differences between the LCDs, we did not consider potential expenses associated with endurance, support, and maintenance in the overall cost.

Within the experimental context of the study we wanted to reflect daily practice. This is why both radiologists and residents participated in the radiological assessment. Since radiologists are more experienced than residents in radiology, they could be more certain about image quality in comparison with residents. This might explain the small differences between the average scores reported by the radiologists and the average scores reported by the residents. However, the standard deviations are completely overlying. Several studies have found that the technical specifications and performance can vary among different monitor types (LCD, CRT) and also among hard-copy radiograph films; nevertheless, very few differences have been reported in their accuracy for clinical evaluation.[@b2-mder-7-371],[@b3-mder-7-371],[@b6-mder-7-371],[@b8-mder-7-371],[@b11-mder-7-371]--[@b14-mder-7-371] In the present study, although slight technical and radiological differences were noted, the evaluated medical grade diagnostic 3-MP and 6-MP LCDs showed comparable image quality when calibrated accurately. The lack of significant differences in overall scores in this study may have been due to the apparently high (medical grade) quality of all the tested LCDs. Care should be taken to calibrate the LCDs to suit the task at hand and to verify the calibration with technical testing.[@b1-mder-7-371],[@b22-mder-7-371]

Conclusion
==========

We showed that, despite a large price difference, there were no significant differences in image quality between the investigated LCDs. In deciding which LCD to purchase, in addition to the price, maintenance costs and stability should be taken into consideration.

**Disclosure**

The authors report no conflicts of interest in this work.

![Radiological assessments for each type of LCD, according to AAPM TG-18 criteria. Symbols indicate different LCD types; scores are represented as the mean and standard deviation on a scale from 0--10 (n=19 evaluators).\
**Abbreviations:** AAPM, American Association of Physicists in Medicine; TG-18, Task Group 18; LCD, liquid crystal display; MP, megapixels; sd, standard deviation; CT, computed tomography; MR, magnetic resonance.](mder-7-371Fig1){#f1-mder-7-371}

###### 

Short descriptions of technical parameters assessed for each liquid crystal display

  Parameter                       Section in reference[@b19-mder-7-371]   Test pattern TG-18   Short description
  ------------------------------- --------------------------------------- -------------------- ----------------------------------------------------------------------------------------------------------------
  Geometric distortion            III.A                                   QC, LPH, LPV         Visual evaluation of whether lines appear straight and parallel and whether squares appear square
  Reflection                      III.B                                   AD                   Visual evaluation of low-luminance, low-contrast patterns in near total darkness and in ambient lighting
  Luminance response              III.C                                   LN12                 Quantitative measurement of luminance at 18 grayscales to determine conformity with grayscale display function
  Luminance: nonuniformity        III.D                                   UN-L10, UN-L80       Quantitative measurement of luminance at four corners and in the center of the display
  Luminance: angular dependence   III.D                                   LN12-01, LN12-18     Quantitative determination of view angle at which 70% luminance of frontal position is maintained
  Resolution                      III.E                                   QC                   Visual score of CX patterns in the center and four corners of the display
  Noise                           III.F                                   AFC                  Visual evaluation of number of visible low-contrast squares in the center and all corners of the display
  Veiling glare                   III.G                                   GV, GVN              Visual evaluation of number of visible globes to determine light spread within the display
  GQ                              Quantitative measurement of glare                            
  Chromaticity                    III.H                                   UNL80                Quantitative measurement of color of pattern in the center and four corners

**Note:** Section numbers and test patterns are as designated by the American Association of Physicists in Medicine in an Executive Summary report.[@b19-mder-7-371] Copyright © 2005. American Association of Physicists in Medicine. Adapted from Samei E, Badano A, Chakraborty D, et al., Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report. *Med Phys*. 2005;32(4):1205--1225.[@b19-mder-7-371]

**Abbreviation:** TG-18, Task Group 18.

###### 

American Association of Physicists in Medicine Task Group-18 criteria used for radiological assessments

  Test pattern                                                                                                                       Evaluation criteria
  ---------------------------------------------------------------------------------------------------------------------------------- ------------------------------
  TG-18-CH (chest X-ray)                                                                                                             Difficulty of interpretation
  Overall contrast                                                                                                                   
  Overall sharpness                                                                                                                  
  Symmetrical reproduction of the thorax, as shown by the central position of the process between the medial ends of the clavicles   
  Medial border of the scapulae                                                                                                      
  Reproduction of the whole rib cage above the diaphragm                                                                             
  Visually sharp reproduction of the vascular pattern in the lungs                                                                   
  Sharp reproduction of the trachea and proximal bronchi                                                                             
  Sharp reproduction of the borders of the heart and the aorta                                                                       
  Sharp reproduction of the diaphragm                                                                                                
  Visibility of the retrocardial lung and the mediastinum                                                                            
  Visibility of the subdiaphragmatic features                                                                                        
  Visibility of the spine through the heart shadow                                                                                   
  Visibility of small details in the whole lung, including retrocardial areas                                                        
  Visibility of linear and reticular details of the lung periphery                                                                   
  TG-18-KN (knee X-ray)                                                                                                              Difficulty of interpretation
  Overall contrast                                                                                                                   
  Overall sharpness                                                                                                                  
  Reproduction of trabecular detail                                                                                                  
  Reproduction of bony and soft tissues                                                                                              
  TG-18-MM1 and TG-18-MM2 (mammograms)                                                                                               Difficulty of interpretation
  Overall contrast and brightness                                                                                                    
  Overall sharpness (no blur)                                                                                                        
  Sharp appearance of cooper's ligaments                                                                                             
  Clip structure and the presence of the apical gap (TG18-MM1 only)                                                                  
  Appearance and visibility of subtle microcalcifications (TG18-MM1 only)                                                            
  Visibility of structures at the margins of the breast (TG18-MM1 only)                                                              
  Chest CT                                                                                                                           Difficulty of interpretation
  Overall contrast                                                                                                                   
  Overall sharpness                                                                                                                  
  Brain MR                                                                                                                           Difficulty of interpretation
  Overall contrast                                                                                                                   
  Overall sharpness                                                                                                                  

**Notes:** Each question was scored on a visual analog scale of 0--10 for each liquid crystal display. Criteria are as designated by the American Association of Physicists in Medicine in an Executive Summary report.[@b19-mder-7-371] Copyright © 2005. American Association of Physicists in Medicine. Adapted from Samei E, Badano A, Chakraborty D, et al., Assessment of display performance for medical imaging systems: executive summary of AAPM TG18 report. *Med Phys*. 2005;32(4):1205--1225.[@b19-mder-7-371]

**Abbreviations:** CT, computed tomography; MR, magnetic resonance; TG-18, Task Group 18.

###### 

Technical assessments of the different LCD models

                                                                              Description                                                                   Criterion   NEC^®^    Eizo^®^   Barco^®^ 3-MP   Barco^®^ 6-MP
  --------------------------------------------------------------------------- ----------------------------------------------------------------------------- ----------- --------- --------- --------------- ---------------
  Geometric distortion                                                                                                                                      Absent      Absent    Absent    Absent          
  Reflection                                                                  Number of visible patterns in total darkness                                              40        34        34              34
  Number of visible patterns in ambient lighting                                                                                                            39          27        6         34              
  Luminance response                                                          Maximum, cd/m^2^                                                              ≥170        410.9     394.0     399.7           328.1
  Deviation from 400 cd/m^2^ (calibrated maximum luminance), %                ≤5                                                                            2.7         1.5       0.1       18.0            
  Deviation from GSDF over 18 grayscales, % (mean±SD)                                                                                                       3.7±2.8     2.4±2.5   2.1±1.9   3.2±2.2         
  Maximum deviation from GSDF, %[\*](#tfn5-mder-7-371){ref-type="table-fn"}   ≤10                                                                           11.9        11.1      7.3       8.3             
  Luminance: nonuniformity                                                    Nonuniformity for UNL10 (low luminance pattern), %                            ≤30         0.4       0.4       0.2             0.2
  Nonuniformity for UNL80 (high luminance pattern), %                         ≤30                                                                           8.9         8.4       3.3       10.5            
  Luminance: angle dependencies                                               View angle for LN12-01 (low luminance pattern), degrees                                   35        35        40              55
  View angle for LN12-18 (high luminance pattern), degrees                                                                                                  25          25        25        45              
  Resolution                                                                  Similarity between CX patterns[\*\*](#tfn6-mder-7-371){ref-type="table-fn"}   ≤4          1         1         1               1
  Noise                                                                       Number of visible squares                                                     ≥15         15        16        16              16
  Veiling glare                                                               Number of visible globes                                                      ≥3          5         5         4               4
  Glare ratio                                                                 \>400                                                                         1,711       1,406     485       364             
  Chromaticity                                                                Difference in color coordinates                                               ≤0.01       0.0017    0.0042    0.0023          0.0024

**Notes:**

For all four displays, this was measured at the lowest gray level (LN12-01 pattern);

lower scores indicate better resolution.

**Abbreviations:** GSDF, grayscale standard display function; LCD, liquid crystal display; MP, megapixels; SD, standard deviation.

###### 

Reliability of image assessments for predicting liquid crystal display quality, determined with Cronbach's alpha

                Cronbach's alpha[\*](#tfn8-mder-7-371){ref-type="table-fn"}
  ------------- -------------------------------------------------------------
  Chest X-ray   0.990
  Knee X-ray    0.989
  Mammogram     0.963
  Chest CT      0.974
  Brain MR      0.975

**Note:**

Alpha values \>0.70 indicate acceptable reliability.

**Abbreviations:** CT, computed tomography; MR, magnetic resonance.

###### 

*P*-values for comparisons of radiological assessments made with different liquid crystal displays

                Degree of difficulty                                  Overall contrast                                      Overall sharpness                                     Anatomy
  ------------- ----------------------------------------------------- ----------------------------------------------------- ----------------------------------------------------- ---------------------------------------------------
  Chest X-ray   0.109[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.236[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.008[\*\*](#tfn12-mder-7-371){ref-type="table-fn"}   0.067[\*](#tfn11-mder-7-371){ref-type="table-fn"}
  Knee X-ray    0.346[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.140[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.088[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.095[\*](#tfn11-mder-7-371){ref-type="table-fn"}
  Mammography   0.023[\*\*](#tfn12-mder-7-371){ref-type="table-fn"}   0.031[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.059[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.072[\*](#tfn11-mder-7-371){ref-type="table-fn"}
  Chest CT      0.416[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.637[\*\*](#tfn12-mder-7-371){ref-type="table-fn"}   0.089[\*](#tfn11-mder-7-371){ref-type="table-fn"}     
  Brain MR      0.153[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.166[\*](#tfn11-mder-7-371){ref-type="table-fn"}     0.024[\*\*](#tfn12-mder-7-371){ref-type="table-fn"}   

**Notes:** After Bonferroni adjustments for multiple comparisons, *P*≤0.0028 was considered statistically significant.

*P*-value according to the Huynh-Feldt statistic

*P*-value according to Mauchly's test (spherical data).

**Abbreviations:** CT, computed tomography; MR, magnetic resonance.
