currently , radiological diagnosis is based on subjective judgment by radiologists , in which image - based findings and text - based clinical information are integrated to reach a decision . this ability is acquired by education and personal experience , and the exact reasoning leading to the final conclusion is difficult to document and share . this is a major obstacle to adopting evidence - based medicine in radiology , which calls for a systematic integration of past evidence for medical decision making ( hsu , 2012 ) . presently , radiological data are stored in pacs ( picture archive and communication system ) . once a diagnosis is completed , this rich data source is rarely used to enrich future daily diagnostic capability . the heart of the problem is that the data in pacs are not fully structurized , which makes information retrieval impractical . the pacs contains two types of information : text and images . the text is usually composed of a few structured and searchable fields , such as name , gender , and age , as well as free - text fields with image descriptions and diagnosis . there have been many ongoing attempts to structurize the pacs free - text fields ( radiological society of north america , 2014a , b ; hussein , 2004 ) . if successful , physicians would be able to search , for example , for  past cases with a diagnosis of alzheimer 's disease ( ad ) ,  and , in return , say , they would retrieve 100 images of ad patients . however , the retrieved 100 images may not be directly useful in supporting routine clinical decisions . to unlock the rich information stored in pacs , we also need to structurize the image data to support query , search , and relational analysis of anatomical information in the images . for example , physicians would submit an image of a new patient and search 100 images with a similar anatomical phenotype . the clinical reports related to these images would be retrieved and a statistical report of the diagnosis and prognosis could be generated . the technology for direct image searching , called content - based image retrieval ( cbir ) , is similar to that involved in face recognition ( for review , see mller , 2004 ) , and is highly anticipated advancement in medical imaging for ct and mri ( greenspan and pinhas , 2007 ; rahman et al . , 2007 ; robinson , 1996 ; orphanoudakis et al . , 1996 ; el - kwae et al . , 2000 ; while cbir is certainly promising , to date , very few image retrieval systems have been used for education or for routine clinical practice . some difficulties in applying these established cbir systems to the human brain are the complexity of the structures and the importance of location information . we report our initial attempt to extend cbir to three - dimensional feature extraction and structuration of the brain anatomy . the most obvious approach would be to align all the images in a database into a common template space . with a resolution of 1  mm , each image would be converted into a vector with about 1  million - voxel elements and vector - to - vector correlation analysis could be performed to find past similar cases . alternatively , subtraction between a pair of images could be used to evaluate the similarity . first , the number of observation ( voxels ) is too large and the information is too noisy . , we assume that the conversion of the 1  million - voxel image representation needs to be contracted to a far smaller number of anatomical representation for subsequent information storage , searching , and correlation analyses . the recent advances in the structural parcellation technique of the entire brain ( shen and davatzikos , 2002 ; shi , 2010 ; wang , 2013 ) now provides tools to structure raw mr images into a standardized and quantitative indices . in this paper , we test the efficacy of the parcellation - based image structuration approach using t1-weighted images of a population with primary progressive aphasia ( ppa ) and age - matched controls . ppa is an ideal model because , while brain atrophy is a common feature , its extension and location vary . in addition , the atrophy is visible and the results can be compared with subjective human judgment . we tested whether the structured anatomical data actually captured the anatomical features that can be perceived by trained clinicians . using principal component analysis ( pca ) and partial least squares - discriminant analysis ( pls - da ) , we characterized the anatomical variability in ppa , tested our image search engine , and performed individual evaluations and integrative analyses of anatomical ( image ) and clinical ( diagnosis ) phenotypes . we enrolled 30 participants with ppa who were seen in one author 's ( ah 's ) outpatient clinic . table 1 summarizes demographic and clinical information ; the appendix table contains the complete clinical information . these patients gave written informed consent and this study was approved by the local irb . they were diagnosed with ppa on the basis of having a predominant and progressive deterioration in language in the absence of a major change in personality , behavior , or cognition ( other than praxis ) for at least 2  years . they were classified as logopenic variant ( lv ; n  =  19 , 4 male , age : 67.4    6.4 ) or semantic variant ( sv ; 7 male , age : 65    6.5 ) , according to recent guidelines ( gorno - tempini , 2011 ) . axial mprage t1    wis ( tr / te  =  8.4/3.9  ms ) were acquired using a 3  t mri scanner , with a 256    256 matrix . sixteen participants were scanned with a field of view ( fov ) of 230    230  mm and 120 slices of 1  mm thickness ; fourteen were scanned with a fov of 212    212  mm and 140 slices of 1.1  mm thickness . healthy controls ( n  =  24 ) with same age , gender , and image protocol distribution were obtained from our normal database . to measure the volume of each anatomical region , we performed an atlas - based analysis ( aba ) , in which an atlas image ( mori , 2008 ) with a pre - defined anatomical parcellation was warped to each participant 's brain , thereby automatically parcellating each brain ( fig .  1 ) . the process was performed by diffeomap and roieditor ( laboratory of brain anatomical mri and center for imaging science at johns hopkins university , 2014 ) and was made possible due to the high accuracy of the mapping algorithm , the large deformation diffeomorphic metric mapping , lddmm ( miller , 2005 ; wang , 2007 ) . the dual - contrast lddmm ( ceritoglu , 2009 ) was based on t1-wis and cerebrospinal fluid ( csf ) maps ( djamanakova , 2013 ) . the template was the jhu - mni  eve ,  a single - subject atlas in the icbm-152 space , extensively parcelled and labeled to 211 regions ( oishi , 2009 ) . the 52  coarse  peripheral regions ( oishi , 2008 ) were subsegmented in each native individual space to separate the cortex , the white matter beneath the cortex , and the cerebrospinal fluid ( csf ) , using tissue maps from spm8 , which resulted in the final 211 rois for each participant . this parcellation  segmentation combined approach circumvents the mis - registration problems in the cortical areas , although mis - registration may exist between the parcels ( e.g. , in the boundary between adjacent gyri or between peripheral and deep white matter parcels ) . in order to compare different degrees of granularity , we combined lobar parcels and deep white matter areas , and created a  low granularity  parcellation that contains 14 regions . both the high and low granularity parcellations were tested against the visual analysis by 3 experienced clinicians ( two neuroradiologists  avf and sy  and one neurologist  ko ) . for the voxel - based analysis ( vba , fig . 1 ) in the template space , we used the jacobian determinant ( i.e. , the local expansion factor ) of the lddmm deformation fields as a quantitative metric of local volume changes ( chung , 2001 ; riddle , 2004 ; thompson , 2000 ) . for the aba , we used the volume of each parcel in the native space , normalized by the intracranial volume . for each ppa patient , regional z - scores were calculated using the average and standard deviations of the controls . the principal component analysis ( pca ) was used to explore the anatomical characterization . for image classification ( lv vs. sv ) , and to exemplify how the image search engine works , we used pls - da . to guarantee classifier stability , we trained the model using participants scanned with one protocol ( n  =  14 ) and tested it with participants scanned with another protocol ( n  =  16 ) . these groups had no other significant clinical or demographic differences other than the image protocol . we report the variables ( i.e. , the anatomic regions ) with the highest weight to explain the population variance ( in the pca case ) and characterize and segregate groups ( in the pls - da case ) . to determine to what extent the visual analysis and the aba agreed , we asked the neurologist / neuroradiologists to visit each parcel automatically defined in the native space of all the subjects and to classify the atrophy in degrees of severity ( 0  =  no atrophy , 1  =  mild , 2  =  moderate , 3  =  severe atrophy ) . the average scores of visual analysis in each region were correlated with the z - scores of volume obtained from the automated aba , using a non - parametric spearman test . in the same way , we calculated the inter - rater agreement . we also  dichotomized  the data ( visual scores : 0 and 1 in class 1 , and scores 2 and 3 in class 2 ; aba z - scores : 2 were class 1 and > 2 were class 2 ) and performed an accuracy analysis to measure the sensitivity and specificity of the aba compared to the subjective visual impression . fig . 2 is a volumetric comparison of controls and ppa participants with vba and aba , by t - tests at a p - value  <  0.05 , corrected for multiple comparisons using the false discovery rate ( fdr ) . both methods detected atrophy in the white matter and the deep gray matter , as well as enlargement of the ventricles . first , the aba examines the volumes structure - by - structure , while the vba examines each voxel independently and , therefore , the report does not follow anatomical boundaries . second , vba results are largely silent in the cortical areas , probably due to mis - registration of voxels in the csf , the cortex , and the peripheral white matter across patients . isotropic filters do not ameliorate this fact . this is less of an issue for aba because these three compartments are measured and averaged in the native space . fig . 2 reports the results of a population - based analysis , in which data within a group are congregated and atrophic locations that are common among the patients are characterized . this analysis is , however , not compatible with clinical diagnosis , which requires information for each patient . a simple approach to report quantitative individual features is to use a z - score map of each patient ( fig . 3 ) . the individual analysis based on vba is , again , mostly silent in peripheral areas . fig . aba converts the native t1-wis images into a 211- ( high granularity ) or 14- ( low granularity ) element z - score vector , which is visualized by a bar - code - type graph . the agreement of the subjective analysis ( inter - evaluators ) was higher using the low - granularity parcellation ( r  =  0.81 ) than with the high - granularity ( r  =  0.54 ) . the agreement between the automated z - score and the averaged visual scores ( r  =  0.8 ) was virtually the same as the inter - evaluator agreement , showing that the automated method s performance can be considered as good as that of one of the raters . the agreement between visual and quantitative analysis improved at moderate and severe degrees of atrophy ( r  =  0.68 for visual scores > 1 , against r  =  0.22 for visual scores < 1 , at high granularity ) . the sensitivity , specificity , and predictive values of evaluators and quantitative analysis were comparable ( table 2 ) . 4 shows the tridimensional pca plot , where the principal components ( pcs ) were linear combinations of the volume of the parcels . the definition of the three groups ( controls and two ppa variants ) was based on clinical information , not on anatomical information . for example , a patient clinically diagnosed as sv ( indicated as  patient nearby controls  ) is  anatomically  close to the controls on the pca plot . this is understandable if we examine the mri , which seems to demonstrate a normal appearance . similarly , controls  anatomically  located near the ppa patients ( such as  control nearby patients  ) had enlarged ventricles and parenchyma atrophy . the images of six subjects numbered in fig . 4 and shown in fig . 5 were chosen based on their locations in the pca space . two neighboring cases in the pca space ( # 1 and 2 , # 3 and 4 , or # 5 and 6 ) had visually similar anatomical features , while the cases distant from each other ( # 1 , 3 , and 5 or # 2 , 4 , and 6 ) had markedly different anatomical features , indicating that the structured anatomical vectors captured the gross anatomical features . the highest loading weights in the first component were widespread in tissue ( negative ) , particularly at the left , and at the ventricles ( positive ) , characterizing global tissue atrophy and ventricle enlargement . the other components were marked by high ( absolute ) weights at the deep gray matter and the core white matter ( second ) , and at the frontal , parietal , and temporal lobes , particularly at right , in the latter ( third ) . this reflected the common phenotype of ppa , as well as particular characteristics of each variant , such as the predominance of atrophy in the frontal and parietal areas in lv , and at the left temporal in sv ( wilson , 2009 ) . differences in protocol contributed to 5% of the data variance , suggesting that the anatomical effect size in this population was large enough to be delineated beyond the protocol . the pls - da resulted in a model with an accuracy of 0.875 in distinguishing lv from sv ( fig . 7 ) . in the first component , which had the greatest power to segregate the groups , the loading ( absolute ) weights were higher in the frontal and parietal areas in lv , and in the left temporal areas in sv , which agreed with the pattern of atrophy the second component , much less powerful in segregating the groups , revealed high loading weights in the frontal areas , the deep gray and white matter , in some temporal , parietal , and occipital left areas , and in the frontal horn of left ventricle , which is a common pattern of atrophy in this population . we demonstrated the concept of converting images to a standardized vector to be used for the quantitative analysis of atrophy at population and individual levels . the accuracy of this conversion hinges on the accuracy of the image transformation and subsequent atlas warping . our past publications reported a high level of accuracy using the lddmm algorithm for populations with marked atrophy ( oishi , 2009 ; djamanakova , 2013 ) , although it depends on the structure in question . in addition , our healthy young adult atlas may not be the best template to achieve high transformation accuracy . more sophisticated approaches , such as multi - atlases ( aljabar , 2009 ; heckemann , 2006 ; iosifescu , 1997 ; jia et al . , 2012 ; klein and hirsch , 2005 ; lao , 2004 ; liu et al . , 2004 ; rohlfing et al . , 2004 ; wang , 2010 ; warfield et al . , 2004 ; wu et al . , 2007 ) , may increase the accuracy in the future . for the image - vector conversion both approaches share the same image transformation scheme and , therefore , they also share the same level of transformation accuracy . the difference is the spatial filtering : in this study no filter was employed for vba while one extra step to group voxels within pre - defined structures was employed for aba . this makes the impact of filtering comparable . assuming the brain volume is 1300  ml , there are 1.3  million  pixels in an image with 1    1    1  mm resolution . if these pixels are treated independently , the subsequent image structuration ( fig . another important difference is the way peripheral areas ( the csf , the cortex , and the white matter beneath the cortex ) are defined . because of their large anatomical variability , a problem that is aggravated by the atrophy , we can not expect accurate cross - subject registration ( diaz - de - grenu , 2014 ) , even with highly elastic transformation by lddmm . , the separation of these three compartments is performed in the native space of each subject ( fig . 1 ) , reducing the effect of voxel - by - voxel mis - registration . averaging of a large number of voxels in aba may also increase the sensitivity to detect diffuse atrophy patterns , although it could be insensitive to atrophy confined in a region significantly smaller than a parcel . a drawback of aba is that it requires pre - defined structural definitions . in this study , we used a parcellation based on classical anatomical definitions ( mori , 2008 ; oishi , 2009 ) . there are multiple other ways to parcellate the brain based on different criteria , for example , vascular , cytoarchitectonic , or functional maps . the employed parcellation may not optimally capture important pathological features ; the best parcellation scheme depends on many factors , including the pathology in question . another difficulty is to define the ideal level of granularity ( the number of parcels ) . if an abnormality exists in a confined area , the granularity must be high enough to detect it . on the other hand , if the abnormality is widespread and subtle , to use lower granularity is a better choice . it is known that the brain structures are based on hierarchical relationships and brain pathology could affect at the different levels of the hierarchy . for example , in a cohort of neurodegenerative diseases , various types of brain atrophy are known to exist depending on the patient pathology , such as left - dominant brain atrophy , focal temporal lobe atrophy , or hippocampal atrophy . as shown in fig . 3 , evaluation of the brain anatomy with multiple granularity levels could be an interesting approach to characterize atrophy patterns of neurodegenerative patients . in this study , we were interested in the level of granularity in which trained radiologists evaluate images ; it is certainly not voxel - by - voxel . judged from the level of inter - rater agreement , the granularity level seems to be less than 211 parcels . at level of 14 lobar - based parcels , we found good agreement among raters as well as between the human and automated analysis . more comprehensive study with more granularity levels may be needed to further investigate how human examines images . for conditions like ppa , which is known to contain heterogeneous pathological subtracts , the population - based analysis such as what shown in fig . if we want to incorporate anatomical phenotype information for clinical use , alternative analytical frameworks , in which individual data are retained and not population - averaged , are needed . using pca , we explored the interpretation of individual images within a context of the population data . in addition , a concept of similarity search based on the feature vectors was demonstrated ( figs . based on the nearest 10 cases in the pca plot , one can conclude that this is probably an svppa patient because 70% of his  neighbors  are sv , 20% are lv , and 10% are controls . this type of image search can provide evidence - based clinical reports ( sinha et al . , 2001 ) ; the pacs data can be searched purely on anatomical features , and statistical reports of associated clinical phenotypes can be provided , aiding the diagnosis and prognosis . of course , the searching approach described above considers only the anatomical phenotypes , which may not encode complete information about the pathology . one potential extension of the pca is to add other dimensions of information , such as clinical data , to create classifiers . for example , using pls - da , we achieved a reasonable accuracy for classifying individuals according the ppa variants ( fig . the idea of creating classifiers based on image features is not new and was successfully applied before to various neurodegenerative diseases , such as alzheimer 's , achieving a level of classification accuracy that suppress the human analysis ( cuingnet , 2011 ; eskildsen , 2013 ; fritzsche , 2010 ; gray , 2013 ; klppel , 2008a , b ) . while the level of the classification accuracy is one of the hallmarks of the success , our main interest in this paper is to examine if the resultant anatomical features that would discriminate the patient populations actually make sense in the light of clinicians ' accumulated knowledge , which they did ( gorno - tempini , 2011 , 2004 ; mesulam , 2009 , 2012 ) . these features can be easily translated , sought , and confirmed by the qualitative analysis , which is not always possible using vba . in the present study , we chose to use just the regional volumes , obtained from t1-wi analysis , as the image feature because the pathology in question causes brain atrophy primarily . however , this methodology is extensible to any other image contrast , such as t2-wis , dti derived - contrasts , swis , mrsi , or even to correlation matrices derived from resting state fmri , since they share the same anatomical space as shown in a previous publication ( faria , 2012 ) . the different image features , that can be contrasts such as fa , md , or metabolite concentrations , or connectivity maps , can be used isolated or combined as a matrix of structures by image features that would characterize each individual . then the same type of analysis can be performed and would be useful in cases where different domains of the anatomical phenotype affected ( zhang , 2011 ) . finally , we do not expect perfect diagnostic accuracy with the discriminant analysis ; first , as described above , we can not assume that the anatomy contains enough information about the pathology . second , our analysis does not include several critical information , such as clinical severity or distance from the onset . in addition , the diagnosis by clinicians , which is used as the gold standard for the classification , is known to contain errors and it is reasonable that the diagnoses change over time . in this context , the goal of this study was to maximize the usefulness of the anatomical information delineated by mri by systematically integrating the anatomical and clinical information from past cases . we achieved this goal by demonstrating effective structuration of image data through image - vector conversion , which provides new opportunities to mine existing clinical databases for medical decision support . this study is , however , still preliminary to judge the usefulness of the proposed approach in real clinical settings . to replicate the situations physicians routinely encounter in daily practices , another practical , yet , highly important question is if automated image analysis tools are robust enough to accurately analyze clinical data in pacs , which are often suboptimal in terms of image resolutions and contrasts . although our tool has been tested for pediatric ( faria , 2010 ) and geriatric populations ( oishi , 2009 ; djamanakova , 2013 ) , protocol impacts have not been rigorously tested . also , in order to determine the regional pattern of failure we need extensive manual delineation of hundreds of tridimensional structures in a large database . it is in our future scope to improve and test the robustness through technical advancement such as multiple atlas segmentation . the following is the supplementary data related to this article.appendix tableclinical information of ppa participants .
radiological diagnosis is based on subjective judgment by radiologists . 
 the reasoning behind this process is difficult to document and share , which is a major obstacle in adopting evidence - based medicine in radiology . 
 we report our attempt to use a comprehensive brain parcellation tool to systematically capture image features and use them to record , search , and evaluate anatomical phenotypes . anatomical images ( t1-weighted mri ) 
 were converted to a standardized index by using a high - dimensional image transformation method followed by atlas - based parcellation of the entire brain . 
 we investigated how the indexed anatomical data captured the anatomical features of healthy controls and a population with primary progressive aphasia ( ppa ) . 
 ppa was chosen because patients have apparent atrophy at different degrees and locations , thus the automated quantitative results can be compared with trained clinicians ' qualitative evaluations . 
 we explored and tested the power of individual classifications and of performing a search for images with similar anatomical features in a database using partial least squares - discriminant analysis ( pls - da ) and principal component analysis ( pca ) . 
 the agreement between the automated z - score and the averaged visual scores for atrophy ( r  =  0.8 ) was virtually the same as the inter - evaluator agreement . 
 the pca plot distribution correlated with the anatomical phenotypes and the pls - da resulted in a model with an accuracy of 88% for distinguishing ppa variants . 
 the quantitative indices captured the main anatomical features . 
 the indexing of image data has a potential to be an effective , comprehensive , and easily translatable tool for clinical practice , providing new opportunities to mine clinical databases for medical decision support .