clustered microcalcifications ( mcs ) can be an important early sign of breast cancer in women . mcs are calcium deposits of very small dimension and appear as granular bright spots in a mammogram ( e.g. , figure 1 ) . due to their subtlety in appearance and variation in size and shape in mammogram images , accurate diagnosis of mc lesions as benign or malignant is a very challenging clinical problem for radiologists . in recent years , there has been significant research in development of computer - aided diagnosis ( cadx ) techniques for clustered mcs , aiming to provide a second opinion to radiologists in their diagnosis in order to improve their performance and efficiency [ 13 ] . laboratory observer studies have shown that with cadx radiologists can improve their biopsy recommendation by sending more cancer cases and fewer benign cases for biopsy [ 25 ] . in cadx , a pattern classifier is typically first pretrained on a set of existing cases and subsequently applied to predict the likelihood that a given lesion is malignant or benign . for this purpose , many different machine - learning methods have been investigated , for example , [ 69 ] . in recent years , content - based image retrieval ( cbir ) has been studied as an alternative approach in cadx [ 1012 ] . instead of predicting likelihood , this approach is to provide radiologists with examples of lesions with known pathology that are similar to the lesion being evaluated . the purpose is to provide relevant information from the retrieved cases to boost the diagnostic accuracy on the case under consideration . in the literature , there exist a number of studies on the predictive value of retrieved mammogram cases . for example , the correlation in disease condition between the query and retrieved cases was examined in [ 14 , 15 ] . the fraction of malignant cases among all retrieved cases was used as a useful predictor for the query by floyd et al . the similarity level between a retrieved case and the query was used as a weighting factor in the prediction by zheng et al . . a genetic algorithm was used to adjust the weighting factors of the retrieved cases by mazurowski et al . . an observer study was used to investigate the potential diagnostic value of similar cases by nakayama et al . . recently , we have been exploring a case - adaptive approach to boost the performance of a cadx classifier by using retrieved similar cases [ 21 , 22 ] . the basic idea behind this approach is as follows : for a query case under consideration , we will first apply cbir to obtain a set of similar cases from a reference library of known cases ; we will then use these retrieved cases to modify the decision boundary of an existing classifier ( baseline ) in the neighborhood of the query case so as to improve its classification accuracy on the latter . in particular , we demonstrated this case - adaptive approach on a classifier based on logistic regression . the adaptive classifier was obtained through retraining with a weighted mixture of the retrieved cases and the training cases of the baseline classifier . this adaptive approach was demonstrated to yield improved classification accuracy when compared to its baseline counterpart . moreover , it could even outperform the classifier when it was retrained with all the cases in the reference library . based on this prior success , in this work , we further develop this case - adaptive classification approach by using a regularized adaptive classifier . one drawback of the adaptive classifier in   is the extra cost associated with retraining of the classifier for each query case . it also needs access to the training cases of the baseline classifier , which are required for retraining the classifier . to reduce this computational complexity associated with the adaptive classifier , we will use a prior to regularize the adaptive classifier as opposed to reusing the entire training set of the baseline classifier each time for a new query . this prior is derived from the baseline classifier , and it plays the following two key roles : ( 1 ) incorporate the information of the baseline classifier , and ( 2 ) prevent overfitting by the adaptive classifier when the number of retrieved samples is small . we will consider two specific forms for this prior : one is uniform for all query cases , and the other varies adaptively with the query . our results demonstrate that such a regularized adaptive classifier not only can be much simpler computationally , it also can adapt faster in performance with a small number of retrieved cases . regularization techniques are often used in machine learning to deal with ill - posed problems or to prevent over - fitting by an underlying model . they usually assume the form of a penalty to the complexity of the model , such as l2-norm penalty in ridge regression , which penalizes the length of the solution in a least - square problem . in the well - known support vector machine ( svm ) , the separation margin of the classifier is used in the form of an l2-norm penalty term . parallel to l2-norm , l1-norm penalty has also been used for regularization , for example , the lasso algorithm . in this work the regularization term has the form of l2-norm penalty , which can also be viewed as a prior distribution of the solution . the rest of the paper is organized as follows : the development of the adaptive classification schemes with regularization is given in section 2 . the problem we consider can be stated as follows : for a given query lesion x , we first obtain from a reference library a set of known cases which have similar image features to x ; our goal is to make use of these similar , known cases to improve the classification accuracy on x. to motivate the proposed development , below we first briefly review the case - adaptive approach developed previously in . for simplicity however , it can be readily extended to a nonlinear classifier by using the kernel trick as in . consider a linear classifier of the form : 
 ( 1)f(x)=wtx , 
 where x is a vector denoting an input pattern ( i.e. , lesion ) , and f(x ) is the classifier output which is typically compared against an operating threshold for decision on x. for notational simplicity , in ( 1 ) the input vector x is augmented by a constant element 1 so that the bias term is absorbed into the discriminant vector w. in practice , the unknown vector w is determined from a set of training samples { ( xi , yi ) , i = 1 ,  , n } , where the labels yi  { 0,1 } are given for each sample xi . in , we considered logistic regression , in which w is determined by maximizing the following log - likelihood function : 
 ( 2)l(w)=i=1nlogp(yi , xi;w ) , 
 where  p(yi = 1 , xi ; w ) = [ 1+exp(wxi ) ] . now , consider a query lesion x , and a set of nr retrieved cases { ( xj , yj ) , j = 1 , , nr } which are similar to x. the adaptive classifier for x is obtained by modifying the objective function ( 2 ) as 
 ( 3)lada(w)=i=1nlogp(yi , xi;w)+j=1nrjlogp(yj(r),xj(r);w ) , 
 where the weighting factors j are defined according to the similarity of xj to the query x . the idea is to put more emphasis on the retrieved samples , particularly those more similar to the query , so as to refine the decision boundary of the classifier in the neighborhood of x. for retrieval of similar cases , the euclidean distance between their image features to the query was used in , and it is also used in this work . observe that the objective function in ( 3 ) consists of two terms : the first term is that of the baseline classifier in ( 2 ) , and the second term is the weighted sum of the log - likelihood of the retrieved cases . conceptually , the first term can be viewed as a stabilizer for the adaptive classifier to avoid over - fitting for the retrieved cases ( which would lead to poor generalization on the query ) . however , this term involves all the training cases of the baseline classifier , which can be computationally demanding particularly when the number of retrieved cases is much smaller than the number of existing training cases , that is , nr  n. to address this problem , we propose a regularized approach for designing the adaptive classifier , as illustrated in figure 2 . the idea is to replace the baseline classifier term in ( 3 ) by a prior term on the discriminant vector w. naturally , this prior term is desired to be predetermined from the training set { ( xi , yi ) , i = 1 ,  , n } , so that the resulting adaptive classifier will be computationally more efficient for online implementation . let vector w- denote the solution of the baseline classifier in ( 2 ) , that is , the likelihood function l(w ) assumes maximum at w- . noting that the gradient l(w ) = 0 at w- , we can apply taylor series expansion about w- and rewrite l(w ) as 
 ( 4)l(w)l(w)+12(ww)t2l(w)(ww ) . thus , we can rewrite the modified objective lada(w ) in ( 3 ) as ( after ignoring the constant term ) 
 ( 5 )  lada(w)j=1nrjlogp(yj(r),xj(r);w)+12(ww)t2l(w)(ww ) . the second term in ( 5 ) can be viewed as a penalty term defined by a multivariate gaussian prior which has mean w- and covariance matrix [ -2l(w-)]-1 . consequently , the objective function in ( 5 ) assumes the form of maximum a posteriori estimation ( except that the log - likelihood terms of the retrieved cases are weighted according to their similarity level to the query ) . note that the hessian matrix 2l(w- ) in ( 5 ) can be pre - computed from the likelihood function l(w ) of the trainings samples . by comparing to ( 3 ) , we can see that the numerical complexity associated with the objective function in ( 5 ) is much reduced , because it consists of far fewer data terms than ( 3 ) when nr  n. furthermore , there is no longer need in ( 5 ) to access the training cases , which can be advantageous in practice . to further simplify the computational complexity of the adaptive classifier , in this study we assume that the components of w are independent and approximate the covariance matrix [ -2l(w-)]-1 in ( 5 ) by ci , where c is a constant . upon such approximation , we can further simplify the objective function in ( 5 ) as 
 ( 6)luni(w)=j=1nrjlogp(yj(r )    xj(r);w)c2||ww||2 . the constant c in ( 6 ) can be viewed as a parameter to control the influence of the regularization term , which has two important roles . first , it is used to prevent over - fitting by the adaptive classifier especially when nr is small . second , and more importantly , it is used to also enforce the fidelity of the adaptive classifier to the training cases { ( xi , yi ) , i = 1 ,  , n } as in ( 3 ) . consequently , luni(w ) in ( 6 ) consists of information from both the retrieved cases and the existing training cases . in particular , in the extreme case that c = 0 , the objective function luni(w ) in ( 6 ) simply amounts to retraining the classifier with only the retrieved cases ; on the other hand , when c =  , the adaptive classifier in ( 6 ) coincides with the baseline classifier w- . in this study , the weighting coefficient for a retrieved case xj in ( 6 ) is defined as 
 ( 7)j=jk=1nrk ,  where  j = exp(||xj(r)x||22 ) . that is , j varies according to the distance between xj and x. the parameter  is used to adjust the sensitivity of j with respect to the distance . in our experiments , the newton - raphson algorithm was used for optimization of the adaptive classifier in ( 6 ) . for completeness , the detailed algorithm is provided in the appendix in ( 6 ) , the regularization term is the same for all query cases , the purpose of which is to keep the adaptive classifier from being too different from the baseline classifier . as an alternative , it might be advantageous to adjust this term according to the input feature of the query case . below , we consider such an approach in which the mean vector w- in the regularization term is allowed to vary for each query case . that is , we modify the objective function in ( 6 ) as 
 ( 8)lvar(w)=j=1nrjlogp(yj(r )    xj(r);w)c2||wwvar||2 , 
 where w - var is now varied with the query case x. to determine the mean vector w - var , we first pre - determine an adaptive vector wi customized for each case in the training set , that is , { ( xi , yi ) , i = 1 ,  , n } , as described below . afterward , for a given query x , the mean vector w - var is interpolated from the adaptive vectors of the training cases according to their distances to the query . specifically , we have 
 ( 9)wvar=i=1nciwi , 
 where the weighting coefficients ci are so defined that those cases closer to x will have more contributions to w - var. in this study , the following is used for ci : 
 ( 10 )  ci=ik=1nk ,  where  k = exp(||xxi||22 ) . to determine the adaptive vectors wi for the cases in the training set { ( xi , yi ) , i = 1 ,  , n } , we modify the objective function in ( 2 ) as 
 ( 11)l~(w1,  ,wn)=i=1nlogp(yi , xi;wi)c2i=1n||wiwi||2 . the rationale for the introduced penalty term in ( 11 ) is that those cases with similar features should also have similar discriminant vectors . the parameter c is used to control the trade - off between this penalty and the likelihood term . in ( 11 ) , w - i denotes the weighted average of the adaptive vectors from the rest of the cases as in ( 9 ) with xi treated as the query in ( 10 ) . this dataset consists of digitized , standard - view , screen - film mammographic images collected from two sources : one from the department of radiology , the university of chicago ( uc ) , and the other from the ddsm dataset maintained at the university of south florida . altogether , there were a total of 1,006 cases ( 646 benign , 360 malignant ) in the dataset , all containing clustered mcs . to characterize the mc lesions , we use a set of nine features previously determined in , namely , ( a ) number of mcs in the cluster , ( b ) density of the cluster , measured by the number of mcs in a unit area , ( c ) mean of the mc size in the cluster , ( d ) eccentricity of the cluster , ( e ) standard deviation of the distance from individual mcs to the geometric center of the cluster , ( f ) maximum of the mean intensity of mcs , ( g ) mean of the average intensity in each mc window , ( h ) standard deviation of the contrast of mcs , and ( i ) standard deviation of the 4th order central moment of mcs . these features are used to form a vector x for each lesion in the dataset . a detailed description for the construction of this dataset can be found in . to demonstrate the proposed approach for case - adaptive classification the dataset of all 1,006 cases was first randomly divided into three subsets , denoted by s1 , s2 , and s3 , respectively , such that s1 and s2 consisted of 175 cases ( 100 benign , 75 malignant ) each , and s3 had the remaining 656 cases ( 446 benign , 210 malignant ) . these three subsets were used as follows : s1 was used as the training set , s2 was used as the test set for performance evaluation , and s3 was set aside as a library of known cases for retrieval for adaptive classification . the distribution of the different cases among the three subsets was out of the consideration to balance the malignant and benign cases for both training and testing while maintaining a large number of cases for retrieval . to avoid any potential bias , the training set s1 was used to determine the parameters c , c of the classifiers from the following candidate values : [ 0.001 , 0.01 , 0.05 , 0.1 , 0.2 , 0.5 , 1 , 5 , 10 , 100 ] using a 10-fold cross - validation . for parameter , we follow our previous work   and set it to be 1.63 , which corresponds to the 10th percentile of the inter - distance among training cases ; the test set s2 was used exclusively for evaluation . when testing the adaptive classifiers , for each case in s2 , a set of cases similar to the test case was retrieved from s3 , and subsequently used to train the adaptive classifiers . this was to ensure that the test case itself will not be used in any way for boosting the adaptive classifier . to evaluate the classification performance , we conducted a receiver operating characteristic ( roc ) analysis , which is now routinely used for performance evaluation in classification tasks . an roc curve is a plot of the classification sensitivity ( i.e. , true positive fraction ) as the ordinate versus the specificity ( i.e. , false positive fraction ) as the abscissa ; for a given classifier , it is obtained by continuously varying the threshold associated with its decision function . as a summary measure of overall diagnostic performance , the area under an roc curve ( denoted by auc ) the rockit program   was used to calculate the auc values for the different classifiers . to remove the effect of case distributions a total of 2,000 bootstrap sample sets were used [ 29 , 30 ] , of which each was obtained by sampling with replacements from the cases in s2 . the classifier performance was subsequently obtained over each bootstrap sample set . in our evaluation , the proposed adaptive approach was compared against the following different classifiers : ( 1 ) the baseline classifier ( 2 ) trained with s1 ; ( 2 ) the adaptive classifier in ( 3 ) . in addition , to demonstrate the effect of regularization , we also tested the adaptive classifier without using regularization , that is , by ignoring the regularization term in ( 6 ) . finally , we also considered the classifier trained with both s1 and s3 , which represents the scenario where all the cases in the retrieval library were used for training the classifier . in figure 3 , we show the performance results obtained by the regularized adaptive classifiers with uniform prior in ( 6 ) ( ada - reg - uni ) and with adaptive prior in ( 8) ( ada - reg - var ) . to demonstrate the effect of retrieved cases , the results are shown for the number of retrieved cases nr varied from 6 to 300 . for comparison , results are also given in figure 3 for the following classifiers : the baseline classifier ( lr ) , the adaptive classifier previously developed in   ( ada - lr ) , the adaptive classifier without regularization ( ada - reg - non ) , and , finally , the classifier trained with all the cases in s1 and s3 ( lr - all ) . from figure 3 , it can be seen that the regularized adaptive classifiers ada - reg - uni and ada - reg - var both could outperform the baseline classifier lr . in particular , with nr = 50 , ada - reg - uniachieved auc = 0.8111 and ada - reg - var achived auc = 0.8059 , compared with auc = 0.7329 for the baseline classifier lr ( p - value = 0.001 for ada - reg - uni , and 0.004 for ada - reg - var ) . with nr = 100 , ada - reg - uni obtained its best performance of auc = 0.8215 ; ada - reg - var achieved its best performance of auc = 0.8192 with nr = 200 . we believe that this is because that the benefit from additional retrieved cases diminishes as they are not sufficiently similar to the query . furthermore , from figure 3 , it can be seen that when nr < 50 ada - reg - var and ada - reg - uni are both higher in auc than ada - lr . this indicates that ada - reg - var and ada - reg - uni could adapt faster to the local decision boundary with a small number of retrieved cases . this could be attributed to the use of the prior in the regularized adaptive classifiers . with nr further increased ( above 100 ) , ada - reg - uni and ada - reg - var became similar in performance ; this is because with a large nr the retrieved cases became more influential than the prior on the classifier . the respective effects of retrieved cases and regularization can be illuminated by examining the results achieved by ada - reg - non , that is , when no regularization was used in the adaptive classifier . with nr < 50 , ada - reg - non was much lower in performance than even the baseline classifier lr ; this was clearly due to the issue of over - fitting . however , with increased nr , its performance auc was improved from 0.7633 with nr = 50 to 0.7927 with nr = 200 , approaching its regularized counterparts . furthermore , the regularized adaptive classifiers ada - reg - uni and ada - reg - var could also outperform the baseline classifier lr - all ( auc = 0.7643 ) which was trained with all the available cases in s1 and s3 . specifically , the ada - reg - uni and ada - reg - var outperformed lr - all with p - value = 0.004 ( nr = 100 ) and p - value = 0.005 ( nr = 200 ) , respectively . the rationale behind the proposed regularization - based approach for adaptive classification is to use a prior to regularize the adaptive classifier in order to prevent it from over - fitting to the retrieved cases . as can be seen from ( 6 ) , the regularization parameter c is used to control the balance between the retrieved cases and the baseline classifier . a larger c means more influence of the prior on the adaptive classifier ( and less influence by the retrieved cases ) , and vice versa . to demonstrate this effect , in figure 4 , we show the resulting performance achieved by the classifier ada - reg - uni with the parameter c varied over a large range . note that , as c  0 , the classifier performance approaches that of ada - reg - non ( i.e. , retrieval only , nr = 100 ) ; on the other hand , as c   , the classifier performance approaches that of the baseline classifier lr ( i.e. , no retrieval ) . the best performance was obtained with c varied between these two extreme cases . in figure 5 , we show the execution time taken for classifying the cases in the test set by the different adaptive classifiers ada - reg - uni , ada - reg - var , and ada - lr . for comparison , as can be seen , the regularized classifiers ada - reg - uni and ada - reg - var were similar in execution time . while slower than the baseline classifier lr , both were notably faster than ada - lr . specifically , with nr < 100 , the regularization - based approaches were about 10 times faster . for nr larger than 100 , the execution time increased for the adaptive classifiers as more samples were used in training , but still much lower than that of ada - lr . interestingly , the execution time for ada - lr slightly decreased with nr larger than 10 . we believe that this was due to improved conditioning in the hessian matrix of the objective function with increased nr . the higher numerical efficiency of the regularized classifiers over the adaptive classifier was due to their much simplified objective functions in which only retrieved cases were used . in this work , we investigated a regularization based approach for case - adaptive classification of microcalcification lesions in mammograms . deviating from a previously developed adaptive approach , in which a set of retrieved cases was used in conjunction with the training cases of a baseline classifier to re - retrain an adaptive classifier , we derived a prior in place of the baseline classifier as a regularization term in the adaptive classifier . this prior was used together with the retrieved cases from a reference library to optimize the classification on a query case . our goal was to reduce the numerical complexity associated with online training of the adaptive classifier . we explored two different forms for the regularization prior : one is invariant for the different query cases and the other is allowed to vary with respect to the features of the query cases . the results show that it could achieve significant improvement in numerical efficiency ( around 10 times in execution speed ) while maintaining similar ( or better ) improvement in classification accuracy compared to a previous nonregularization approach . the regularization approach was also observed to achieve faster adaption in performance with a small number of retrieved cases .
we propose a regularization based approach for case - adaptive classification in computer - aided diagnosis ( cad ) of breast cancer . 
 the goal is to improve the classification accuracy on a query case by making use of a set of similar cases retrieved from an existing library of known cases . in the proposed approach , a prior 
 is first derived from a traditional cad classifier ( which is typically pre - trained offline on a set of training cases ) . 
 it is then used together with the retrieved similar cases to obtain an adaptive classifier on the query case . 
 we consider two different forms for the regularization prior : one is fixed for all query cases and the other is allowed to vary with different query cases . in the experiments 
 the proposed approach is demonstrated on a dataset of 1,006 clinical cases . 
 the results show that it could achieve significant improvement in numerical efficiency compared with a previously proposed case adaptive approach ( by about an order of magnitude ) while maintaining similar ( or better ) improvement in classification accuracy ; it could also adapt faster in performance with a small number of retrieved cases . 
 measured by the area of under the roc curve ( auc ) , the regularization based approach achieved auc = 0.8215 , compared with auc = 0.7329 for the baseline classifier ( p - value = 0.001 ) .