text:
Description: Myrte - Hyves
Image was hearted from myrte-cupcake.hyves.nl
Other images from this site
myrte was first to heart it
11 people have hearted this image
This image is in 1 collection
You might like these too
Now available in your language!Use in your language! Go back to English
Other images from this site
You might like these too
Sign in with Facebook or Twitter to start your gallery.
Sign in with Facebook Sign in with TwitterDon't have Twitter or Facebook? Skip this step!


text:
To embed, copy and paste the code into your website or blog:
Last week, the Pacific Gas & Electric Company filed its General Rate Case Phase 2 Application with the California Public Utilities Commission (CPUC). The application requests that the CPUC approve new electric rates for PG&E's customers for 2014 through 2016. The proposed rates will allocate costs among the utility's different classes of customers. Rate changes to some customer classes are likely to be substantial as PG&E has requested a cumulative $5.3 billion in increased revenue in a related Phase 1 proceeding. Protests to the application are due by May 17. Public agencies wishing to protect their interests in this proceeding, as either individual customers of electricity or on behalf of their residents, should act soon.
As a CPUC-regulated utility, PG&E must receive CPUC approval before increasing or otherwise modifying its rates. Generally, utilities update their rates every three years through a "general rate case." These cases are conducted in phases. Importantly, the first phase requests the total amount of the requested revenue increase. The second phase then determines how to allocate these increased revenues among the utility's various customer classes.
Here, PG&E filed Phase 1 of its 2014-2016 General Rate Case in late 2012. The Phase 1 proceeding seeks approximately $5.33 billion in new revenues over the three-year rate cycle, an approximate 33 percent increase, with individual annual increases of $1.28 billion for 2014, an additional $492 million for 2015 and another $504 million for 2016. The Phase 1 proceeding is on-going.
On April 18, 2013, PG&E filed Phase 2 of the rate case. This phase is in many ways more important for different classes of customers because if a customer class is not adequately represented in the proceeding, it may disproportionately bear the weight of the increases.
The deadline to protest the Phase 2 application is May 17, 2013. If you wish to participate in the proceeding, BB&K is able to assist your agency. For example, BB&K is currently representing a coalition of water and school districts in a San Diego Gas & Electric rate proceeding. In that proceeding, SDG&E has proposed rate structures that disproportionately affect public agency customers with solar generation projects. We believe that the coalition's participation in the SDG&E proceeding will protect the agencies' investment in these projects.
For more details regarding the PG&E proceeding, or if you wish to participate individually or through a coalition of BB&K clients, please contact Jason Ackerman, Joshua Nelson or your BB&K attorney.
See more »
DISCLAIMER: Because of the generality of this update, the information provided herein may not be applicable in all situations and should not be acted upon without specific legal advice based on particular situations.
© Best Best & Krieger LLP
Your first step to building a free, personalized, morning email brief covering pertinent authors and topics on JD Supra:
Sign up to create your digest using LinkedIn*
Back to Top
Copyright ©

subset: commoncrawl


text:
People tend to check in during these times:
Add hours
Margharita pizza is awesome!
Food was very dry and we paid 6 bucks for draft beers. We sat outside and watched everyone go in to MoeJoes. Looks like we picked the wrong place to go.
Great burgers and tater tots.
These are the best burgers around!
3mile burger is great
Play trivia on Monday nights at 8pm!!
Open mic Thursdays at 9:30. Sing with the band. Sit in with the band or bring your own band. All you have to do is sign up! Hosted by Igor and G-Spot. House band Dirty O'hara's.
Monday night trivia is always fun starts at 8:15! Free to play, tasty prizes and $60 in prizes! 1st $30 2nd $20 3rd $10
The chicken strip kids meal is a very generous portion an they're actually still good the next day, even cold!
Found a great tip! 20% Off Total Bill
tell Jason you checked in and he will give you an awkwardly long hug:-)
One of the best burgers I've had
Try the meatball sliders, awesome!!!
You have to try the meatball sliders, awesome!!!!!


text:
Machine learning is a powerful approach for describing and predicting classes in microarray data. Although several comparative studies have investigated the relative performance of various machine learning methods, these often do not account for the fact that performance (e.g. error rate) is a result of a series of analysis steps of which the most important are data normalization, gene selection and machine learning.
In this study, we used seven previously published cancer-related microarray data sets to compare the effects on classification performance of five normalization methods, three gene selection methods with 21 different numbers of selected genes and eight machine learning methods. Performance in term of error rate was rigorously estimated by repeatedly employing a double cross validation approach. Since performance varies greatly between data sets, we devised an analysis method that first compares methods within individual data sets and then visualizes the comparisons across data sets. We discovered both well performing individual methods and synergies between different methods.
Support Vector Machines with a radial basis kernel, linear kernel or polynomial kernel of degree 2 all performed consistently well across data sets. We show that there is a synergistic relationship between these methods and gene selection based on the T-test and the selection of a relatively high number of genes. Also, we find that these methods benefit significantly from using normalized data, although it is hard to draw general conclusions about the relative performance of different normalization procedures.
Machine learning methods have found many applications in gene expression data analysis, and are commonly used to classify patient samples into classes, corresponding to for example cancer sub-type, based on gene expression profiles. Supervised learning is a powerful tool in these studies since it can be used both to establish whether the classes of interest can be predicted from expression profiles and to provide an explanation as to what genes underlie the differences between classes. The expression data in such studies typically undergo an analysis pipeline in which the most important steps are data normalization, gene selection and machine learning. Although there are several comparative studies of methods for normalization, gene selection and machine learning, none have studied how all of these analysis steps influence each other and the final model performance.
A wealth of methods exists for microarray normalization, gene selection and machine learning. Normalization of microarray data involves several possible steps [1], including background correction [2] and dye-normalization [3]. The relative performance of different normalization approaches, although not in the context of machine learning, has previously been evaluated using spike-in data sets [4,5]. Previous studies have also shown that normalization has an impact on clustering [6,7]. One of the challenges in using machine learning and gene expression data to study medical diagnosis is the large number of genes (features) compared to the relatively limited number of patients (observations). Many gene selection methods have therefore been developed to cope with this problem [8,9]. Approaches to gene selections are either filter methods or wrapper methods. Filter methods score, rank and select the best individual genes before the machine learning methods is applied, while wrapper methods score subsets of genes according to the performance of machine learning models induced from the subset. Machine learning methods are commonly used in bioinformatics applications both for clustering (i.e. unsupervised learning) and for inducing predictive models from examples (i.e. supervised learning) [10].
Since gene selection is a necessary step in machine learning-based analysis of microarray patient data, all existing comparative studies have investigated the effect of gene selection and machine learning methods on classification performance. Most of these studies considered tumor classification. However, to the best of our knowledge, no study has also taken data normalization methods into account. Pirooznia et al. [11] studied the performance of three gene selection methods and six machine learning methods on eight microarray data sets, and mainly highlighted the importance of gene selection and the number of selected genes. Romualdi et al. [12] investigated four gene selection methods and six machine learning method using both simulated data and two microarray data sets, and demonstrated that non-parametric methods such as Support Vector Machines (SVMs) and Artificial Neural Networks (NNs) were more robust than parametric methods. Lee et al. [13] performed an extensive comparison of 21 machine learning methods and three gene selection approaches on seven microarray data sets. Their main conclusions were that more sophisticated classifiers such as SVMs perform better than classical methods and that the choice of gene selection method has a large effect on the performance. Li et al. [14] investigated eight gene selection methods and seven machine learning methods using nine data sets, and concluded that SVM methods perform best and that the choice of machine learning methods is more important than the choice of gene selection methods. Statnikov et al. [15] applied four gene selection methods and four machine learning methods, as well as several different SVM methods and some ensemble methods, to classify patients in 11 data sets. They also concluded that SVM methods performed better than non-SVM methods, that no significant improvement was obtained using ensemble methods and that gene selection improved all machine learning methods. All comparative studies used cross validation to evaluate the performance of different methods. Several previous studies have stressed the challenges related to the applications of machine learning methods and the importance of objective evaluation [16-18]. In particular, Zervakis et al. [19] used several gene selection and machine learning methods to show how the performance of gene selection methods vary with different validation strategies and concluded that independent test sets are important for validation.
The previous comparative studies have shown that classification performance varies a great deal from data set to data set. This is a challenge in comparative studies since it limits our ability to find general trends in terms of methods and combinations of methods that perform best across data sets. Here we approach this problem by comparing methods and pairs of methods on individual data sets, and then by visualizing trends across data set using heat maps. Thus we are able to draw general conclusions both about the individual effect of normalization, gene selection and machine learning on classification performance, and also to say something about synergistic effects that occur when these methods are used in combination. Our approach to studying synergy between methods, and the fact that we study the effect of normalization as well as gene selection and machine learning, makes our study unique. Our main conclusions are that Support Vector Machines (SVMs) with a radial basis kernel, linear kernel or polynomial kernel of degree 2 perform best across data sets. We show that these methods exhibit a synergistic relationship with gene selection based on the T-test and the selection of a relatively high number of genes. All these methods perform better on normalized than on non-normalized data, however, while the radial basis kernel and the linear kernel perform best when the data is not background corrected, the polynomial kernel benefit from background corrected data.
We evaluated classification models induced from seven different two-channel microarray expression data sets with two known classes (Table 1). Each classification model is the result of a combination of different computational methods for microarray normalization, gene selection, number of selected genes and machine learning. We included five different approaches to normalization, three gene selection methods, 21 different numbers of genes and eight different machine learning methods (Table 1). In total 14685 models were induced corresponding to all possible combinations of the different methods and data sets. We will refer to data sets and methods according to the acronyms given in Table 1 (see Methods for more details).
Table 1. Overview of the data sets and the methods used in this study
Combinations of methods were validated by employing a double cross validation (CV) approach where the inner loop was used to find good parameter settings (e.g. number of units in the hidden layer of a Artificial Neural Network (NN)) and the outer loop was used to estimate the predictive power of the final model in terms of average error rate (Figure 1). Normalization was done initially, before the cross validation, while gene selection was done inside the outer cross validation.
Figure 1. Method overview. The figure illustrates the analysis pipeline used to induce and validate models. First data was normalized (or raw data are used: No 0). Then a 5-fold cross validation (CV) was conducted to divide data into training and test sets. The training sets were used to train the models (red box), while the test sets were used to validate their classification power. In order to induce a classification model, some parameters had to be tuned. For example, the different Support vector machines (SVMs) employed different kernels with one or more parameters. The parameter sigma in the radial basis kernel was tuned by conducting a grid search and choosing the values with the lowest error rate during a 10-fold CV. The selected parameter value was finally used to induce a model from the training set in the outer CV and to classify the observations in the corresponding test set. The outer 5-fold CV was performed 10 times resulting in 50 test sets from which we evaluated 50 different models trained on 50 different trainings sets. As a measure of classification performance we used the average fraction of misclassified observations (i.e. error rate) in these 50 test sets.
Method choices explain variation in classification performance within data sets
The average error rate obtained by evaluating all the different combinations of methods on the seven data sets was 0.1512 with a standard deviation of 0.1195 (Additional file 1). The observed variation has two causes; the varying performance of different combination of methods used to induce models and the varying difficulty of discriminating the two classes in different data sets. The later could to some degree be explained by the varying distributions of observations over the two classes in the data sets. To reduce this effect, error rates were adjusted by dividing by the theoretical error rate obtained by random class assignment from the known distribution of classes. Thus adjusted error rates below one correspond to models performing better than random class assignment. Henceforth, when we discuss error rates, we will refer to the adjusted version of error rates if not specifically stated otherwise. The average adjusted error rate from all the 14685 models was 0.3668 with a standard deviation of 0.3159. This indicates that, although the adjustment reduces the influence due to unbalanced class sizes, most of the variation has other causes. Figure 2 shows that the error rates vary a lot between different data sets, but also that there is variation within data sets. It is this latter variation, due to the choice of combinations of methods, that is of interest to us in this paper.
Additional file 1. Error rates obtained for all combinations of methods and data sets. A table of the 14 686 different combinations of methods and data sets. Columns include Data set (D), Machine learning method (M), Gene selection (G), Number of selected genes (N), Normalization method (No), Error rate (E), Standard deviation of error rate (Esd), Adjusted error rate (Eadj) and Standard deviation of adjusted error rate (Esdadj).
Format: XLS Size: 2.9MB Download file
This file can be viewed with: Microsoft Excel Viewer
Figure 2. Overall error rates for each data set. Box plots showing the error rates resulting from all the different combinations of methods applied to the seven data sets.
To analyze the effect that different methods have on the classification performance, we used multiple linear regression [20] with methods as predicting variables and error rate as the response variable. Predicting variables included both first order terms and second order interaction terms constructed from Data set (D), Normalization (No), Gene selection method (G), Number of selected genes (N) and Machine learning method (M) (see Table 1). The regression model had an adjusted R-squared [20] value of 0.9589. The analysis showed that Data set (D) is by far the most explanatory predicting variable completely overshadowing the other variables. This is in accordance with what we already saw in Figure 2, and provides little information as to what is the best combination of methods. Thus, we performed a new regression analysis without Data set (D) as predicting variable, resulting in an adjusted R-squared value of only 0.0283. As a consequence of the lack of general, interpretable results from the regression analysis across data sets, we changed strategy to first analyze each data set individually and then search for general trends across data sets. This seems to be a viable strategy; adjusted R-squared values for these new regression models from the seven individual data sets range from 0.74 to 0.97, indicating that variation in error rate within data sets indeed can be explained by method choice. Also, all the first order terms (except G in one data set) and most interaction terms (except N+G and N+M in one data set) have significant explanatory power in the regression models (see Methods).
Significant differences in performance of individual methods
To study the performance of individual methods, e.g. the gene selection method Relief, we plotted error rates resulting from applying all relevant combinations of methods to individual data sets and to all data sets (Figure 3, Figure 4, Figure 5 and Figure 6). We also compared all methods head-to-head and visualized statistically significant differences between pairs of methods across data sets (see Figure 7, Additional file 2 and Methods).
Figure 3. Classification performance relative to normalization method. Box plots showing the error rates resulting from all method-combinations utilizing a specific normalization method. Since the Finak and Galland data sets could not be normalized with methods 1 and 2, these methods are absent from the relevant plots.
Figure 4. Classification performance relative to gene selection method. Box plots showing the error rates resulting from all method-combinations utilizing a specific gene selection method.
Figure 5. Classification performance relative to different numbers of genes. Box plots showing the error rates resulting from all method-combinations utilizing a specific number of genes.
Figure 6. Classification performance relative to machine learning method. Box plots showing the error rates resulting from all method-combinations utilizing a specific machine learning method.
Figure 7. The predictive performance of individual methods across data sets. The heatmap visualize the number of data sets in which one method (row) performed significantly better than another method (column). The Wilcoxon signed-rank test was used to compare the error rates of all combinations containing one method against the error rates of all combinations containing the other method. Significance was determined using a Bonferroni corrected p-value threshold (i.e. 0.05 divided by the number of tests).
Additional file 2. The predictive performance of individual methods across data sets. Comparisons of the predictive performance of individual methods (data material for Figure 7).
Format: XLS Size: 21KB Download file
This file can be viewed with: Microsoft Excel Viewer
Across data sets, the normalization methods perform rather similar (Figure 3); the somewhat worse results for No 1 and 2 (i.e. print-tip normalization) can be explained by the fact that these normalization methods are not relevant to the Agilent data sets (i.e. Finak and Galland) of which the Finak data set results in extremely well-performing models for almost all methods (Figure 2). Within data sets, however, the picture is rather complex (Figure 3 and Figure 7). In the Sørlie data set, models based on non-normalized data (No 0) somewhat surprisingly outperform models from normalized data (No 1-4) by a large margin, but non-normalized data is also significantly better than all normalized data in Herschowitz. On the other hand, all normalized data is statistically better than non-normalized data in Ye, Alizadeh and Jones. Thus the major differences are observed between non-normalized and normalized data. Significant differences between normalization methods (No 1-4) are sporadic, although No 4 is significantly better than No 2 and No 3 in three data sets.
Gene selection
Also gene selection methods exhibit a rather complex pattern with some data sets showing a gene selection preference (Figure 4 and Figure 7). The T-test outperforms the two other methods in Ye and Galland, and is significantly better than Paired distance and Relief in five and four data sets, respectively. Relief is significantly better than Paired distance in Alizadeh, Galland, Herschkowitz and Ye. However, Paired distance is significantly better than the two others in Jones and Sørlie. Hence, while the general trend is that the T-test performs best followed by Relief and Paired distance, the Sørlie data set again shows the opposite trend.
Number of selected genes
Figure 5 shows that there is a clear improvement in error rates when going from selecting only the two best genes from the gene selection methods to selecting several genes. We also see that, for several data sets including Ye, Sørlie and Galland, more genes imply better performance, although this trend decreases as we add many more genes than we have observations. From Figure 7 it is clear that there is no statistically significant improvement in a majority of the data sets from including more than 200 genes. Since we are applying the cross validation approach, and thus estimate error rates on data that is unseen by the models, we expected the predictive power of the models to suffer somewhat from including many genes. What we instead observe is that the performance keeps improving or stays constant, but never gets worse. Interestingly, the three data sets that keep improving even when more than 200 genes are used (i.e. Ye, Sørlie and Galland) also have the fewest number of differentially expressed genes among all data sets (Table 2).
Table 2. Differently expressed genes in the datasets
Machine learning
In general, Support Vector Machines (SVMs) have the best classification capabilities followed by Artificial Neural Networks (NNs), while Decision Trees (DTs) have the worst performance (Figure 6 and Figure 7). However, DTs clearly outperform other machine learning methods in Sørlie, which again shows the opposite trend of the other data sets. Although there are some clear conclusions to be drawn from the performance of machine learning methods, Figure 7 also portray a rather complex picture where most methods perform significantly better than most other methods in at least one data set. Notably, SVMs with a radial basis (Rb) kernel are only outperformed in single data sets, but SVM Linear and SVM Poly 2 are also performing well.
Synergistic relationships exists between methods
One of the main questions we ask in this study is to what degree we can observe synergistic relationships between methods. In order to answer this question we need to study combinations of methods. To have enough data to claim statistical significance, we limited our study of synergy to comparing all method-pairs of the same type, i.e., machine learning-normalization (M-No), machine learning-number of genes (M-N), machine learning-gene selection (M-G), gene selection-normalization (G-No), gene selection-number of genes (G-N) and normalization-number of genes (No-N) (see Figure 8, Figure 9, Figure 10, Additional file 3 and Methods).
Additional file 3. The predictive performance of pairs of methods across data sets. Comparisons of the predictive performance of pairs of methods (data material for Figures 8-10 and additional comparisons).
Format: XLS Size: 338KB Download file
This file can be viewed with: Microsoft Excel Viewer
Figure 8. The predictive performance of machine learning and normalization methods across data sets. The heatmap visualize the number of data sets in which one pair of methods (row) performed significantly better than another pair of methods (column). The Wilcoxon signed-rank test was used to compare the error rates of all combinations containing one pair against the error rates of all combinations containing the other pair. Significance was determined using a p-value threshold of 0.05.
Figure 9. The predictive performance of machine learning and gene selection methods across data sets. See figure text of Figure 8.
Figure 10. The predictive performance of gene selection and normalization methods across data sets. See figure text of Figure 8.
Obviously, the performance of individual methods affects the performance of pairs of methods. However, in practice we always select one method of each type (i.e. one method for normalization, gene selection, number of genes and machine learning), thus methods that do not significantly deteriorate the performance of other individually well-performing methods are of interest as well as methods that decrease performance of other methods and therefore represent combinations that should be avoided. Method-pairs that contain the number of selected genes (N) are generally exhibiting the pattern already seen for single methods; too few genes severely reduce the performance of most methods while more than 200 genes do not significantly improve performance. Thus we will mainly focus on pairs consisting of methods for machine learning, normalization and gene selection (all comparisons can be seen in Additional file 3). Furthermore, we are particularly interested in pairs where the best machine learning methods (SVM Rb, Linear and Poly 2) are significantly affected by normalization or gene selection. The overall performance of a method-pair can be summarized by counting the number of data set where this pair is significantly better than other pairs (i.e. the sum of one row in Figure 8, Figure 9 and Figure 10, see Additional file 3 for these calculations).
Machine learning and normalization (Figure 8)
Compared to all other pairs of machine learning and normalization, pairs containing SVM Rb perform the best. SVM Rb performs better on normalized than on non-normalized data. The best normalization method to use with SVM Rb is No 3 followed by No 1, although the improvement over No 2 and No 4 is rather small. The same pattern can be seen for the second best machine learning method, SVM Linear, indicating that these methods combine well with normalization methods not utilizing background correction (i.e. No 1 and No 3). However, the picture is rather complex. For example, the third best method, SVM Poly 2, works best with background corrected normalizations (No 2 and No 4), while normalizations without background correction is no better than non-normalized data.
Machine learning and gene selection (Figure 9)
A clear trend is that the T-test is the best gene selection partner to all machine learning methods, although the actual improvement sometimes is rather small. SVM Rb combined with the T-test is the best performing method-pair, and T-test is also the best gene selection partner for SVM Linear and SVM Poly 2.
Gene selection and normalization (Figure 10)
Almost all pairs of gene selection and normalization methods are significantly better than almost all other pairs of this type in at least two data sets. The lack of clear trends is maybe not so surprising considering the complex behavior of these methods when studied individually. Surprisingly, there is a synergy between the T-test and non-normalized data, which is the best pair of gene selection and normalization. If we look at the performance of the normalization methods for each gene selection method separately, we again see the trend from our analysis of individual methods; three to four data sets result in significantly better models when normalized while at least two other data sets actually result in better models when not normalized.
The comparison of method-pairs confirms the strong performance of the individually best performing machine learning methods (SVMs Rb, Linear and Poly2). These methods require at least 150 genes to achieve their best classification performance. A trend is that SVM Rb is somewhat more robust with respect to normalization, gene selection and number of selected genes than the other well-performing machine learning methods. For example, both SVM Linear and Poly 2 perform worse when not using gene selection based on the T-test, while SVM Rb performs almost as well with one of the two other methods. This example illustrates a very important trend in our study; although some methods perform well by themselves, their performance can be severely hampered by unfortunate choices for the other methods. In this context, data normalization is shown to be very important; the wrong normalization method can severely reduce the performance of many of the best machine learning methods. Also, these best methods perform better on normalized than non-normalized data. There is also a very clear positive effect of using normalized data with DT methods, while NN methods actually perform best on non-normalized data.
Of particular interest are synergistic effects where two methods perform significantly better together than any of the two methods do individually. We specifically searched for such patterns (Additional file 4), and confirmed synergistic relationships between the three best machine learning methods (SVM Rb, Linear and Poly2) and both the T-test (significant in at least three datasets) and the selection of at least 200 genes (significant in at least four data sets). There is also synergy between No 3 (normalization without background correction) and both SVM Rb and Linear (significant in three data sets), while SVM Poly2 has a synergistic relationship with background corrected data (No 2 and 4, significant in three and four data sets, respectively). One surprise is that this analysis confirms synergy between non-normalized data and the T-test (significant in four data sets). However, since the best machine learning methods perform well with normalized data and the T-test, this result is a curiosity of little practical importance.
Additional file 4. Synergistic relationships between pairs of methods across data sets. The matrix show the number of data sets in which pairs of methods (given by one row and one column) performed significantly better (upper-right half, positive values) or significantly worse (lower-left half, negative values) than the best/worst method did individually. The Wilcoxon signed-rank test was used to compare the error rates of all combinations containing the pair against the error rates of all combinations containing the best/worst single method. Significance was determined using a p-value threshold of 0.05.
Format: XLS Size: 26KB Download file
This file can be viewed with: Microsoft Excel Viewer
Statistical significance
To establish statistical significance of the reported error rates, we performed permutation tests by inducing models and estimating error rates after randomly shuffling the class labels of all data sets (Figure 11). Since such tests are computationally expensive, we limited the study to one method combination that was chosen based on earlier discussions related to performance and synergy; No 3, T-test, 150 genes and SVM Rb. The permutation tests show that shuffled data sets result in adjusted error rates centered close to 1.0 and that our reported error rates for the original data sets are highly significant with the exception of Sørlie. As discussed earlier, the Sørlie dataset stands out from the other data sets in many ways. For example, Sørlie results in better models when non-normalized and when inducing models using decision trees.
Figure 11. Permutation tests. The plots show the spread of adjusted error rates resulting from randomly shuffling class labels in each data set. The results were obtained using the method combination No 3, T-test, 150 genes and SVM Rb, and each data set was permutated 300 times. Adjusted error rates from the original class labels are marked by red crosses.
In this paper, we have studied the effect of data normalization, gene selection and machine learning on the predictive performance of models induced from cancer-related expression data. Performance was rigorously assessed by repetitively employing a double cross validation approach to each method combination and each data set. We analyzed seven cancer related two-channel microarray data sets published in high-impact journals. We were particularly interested in studying the effect of normalization in two-channel experiments where a generally agreed-upon standard for normalization still does not exist. Indeed we see some trends related to normalization. Normalized data resulted in better models than do non-normalized data when employing the best machine learning methods. In particular, the two best machine learning methods (SVM Rb and Linear) showed a synergistic relationship with normalizations not using background correction (i.e. No 1 and No 3).
The data sets in this study result in machine learning models that perform rather differently both with respect to error rate and also with respect to relative performance between methods. Although all data sets are from two-channel microarray experiments related to cancer, the classes we have used to train and evaluate our models are defined somewhat differently. However, all classes are based on clinical observations (for details see Table 1). We chose to approach this challenge of heterogeneous data sets and classes by initially evaluating the data sets separately and then by looking for general trends across data sets.
Overfitting occurs when the learning framework selects models that performs better on the training set but worse on the external test sets [21]. This is a particularly severe problem for the types of data sets studied here, since we have many more genes than patients, and thus a high risk of selecting genes that discriminate classes in the training set but that do not generalize to the test set [17]. To detect overfitting and obtain robust results, we performed a double cross validation where the inner loop was used to optimize parameters (when applicable) and the outer loop was used to estimate classification performance. A k-fold CV was chosen since previous research has shown that this reduce the bias compared to leave-one-out CV [22]. In addition, we re-ran the CV several times to minimize the effect that particular data splits have on the results.
To simultaneously study the effect of normalization, gene selection and machine learning implies testing a large number of method combinations. In order to reduce running time, we therefore had to make certain adjustments to the analysis pipeline used to induce and validate models. Firstly, while we did gene selection inside the outer CV loop, we chose not to perform a separate selection inside the inner loop (see Figure 1). Since the inner loop is used to optimize parameters only, we found this to be a reasonable compromise to reduce running time while still keeping training and test sets completely separate when estimating the reported error rates in the outer CV loop. Secondly, we decided to fix several parameters in the machine learning methods. Although we carried out tests to make sure we optimized the most important parameters, this approach could give some advantage to methods that have parameters that were in fact optimized. For example, optimizing the Gaussian kernel parameter σ could somewhat benefit SVM Rb since this is the only SVM method left with a parameter that was tuned. Finally, we only performed permutation tests for one method combinations (300 permutations with No 3, T-test, 150 genes and SVM Rb). However, this test was sufficient to show that permutated data results in adjusted error rates centered close to 1.0 and thus that the best methods in this study clearly produce statistically significant results.
Due to the above mentioned risk of overfitting, and also due to practical issues such as running time, gene selection is required on data sets with many genes and few observations (e.g. patients). We were rather surprised to see that all data sets upheld predictive performance even when 1000 genes were selected from the gene selection methods, and that the error rate actually improved for three data sets. Interestingly, the three data sets for which performance continuously improved when more genes were selected, were the same three data sets that have the fewest number of discriminatory genes (Table 2); Ye, Sørlie and Galland. These data sets are also among the four worst performing data sets in terms of average error rate (Galland is slightly better than Herschkowitz). Thus, one might conclude that the inclusion of many genes give better performance for data sets with many weakly discriminatory genes as compared to data sets with strongly discriminatory genes. It was also encouraging to see that our training pipeline seems robust to overfitting in that the inclusion of weakly discriminatory genes (most data sets have fewer than 1000 significantly discriminatory genes, Table 2) did not affect classification performance negatively for any data set.
The gene selection methods considered in this study were all filter-based methods. These methods select genes prior to machine learning by typically ranking genes based on their individual ability to separate classes [9]. Unfortunately, this reduces the possibility for advanced non-linear machine learning methods to find complex discriminatory patterns or decision boundaries based on genes that individually are weakly discriminatory or even completely non-discriminatory. Wrapper-based gene selection methods can in principle find such genes by iteratively testing subsets of genes that result in high-performing models [9]. However, for the data sets that we are studying here, with tens of thousands of genes and only around 100 patients, wrapper methods are unrealistic not only in terms of time complexity, but also due to the risk of overfitting. In this context, we designed a new gene selection method (called Paired distance) that first selects genes with high variance or high absolute mean, and then ranks pairs of genes with high discriminatory power (see Methods). Our hope was that this method would identify discriminatory gene-pairs containing genes that individually were not selected by other gene selection methods and thus would improve models from data sets with only weakly discriminatory genes. Intriguingly, this method turned out to be the best gene selection method on the worst performing data set in this study, which also is the only data set without any significantly discriminatory genes (i.e. Sørlie, see Figure 4).
In this study, we selected methods that are commonly used to analyze gene expression data. Thus, although we see rather small difference between, for example, gene selection methods, this does not mean that gene selection is not important. Initially, we also included a gene selection method that rank genes based on variance, however, the consistently poor performance of this unsupervised method spurred us to omit it from further analyzes.
For comparing the performance of methods, we used the Wilcoxon signed rank test. This test considers the ranking of comparable pairs of method combinations (e.g. when comparing normalizations No 1 and No 2, combinations with equal choices for the other methods are paired, that is, the combination No 1, G T-test, N 100, M SVM Linear is compared to No 2, G T-test, N 100, M SVM Linear). Such paired tests are sensitive, and reveals interesting significant differences even when average performances are rather similar (i.e. Figures 3, 4, 5, 6). For individual methods, the Bonferroni correction was used to decide on statistical significance. However, when pairs of methods were considered, much fewer error rates were available to the statistical test, thus making methods for multiple hypothesis correction too insensitive. In these cases (Figures 8, 9, 10 and Additional files 3-4), we opt to use a fixed p-value threshold of 0.05 to identify interesting synergistic relationship, knowing that among these one must expect an increased number of false positives (se Methods for more details).
The data sets and class definitions investigated in this article have previously been used to compare unsupervised clustering methods [7]. There are some interesting parallels to be drawn between these two studies. First and foremost, both studies experienced huge differences in performance between data sets, and in particular the results from Sørlie did not agree with that of the other data sets. Normalization was shown to have a positive effect on both clustering and machine learning, but it was harder to draw any decisive conclusions about the relative performance of different normalization methods (i.e. No 1-4). Although the clustering study mainly focused on unsupervised gene selection methods, both studies found that relatively high numbers of genes were needed to obtain good performance.
In this study, we have performed a comprehensive study of the effect that normalization, gene selection, the number of selected genes and machine learning method have on the predictive performance of resulting models. A unique aspect of this study was the inclusion of different normalization methods in the comparisons. Indeed, we showed that there is a significant positive effect of normalization on the best methods; however, the relative performance of different normalization methods is complex. The best machine learning methods in this study were Support Vector Machines (SVMs) with a radial basis kernel followed by SVMs with a Linear kernel and SVMs with a polynomial kernel of degree 2. We showed that there is a positive, synergistic relationship between these methods and gene selection based on the T-test and the selection of at least 150 genes.
We used seven previously published 2-channel microarray experiments with a common reference design [23]. All data sets regard different types of human cancer. Five of the experiments used custom made arrays, while the last two used commercially produced arrays (Agilent). Since the design and normalization procedures of one-channel microarray experiments (e.g. Affymetrix) are not comparable with 2-channel experiments, they are not included in the study although they are widely used. The data sets contain from 40 to 133 samples each and are all published in high-impact journals.
Since the aim of this study was to investigate how the performance of various machine learning methods is affected by the choice of normalization method and gene selection, we must have true class partitionings to compare with the predicted classes from the models. To make it easier to compare the results for different models and data sets, we choose to divide data into two distinct classes:
Alizadeh: 133 samples corresponding to patients with different lymphomas, including patients with diffuse large B-cell lymphoma (DLBCL) [24]. Samples also include normal cell samples and a variety of cell lines. We define the two classes as DLBCL (68) and all other samples (65).
Finak: 66 samples profiled using Agilent microarrays [25]. We define the two classes according to the distinguishable tissue types; epithelial (34) and stromal tissue (32).
Galland: 40 samples profiled using Agilent microarrays [26]. The classes are defined as invasive (22) and non- invasive non-functioning pituitary adenomas (NFPAs) (18).
Herschkowitz: 106 samples. We define the two classes according to the level of ER expression; high (59) or low (47) [27].
Jones: 91 samples. We define the classes as the two most distinctive ones in the data set; the cancerous (72) and the non-cancerous (19) samples [28].
Sørlie: 73 samples. The classes are defined according to the level of ER expression; high (55) or low (18) [29].
Ye: 87 samples. We define the two classes as in the original study [30]; metastatic (65) and non-metastatic (22) patients.
Expression values in this study are the log2-ratios between the treated channel and the reference channel (M-values).
The normalization methods we investigated were the four possible combinations of two dye-normalization methods, global MA-loess [31] and print-tip (local) MA-loess [32], and two approaches to background correction, local correction [33] and no background correction. In addition, we also used non-normalized data (i.e. raw data):
No 0: Raw data (no normalization)
No 1: Print-tip (local) MA-loess, no background correction
No 2: Print-tip (local) MA-loess, background correction
No 3: Global MA-loess, no background correction
No 4: Global MA-loess, background correction
No 1 and 2 were not performed for the data sets Galland and Finak, since print-tip normalization is not relevant for the Agilent microarrays.
Filtration and missing value imputation
In each of the seven data sets there were spots flagged by the scanner or the experimentalist, and some spots also had a lower signal than the background (in the cases where background correction was adopted). These were all marked as missing values. A discussion on how to treat flagged spots can be found in [34]. We chose to include samples with less than 50% missing values and genes with less than 30% missing values. Samples and genes were filtered simultaneously and the remaining missing values were imputed [35], [36]. There are several approaches to dealing with missing values; ROW imputation replace missing value with the row median (the median of that gene), while SVD (singular value decomposition) imputation uses a linear combination of the k most significant eigengenes to estimate missing values [37,38]. According to [7], these two approaches do not significantly affect the result of clustering analysis, and since we study the same data sets, we only use the SVD imputation in our study. We use the function svdImpute in the package pcaMethods to impute missing values.
In addition to the above described filtration and missing value imputation, we also remove some samples with incomplete annotations (i.e. class information). To avoid including duplicate genes we choose to compute the mean value of these.
Cross validation and model evaluation
In this study we perform double cross validations; a 10-fold inner CV was used to optimize parameters in the methods and a 5-fold outer CV was used to estimate the final classification performance (see for example [38] for more on CV). The models were trained on the training data and evaluated on test data. The data partitioning was done so that the number of distinct patients in each test set differed by at most one patient. However, in data sets with several samples from the same patient (Ye, Alizadeh and Finak), all samples belonging to the same patient were always placed in the same test set, thus the number of samples per test set sometimes differed by more than one.
We used error rate, i.e. the percentage of misclassified observations in a test set, as a measure of classification performance. The error rates were adjusted by dividing by the theoretical error rate obtained by randomly assigning classes given the distribution of the two classes. Thus adjusted error rates below one correspond to models performing better than random class assignment. We performed the outer CV with 10 different random partitions into training and test sets. Thus the error rate of each combination of methods was estimated from 50 different test sets (10 5-fold CVs).
Gene selection
In order to reduce the number of genes used by the machine learning methods, we always performed gene selection. There are several different types of gene selection techniques [8], but we decided to use three supervised approaches in this study:
T-test: A two sided, two sample t-test was used [37,39] and genes with the lowest p-values were chosen for further analysis. We used the function t.test from the package stats in R.
Relief: The Relief-algorithm assigns a value to each gene based on how well it separates nearest neighbors with different class labels [40]. We implemented a modified version of the function relief from the package dprep in R. The algorithm was modified so that all observations are chosen once rather than being sampled with replacement.
Paired distance: This is, to the best of our knowledge, a new gene selection approach based on investigating pairs of genes. We first chose the 1000 genes with the highest variance and the 1000 genes with the highest absolute mean value, and retained the union of these genes. For each of the chosen genes we then calculated two medians, one for those observations belonging to the first class and one for those observations belonging to the second class. In the two dimensional space spanned by the gene-pairs, we then calculated the Euclidian distance between the two corresponding pairs of medians. Finally, we ranked pairs of genes according to these distances.
The Decision trees (see Machine learning methods) also used an embedded technique for gene selection. Initially, we also used variance as an unsupervised gene selection method, but this strategy was later dropped due to its inferior performance.
Given the ranked lists from the gene selection methods, we selected 21 different numbers of genes to be used for inducing machine learning models (see Table 1). Some machine learning methods could not be run with all numbers of genes. In particular, SVMs could not handle only two genes in a satisfactory way, while NNs were not run with more than 150 genes for "one hidden layer" and 900 genes for "no hidden layer" due to computational costs.
Machine learning methods
We use three different supervised machine learning methods; Support vector machines (SVMs), Artificial neural networks (NNs) and Decision trees (DTs). Most machine learning methods have several parameters that need to be set before inducing the model. To avoid overfitting the models, some of the most important parameters were optimized using an inner cross validation. Parameters not specifically mentioned were set to default values.
Support vector machines, SVMs
The idea behind SVMs is to use an implicit kernel function to map the data into an n dimensional space where the classes are separated by a hyperplane constructed to create the largest possible margin between the classes [38,41]. We use the function ksvm from the package kernlab in R to build SVM models (see [42] for a comparison of different SVM-packages in R). The data were not standardized, i.e. scaled to zero mean and unit variance (default in the ksvm function), before machine learning methods were applied. After some experiments with the different types of SVM classifications (C-svc, C-bsvc and nu-scv) we decided to use nu-scv for all the SVM models. We also experimented with different kernels and decided to use the following four kernels; linear, polynomial with degree 2, polynomial with degree 3 and radial basis kernel. The parameter nu was set to 0.2, which is the default value in ksvm. The inverse kernel width, sigma, for the radial basis kernel function was optimized using a grid search by choosing the value with the lowest cross validation error rate obtained from the ksvm function.
Artificial neural networks, NNs
NNs consists of connected units (neurons) that transform the input values to an output value based on a threshold function applied to the weighted sum of the input values [38,43,44]. We use two types of NNs; one with one hidden layer and one with no hidden layer (i.e. a perceptron). The number of units in the hidden layer was optimized in the inner cross validation (allowed values were 2, 3, 4 and 5). We use the function nnet from the package nnet in R to build NN models [45]. We set the maximum number of iterations to 100 and the value of rang to 0.1. After experimenting with different decay values we choose the value 0.001. Due to computational costs, models were only built from a maximum of 150 genes for one hidden layer and 900 genes for no hidden layer.
Decision trees, DTs
DTs are built by iteratively splitting the data using the most separating gene, thus forming a tree with nodes and leaves [38,43,44]. We used two different splitting criteria; information gain index and Gini index [46]. We used the function rpart from the package rpart in R to build the DT models. The trees were pruned by choosing the number of splits from the tree with the lowest cross-validated error [45].
Including settings (such as kernels in SVMs), we investigated the performance of the following eight machine learning methods:
DT Gini: Decision tree with Gini index as splitting criteria
DT Information: Decision tree with Information index as splitting criteria
NN One layer: Artificial Neural Network with one hidden layer
NN No layer: Artificial Neural Network with no hidden layer
SVM Linear: Support Vector Machine with a linear kernel
SVM Poly2: Support Vector Machine with a polynomial kernel of degree 2
SVM Poly3: Support Vector Machine with a polynomial kernel of degree 3
SVM Rb: Support Vector Machine with a radial basis kernel
Analysis of results
The possible combinations of the different methods described above gave in total 14685 models with calculated error rates and corresponding standard deviations. To analyze these results, we used multiple linear regression models with error rate as response and Data set (S), Normalization (No), Gene selection method (G), Number of selected genes (N) and Machine learning method (M) (see Table 1) as predicting variables. We also included second order interaction terms in the regression models. When examining the data sets individually we removed Data set as predicting variable. We used the functions lm and anova (with default values of parameters) in the package stats in R to conduct the regression analyses. Test for significant predicting variables was done using a Bonferroni corrected threshold (0.05 divided by the number of variables: 10).
We used the Wilcoxon signed rank test to determine whether there was any significant difference between error rates obtained by different individual methods (Figure 7) and between pairs of methods (Figure 8, 9, 10). The tests were performed using the function wilcox.test in R. Significance was determined by a Bonferroni corrected threshold for individual methods and a threshold of 0.05 for pairs of methods and synergistic relationships. Entries (i,j) in the heat maps equal the the number of data sets where method (or pair of methods) i is significantly better than method (or pair of methods) j. The number of significant data sets for combinations including No 1 or No 2 was adjusted by multiplying by 7/5 since these methods were not applied to two data sets. The heat map was created with the function levelplot in the package lattice in R.
Authors' contributions
JÖ was involved in developing the ideas presented in this article, implemented and conducted the machine learning, analyzed and interpreted the results and drafted the manuscript. EF collected the data sets, implemented filtration and missing value imputation and contributed in the general discussions. ML implemented and conducted the normalizations of the data. PR was involved in developing the ideas presented in this paper and interpreted the results. TRH was involved in developing the ideas presented in this article, analyzed and interpreted the results and wrote parts of the manuscript. All authors read and approved the final manuscript.
TRH and JÖ are funded by the Swedish Research Council (VR) and The Swedish Governmental Agency for Innovation Systems (VINNOVA) through the UPSC Berzelii Centre for Forest Biotechnology.
Fahlén J, Landfors M, Freyhult E, Trygg J, Hvidsten TR, Rydén P: Bioinformatic strategies for cDNA-microarray data processing. In Batch Effects and Noise in Microarray Experiments: Sources and Solutions. Edited by Scherer A. John Wiley & Sons; 2009.
Ritchie ME, Silver J, Oshlack A, Holmes M, Diyagama D, Holloway A, Smyth GK: A comparison of background correction methods for two-colour microarrays.
Bioinformatics 2007, 23(20):2700-2707. PubMed Abstract | Publisher Full Text
Quackenbush J: Microarray data normalization and transformation.
Nat Genet 2002, 32(Suppl):496-501. PubMed Abstract | Publisher Full Text
Ryden P, Andersson H, Landfors M, Naslund L, Hartmanova B, Noppa L, Sjostedt A: Evaluation of microarray data normalization procedures using spike-in experiments.
BMC Bioinformatics 2006, 7:300. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text
Qin LX, Kerr KF: Empirical evaluation of data transformations and ranking statistics for microarray analysis.
Nucleic Acids Res 2004, 32(18):5471-5479. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Kim SY, Lee JW, Bae JS: Effect of data normalization on fuzzy clustering of DNA microarray data.
BMC Bioinformatics 2006, 7:134. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text
Freyhult E, Landfors M, Onskog J, Hvidsten TR, Ryden P: Challenges in microarray class discovery: a comprehensive examination of normalization, gene selection and clustering.
Saeys Y, Inza I, Larranaga P: A review of feature selection techniques in bioinformatics.
Bioinformatics 2007, 23(19):2507-2517. PubMed Abstract | Publisher Full Text
Duval B, Hao JK: Advances in metaheuristics for gene selection and classification of microarray data.
Larranaga P, Calvo B, Santana R, Bielza C, Galdiano J, Inza I, Lozano JA, Armananzas R, Santafe G, Perez A, et al.: Machine learning in bioinformatics.
Brief Bioinform 2006, 7(1):86-112. PubMed Abstract | Publisher Full Text
Pirooznia M, Yang JY, Yang MQ, Deng Y: A comparative study of different machine learning methods on microarray gene expression data.
BMC Genomics 2008, 9(Suppl 1):S13. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text
Romualdi C, Campanaro S, Campagna D, Celegato B, Cannata N, Toppo S, Valle G, Lanfranchi G: Pattern recognition in gene expression profiling using DNA array: a comparative study of different statistical methods applied to cancer classification.
Hum Mol Genet 2003, 12(8):823-836. PubMed Abstract | Publisher Full Text
Lee JW, Lee JB, Park M, Song SH: An extensive comparison of recent classification tools applied to microarray data.
Computational Statistics & Data Analysis 2005, 48(4):869-885. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Li T, Zhang C, Ogihara M: A comparative study of feature selection and multiclass classification methods for tissue classification based on gene expression.
Bioinformatics 2004, 20(15):2429-2437. PubMed Abstract | Publisher Full Text
Statnikov A, Aliferis CF, Tsamardinos I, Hardin D, Levy S: A comprehensive evaluation of multicategory classification methods for microarray gene expression cancer diagnosis.
Bioinformatics 2005, 21(5):631-643. PubMed Abstract | Publisher Full Text
Isaksson A, Wallman M, Goransson H, Gustafsson M: Cross-validation and bootstrapping are unreliable in small sample classification.
Pattern Recognition Letters 2008, 29(14):1960-1965. Publisher Full Text
Jin J: Impossibility of successful classification when useful features are rare and weak.
Proc Natl Acad Sci USA 2009, 106(22):8859-8864. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Poggio T, Rifkin R, Mukherjee S, Niyogi P: General conditions for predictivity in learning theory.
Nature 2004, 428(6981):419-422. PubMed Abstract | Publisher Full Text
Zervakis M, Blazadonakis ME, Tsiliki G, Danilatou V, Tsiknakis M, Kafetzopoulos D: Outcome prediction based on microarray analysis: a critical perspective on methods.
BMC Bioinformatics 2009, 10:53. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text
Montgomery DC, Runger GC, Hubele NF: Engineering statistics. 4th edition. Hoboken, N.J.: Weily; 2007.
Mitchell TM: Machine learning. First edition. Singapore: McGraw-Hill; 1997.
Ambroise C, McLachlan GJ: Selection bias in gene extraction on the basis of microarray gene-expression data.
Proc Natl Acad Sci USA 2002, 99(10):6562-6566. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Dobbin K, Simon R: Comparison of microarray designs for class comparison and class discovery.
Bioinformatics 2002, 18(11):1438-1445. PubMed Abstract | Publisher Full Text
Alizadeh AA, Eisen MB, Davis RE, Ma C, Lossos IS, Rosenwald A, Boldrick JC, Sabet H, Tran T, Yu X, et al.: Distinct types of diffuse large B-cell lymphoma identified by gene expression profiling.
Nature 2000, 403(6769):503-511. PubMed Abstract | Publisher Full Text
Finak G, Sadekova S, Pepin F, Hallett M, Meterissian S, Halwani F, Khetani K, Souleimanova M, Zabolotny B, Omeroglu A, et al.: Gene expression signatures of morphologically normal breast tissue identify basal-like tumors.
Breast Cancer Res 2006, 8(5):R58. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text
Galland F, Lacroix L, Saulnier P, Dessen P, Meduri G, Bernier M, Gaillard S, Guibourdenche J, Fournier T, Evain-Brion D, et al.: Differential gene expression profiles of invasive and non-invasive non-functioning pituitary adenomas based on microarray analysis.
Herschkowitz JI, Simin K, Weigman VJ, Mikaelian I, Usary J, Hu Z, Rasmussen KE, Jones LP, Assefnia S, Chandrasekharan S, et al.: Identification of conserved gene expression features between murine mammary carcinoma models and human breast tumors.
Genome Biol 2007, 8(5):R76. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text
Jones MH, Virtanen C, Honjoh D, Miyoshi T, Satoh Y, Okumura S, Nakagawa K, Nomura H, Ishikawa Y: Two prognostically significant subtypes of high-grade lung neuroendocrine tumours independent of small-cell and large-cell neuroendocrine carcinomas identified by gene expression profiles.
Lancet 2004, 363(9411):775-781. PubMed Abstract | Publisher Full Text
Sorlie T, Perou CM, Tibshirani R, Aas T, Geisler S, Johnsen H, Hastie T, Eisen MB, van de Rijn M, Jeffrey SS, et al.: Gene expression patterns of breast carcinomas distinguish tumor subclasses with clinical implications.
Proc Natl Acad Sci USA 2001, 98(19):10869-10874. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Ye QH, Qin LX, Forgues M, He P, Kim JW, Peng AC, Simon R, Li Y, Robles AI, Chen Y, et al.: Predicting hepatitis B virus-positive metastatic hepatocellular carcinomas using gene expression profiling and supervised machine learning.
Nat Med 2003, 9(4):416-423. PubMed Abstract | Publisher Full Text
Dudoit S, Yang YH, Callow MJ, Speed TP: Statistical methods for identifying differentially expressed genes in replicated cDNA microarray experiments.
Yang YH, Dudoit S, Luu P, Lin DM, Peng V, Ngai J, Speed TP: Normalization for cDNA microarray data: a robust composite method addressing single and multiple slide systematic variation.
Nucleic Acids Res 2002, 30(4):e15. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Eisen MB: ScanAlyze. User manual.
Scherer A, (ed.): Bioinformatic Strategies for cDNA-Microarray Data Processing. John Wiley & Sons, Ltd; 2009.
Aittokallio T: Dealing with missing values in large-scale studies: microarray data imputation and beyond.
Troyanskaya O, Cantor M, Sherlock G, Brown P, Hastie T, Tibshirani R, Botstein D, Altman RB: Missing value estimation methods for DNA microarrays.
Bioinformatics 2001, 17(6):520-525. PubMed Abstract | Publisher Full Text
Theodoridis S, Koutroumbas K: Pattern recognition. San Diego, Calif.: Academic Press; 1999.
Hastie T, Tibshirani R, Friedman JH: The elements of statistical learning: data mining, inference, and prediction. New York: Springer; 2001.
Casella G, Berger RL: Statistical inference. 2nd edition. Pacific Grove, Calif.: Duxbury; 2002.
Rendell KKaLA: A practical approach to feature selection.
Proc 9th International Conference on Machine Learning: 1992 1992, 249-256.
Ben-Hur A, Ong CS, Sonnenburg S, Scholkopf B, Ratsch G: Support vector machines and kernels for computational biology.
PLoS Comput Biol 2008, 4(10):e1000173. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Alexandros Karatzoglou DM, Hornik Kurt: Support Vector Machines in R.
Faraway JJ: Extending the linear model with R: generalized linear, mixed effects and nonparametric regression models. Boca Raton: Chapman & Hall/CRC; 2006.
Faraway JJ: Extending the Linear Model with R. First edition. United State of America: Chapman & Hall/CRC; 2006.
Laura Elena Raileanu KS: Theoretical Comparison between the Gini Index and Information Gain Criteria.
Annals of Mathematics and Artificial Intelligence 2004, 41(1):77-93.


text:
Friday, September 28, 2012
Rian Johnson - Brick Q&A
Download my podcast here
Copyright © Unlikely Films, Inc. 2012. All rights reserved.
3 comments:
Telltale signs on the watch will acquaint if you bought a fake.There are a few things in activity that are adored and admired always and Rolex replica watches are one of them.Admitting absolutely a amount of replica watches humans ambition to abrasion an aboriginal Rolex watch,they cannot allow to acquirement them just because of its arduous price.Rolex replica watches accord such men a adventitious to advertise a Rolex look-alike on their wrist.Do not be bamboozled by the name of the watch.Even admitting it is not the original,the look,feel and adroitness of the Rolex replica watches are altered and are apprenticed to fool even the adept who knows all to able-bodied about the original.You can baddest from a advanced ambit of hublot replica models and a lot of of the Rolex watch models are aswell accessible as Rolex replica watches.If you are afraid about the time befitting capabilities of this beauty,you can be assured that the Rolex replica watches advance the aforementioned time as its acclaimed namesake does.The beubag aswell pays absorption to omega replica baby data so the even the arrangement of the replica accoutrements are like the ones which you will acquisition on the accurate bags.Thus,you can be abiding of what you buy from this online store.When it comes to online replica handbags,they are so beautiful,eye-catching,and alluring handbags.Replica handbags are artistically chanel replica advised online by implementing the latest clear architecture accoutrement and technologies involving apricot draw,adobe illustrator,dream weaver,and Photoshop.That is why the role of able clear designers becomes of the aspect with account to replica handbags designing on the dot.The attending of replica handbags is beauteous while the colors of replica handbags are activated all the time.ReplyDelete
Omega is an affluence watchmaker based out ofReplyDelete
louis vuitton replica Switzerland whose on the internet autograph are world-renowned for their adorableness and aloft top quality. Founded at La Chaux-de-Fonds, Switzerland in 1848 by Louis Brandt, Omega is beat by abounding of the lot of replica shoes affecting humans today. Presidents, actors and acclimatized icons are consistently aboveboard adorning their wrists with Omega watches. Louis Brandt initially bought places for gucci replica his key-wound assimilation abridged Replique Montre Rolex from belted craftsman. Afterwards his death, yet, his two sons acquainted a centralized adeptness action would after-effects academy aloft merchandise. This would serve as the alpha of Omega's prominence. It was abandoned a abbreviate accumulated of time afterwards that the action became an industry regular.Each of Brandt's sons, Louis-Paul and C?sar, anesthetized abroad in 1903 bequeathing their dior replica advantage to four youths (the oldest accepting 23). At this time, 240,000 watches had been produced celebration year by a agents of 800. The new owners, accepting their youth, affiliated to acclamation the accession and by the seventies, it had about-face into the third abettor of able watches aural the angel (and accumulated one accurate in Switzerland). Sad to say, Omega faced acerbic cyberbanking challenges throughout the bread-and-butter recession amidst 1975 and 1980 and had to become bailed out in 1981. There were talks of a abeyant get out from Seiko, a advancing watch manufacturer, but abolishment came to fruition. Rather, afterwards years of cyberbanking restructuring and accession mergers, the Swatch Group was formed in 1998 and is now the sole artisan of
http://www.enjoywarwickshire.co.uk and added watch brands.


text:
Demo price: CND$3550. New: $4550. Save up to $2000! Our 2015 fleet of Santa Cruz Bronsons come with a 5 year frame warranty from Santa Cruz. We have small, Medium & Large sizes.
We've changed our location to feature the South End of town.
Our 2015 fleet of Santa Cruz Bronsons and Nomads are on the way for our summer touring and demo season. If you're looking for the best ride, we have it.


text:
Here at Out of Ur we've been hosting a conversation about the themes found in Shane Claiborne's latest book, Jesus for President (part one and part two). As is evident from this conversation, Shane is a guy who provokes a response in those he encounters. Certainly those at the Shift conference who just heard Shane speak about The Scandal of Grace got a taste of this.
Before proceeding, let me tell you how hard it is to summarize Shane Claiborne. The guy is a non-stop storyteller! Stories about growing up in Tennessee attending youth group. Stories about his home in the rough neighborhoods of Philadelphia. Stories about going to Iraq on the eve of the bombing of Baghdad. On top of his stories, Shane quotes incessantly: Mother Theresa, Martin Luther King JR, and Dostoevsky among others. Consider this a plea to check this post in a couple hours when we can post some video of this session.
Update. Here are some video highlights from this session.
In addition to writing and speaking, Shane is a member of the Simple Way, a Christian community in Philadelphia. It is obvious his experience with this community (identified by some as New Monasticism) has deeply impacted how he understands the role of the church in America. Citing the research found in Unchristian, Shane told this room of youth leaders that those outside the church see them primarily as anti-gay, judgmental, and hypocritical. Against this discouraging research, Shane summarized Jesus' reply to John the Baptist's question in Luke 7. "Go back and report to John what you have seen and heard: The blind receive sight, the lame walk, those who have leprosy are cured, the deaf hear, the dead are raised, and the good news is preached to the poor."
And what might it ...
Or search by:
Or search by:
Get helpful articles on:


text:
Kershaw Knives
Kershaw Fillet Knife
Free Shipping Eligible! Up to $15 Shipping Credit.
Kershaw Knives Lifter BlackWash Tanto Knife
$24.99 $39.99*
Save: $15.00 (38%)
Free Shipping Eligible! Up to $15 Shipping Credit.
Kershaw Knives Shuffle Drop Point Knife
Free Shipping Eligible! Up to $15 Shipping Credit.
Learn More About Kershaw Knives
Product Selection
Built For The Field: Kershaw Hunting Knives
Every hunter needs good hunting knives. Depend on Kershaw fixed blade and folding blade knives to deliver lightweight and durable performance in the field.
Focus on the blade design that works best for your hunting needs. From drop point to a knife with several interchangeable blades, choose the knife best suited for the chores its intended to handle.
Decide what size of knife you need. If you need a knife only for cleaning game like birds, consider a smaller knife thats light and fits into your pocket. If youre hunting elk, something bigger and sturdier is more useful. Regardless of the size, make sure the Kershaw knife you select fits comfortably in your hand.
Consider the best type of blade material for your hunting knife. This determines how well you can sharpen your knife and how well it will hold its edge after your sharpen it.
Find hunting knives with value that extend beyond their usefulness in the field. Kershaw knives are keepsakes that can be passed down from one generation to another.


text:
Book your room at Hummingbird Beach Resort
Traditional Room, Garden ViewMax Occupancy: 4 guests (up to 3 children, 1 infant)
Deluxe RoomMax Occupancy: 2 guests (up to 1 child, 1 infant)
Suite, 1 BedroomMax Occupancy: 3 guests (up to 2 children, 1 infant)
Suite, 2 Bedrooms, Hill ViewMax Occupancy: 4 guests (up to 3 children, 2 infants)
Get an overview of this hotelFamily fun on the beach, near Anse Chastanet Beach
With direct access to the beach, this Soufriere resort is within 3 mi (5 km) of Church of the Assumption, Anse Chastanet Beach, and Soufriere Volcano. Sulphur Springs is 2.5 mi (4 km) away.
Resort Features
This resort features a restaurant, an outdoor pool, and a bar/lounge. WiFi in public areas is free. Other amenities include tour/ticket assistance and wedding services.
Room Amenities
All 10 rooms offer free WiFi, sitting areas, and TVs with cable channels.Information missing or incorrect? Tell us!
Hotel Amenities
Hotel Amenities
The resort offers a restaurant. A bar/lounge is on site where guests can unwind with a drink. Public areas are equipped with complimentary high-speed wireless Internet access. This beach resort also offers an outdoor pool and tour/ticket assistance.
- Bar/lounge
- Free parking nearby
- Free WiFi
- Outdoor pool
- Restaurant
- Total number of rooms - 10
- Tours/ticket assistance
- Wedding services
Family Friendly Amenities
- Cribs/infant beds (complimentary)
- Free Wi-Fi
- Outdoor pool
- Rollaway/extra beds available
Available in all rooms: Free WiFi
Available in some public areas: Free WiFi
Free parking nearby
Room Amenities
- Air conditioning
- Cable TV service
- Free cribs/infant beds
- Free WiFi
- Phone
- Private bathroom
- Rollaway/extra beds available
- Separate sitting area
- Shower/tub combination
- Television
Where to Eat
Hummingbird Beach Resort has a restaurant on site.
Hotel Policies
Check-in time starts at 2 PM
Check-out time is noon
Payment types
Children and extra beds
- Children are welcome.
- Rollaway/extra beds are available.
- Free cribs (infant beds)!
You need to know
Extra-person charges may apply and vary depending on hotel policy.
Government-issued photo identification and a credit card or cash deposit are required at check-in for incidental charges.
Special requests are subject to availability upon check-in and may incur additional charges. Special requests cannot be guaranteed.


text:
Searched for: "denial for price discount"
Content Types
About 2 results for "denial for price discount"
Network security for the small office
You want to protect your small office network from malicious attacks, but your security budget is nonexistent. Maybe you should turn to ZoneAlarm 2.1 to safeguard your data.
Site-ings: Insurance for the home-based business
If you own a small, at-home company, you know to protect yourself from viruses and hard-drive crashes. But what about hazards like fire, theft, and personal illness? Here are some ...


text:
376 results
SORT BY Relevancy
Black Thoughts and Theory: The Radio Show- Gun Control & Historical Racism
in Culture
Gun Control & Historical Racism
As Black Americans we must realize that gun control laws, from a historical perspective, serve one purpose, to keep us vulnerable and more easily oppressed. This is not conjecture or rhetoric, it is easily documented fact.
Join us, Alex Primo and Asha Zuri, Thursday October 8th at 9 PM EST as we explore the way America sought to disarm Melanated individuals across the land as part of a ruthless plot to control, intimidate, and in many cases murder innocent slaves and freed men alike.
Brendan O'Neill of the Guardian notes:
Throughout the 1800's, states passed gun-control laws that were fundamentally racist. So, panicked by the prospect of more black rebellions against white landowners, the North Carolina Supreme Court passed a statute in 1840 that said: "If any free negro, mulatto, or free person of color shall wear or carry about his or her person, or keep in his or her house, any shotgun, musket, rifle, pistol, sword, dagger or bowie-knife… he or she shall be guilty of a misdemeanor, and may be indicted therefore."
In 1941, a judge in Florida's Supreme Court called the laws into question when he overturned the conviction of a black man for carrying a handgun without a permit. He overturned the conviction, he said, because this law "was passed for the purpose of disarming the negro laborers … and to give the white citizens in sparsely settled areas a better feeling of security. The statute was never intended to be applied to the white population and in practice has never been so applied."
All of this plus brother Dennis Jefferson will be dropping by with another fantastic episode of Healthy Matters.
Call in to participate at (917)889-2830
Log on here to just listen to the show.
And Remember, Always Keep It Real and Keep It On Point. #One
Michigan's Safe Delivery Program, a discussion with Jean Hoffman
in Family
Jean Hoffman is one of Michigan's Safe Delivery Consultants and was essential in the implementation of the Safe Delivery Statute in 2000. She has been responsible for increasing public awareness and creating an infrastructure to insure it's sustainability. Learn how adoption works alongside this law and how you can promote it in your community. Don't miss this informative discussion!
how obama is working for the queen part 1
in Education
A Genius and Legacy of Robert Welch's Insights - Two Speeches At the JBS Council Dinner in Los Angeles, March 9th, 1974
John Birch Society Founder, Robert Welch in his first speech, he quotes himself from 1958 at the JBS Founding Meeting About Insiders Knowledge and Long-Range Plans For America.
His Second Speech, Which Took Place at the 1974 JBS Council Dinner, Mr. Welch Looks 15 Years Into the Future of America.
Standard YouTube LicenseTHE SCARIEST MOVIE EVER MADE | KJ Ozborne https://youtu.be/fXtAMso1d2s via @YouTube
in News
As a courtesy, we note that on the spreadsheet there is one open matter that involves you (Carmen Sabatino v. City of Modesto, Case No. 620253). It is our understanding that Meyers Nave successfully defended the City in a lawsuit you filed challenging the City's conduct related to the Charter amendment to create Council districts. In the course of litigation, you filed an amended complaint. The City instructed Meyers Nave to file a motion to strike the amended complaint pursuant to California's anti-SLAPP statute. The City prevailed in its anti-SLAPP action, and was awarded all of the attorney's fees it sought in its motion for attorney's fees relating to the anti-SLAPP action. This matter is still open because the City has not received payment.
in History
Myron Fagan exposes the Illuminati | CFR [1967] ../
Published on Apr 12, 2015
Copyright Disclaimer Under Section 107 of the Copyright Act 1976, allowance is made for "fair use" for purposes such as criticism, comment, news reporting, teaching, scholarship, and research. Fair use is a use permitted by copyright statute that might otherwise be infringing. Non-profit, educational or personal use tips the balance in favor of fair use.
Originally Published on Aug 12, 2013 By "The Crypt" Youtube Channel
Secret society known as The Illuminati, and their New World Order.
Mr. Fagan describes with documentary evidence how the ILLUMINATI became the instrument of the House of Rothschild to achieve a "One World Government".
Mr. Fagan relentlessly uncovered plots for major historical events. Myron Fagan is considered to be the archetype for the PCT (Paranoid Conspiracy Theorist)
This Recording is one of the most interesting and yet horrifying, factual stories of some of the most sensational plots in the history of the world.
Statute of Limitations~ Lee Parker shares His Story and His Frustration
in Self Help
Lee Parker's life changed forever at the age of 12. While under the care of Child and Youth Services Lee was placed in a group home where he would meet his abuser.
His abuser was a staff member who he got along with very well.
Lee was allowed to go home with this staff member on house visits and then overnight visits.
His abuser became his foster father and a month later began sexually abusing Lee for the next year and a half. Lee would be at school one day visibly upset and not wanting to go "home." A school nurse would spend the time to hear his story and intervene.
Now at the age of 31, finally coming to terms with what happened to him, he is finding the statutes of limitations is working against him.
Join us in a conversation about the frustrations survivors confront concerning a crime that they can't prosecute because of statute of limitations that prohibit it. In many state the rape of an adult has no limitation. The rape of a child is defined differently and we begin to see many differences. If you follow the link below you begin to see that while on the surface it will appear that there is none, a real look show the discrepancies.
TILA RESCISSION Battleground: Patricia Rodriguez Guest
in Finance
LA Lawyer Patricia Rodriguez (TILA Rescission is boiling over the top. And there are differences between non-judicial states and judicial states. The main problem though is that desperate homeowners are hearing something different than the real message: rescission is as close to a magic bullet as we have seen but it is still susceptible to multiple issues in litigation. Rescission is certainly effective upon mailing and that appears to be true even if the notice is wrongfully sent. The burden of vacating the rescission is on the so-called lender. But there are still circumstances where the notice of rescission itself may well render it void even if mailed. Admitting that the transaction was consummated on a specific date where the rescission letter itself has both the consumamtion date and the date of mailing might encourage judges to strike down the rescission or vacate it because it is void on tis face if the notice was sent after the three year period. It is true that the statute of limitations is an affirmative defense and that the ability to raise an affirmative defense depends upon your standing to raise the issue. Patricia Rodriguez agrees that the argument could still be made that the rescission is effective but the more information on the recission, the more likely the Judge will find the rescission void on its face even if mailed.
The Ucc Show With Dave McDowell
in Education
Learn the maze of the UCC with Dave.
(1) "assigned estate" means all the real and personal estate of an assigning debtor passing to the consenting creditors under an assignment by virtue of Section 23.02 or 23.09(b) of this code;
Sec. 23.02. NATURE AND EFFECT OF ASSIGNMENT. (a) A debtor may assign his real and personal estate under this chapter to an assignee for the benefit of the debtor's creditors.
ll provide in the as(b) An assigning debtor shasignmentfor distribution of all his real and personal estate to each consenting creditor in proportion to each consenting creditor's claim.
(c) The assigning debtor shall sign the inventory required by Subsection (b) of this section and swear that it is just and true.
A) authenticated by a secured party;
(7) "Authenticate" means:
(A) to sign; or
(B) with present intent to adopt or accept a record, to attach to or logically associate with the record an electronic sound, symbol, or process.
10) "Certificate of title" means a certificate of title with respect to which a statute provides for the security interest in question to be indicated on the certificate as a condition or result of the security interest's obtaining priority over the rights of a lien creditor with respect to the collateral.
(12) "Collateral" means the property subject to a security interest or agricultural lien.
How Late Can a Consumer Rescind? Jesinoski Revisited
in Finance
The parties were invited to attend a meeting at which the borrowers signed the papers. The originator is usually not in attendance. The closing agent, usually a title agent, takes the signed documents into escrow. At some later time the alleged loan is "funded" by a wire transfer from an unknown source. The borrower believes that the originator is loaning him the money. In truth it was a "table-funded" loan in which the name of the actual lender was actively concealed from the borrower and the actual "loan" was not funded until some time after the meeting. Later the mortgage is recorded and the note is released from escrow and sent to someone whom the closing agent assumes to be authorized to receive it. When was consummation?
Later the successors to the loan documents will allege that the loan closing was "consummated" at the time of signing, but that is not right. If there was a consummation of the loan at all it didn't occur until hours, days, weeks or months or even years after the meeting. It is a question of fact as to when the alleged loan was consummated. And if there was a table-funded loan there might have been no consummation at all. Instead whoever the real lender is might have some claim for quantum meruit or unjust enrichment to get their money back but if they raise that claim they cannot do it using the note and mortgage. So when does the three day period commence for rescission? When does the three year period begin for rescission? Is there a time when the borrower is prohibited by statue from sending the notice of rescission? And what happens if some company steps up and complies and then asks for the money? They don't have to object to rescission --- they can agree. If rescission is effective when mailed, who can vacate it?
Rastafari SabbatHical Studies - RSS #39 - OVERDRIVE Portion
in Spirituality
| yeH'ggu t'Izaz [y'h'new] | Chukat | "Decree of / Statute"
Torah Reading
Numbers 19:1-22:1
Prophets Reading
Judges 11:1-33
New Testament
Hebrews 9:11-28; John 3:10-21
Rastafari SabbatHical Studies - RSS #39 - Parsha Overview Gathering
in Spirituality
| yeH'ggu t'Izaz [y'h'new] | Chukat | "Decree of / Statute"
Torah Reading
Numbers 19:1-22:1
Prophets Reading
Judges 11:1-33
New Testament
Hebrews 9:11-28; John 3:10-21
Real Men Don't? Feminization Of The Black Man In Amerikkka.
in Motivation
The social oppression and castration of black men is rooted in race and gender. Intersectional theories that explicitly or implicitly suggest that black men are privileged by gender are thus flawed. Black men, instead, are also the victims of "gendered racism."
Indeed, there is a gender analysis implicit in the notion that white supremacy "castrates" black
men – castration itself is an act of gendered racism. Historically, blackmen have thus been targeted for certain types of treatment – including castration – because they are both black and male. Therefore, although men constitute the dominant and privileged group within American society, black men convey a "subordinated masculinity." Black masculinity
as a subordinated form of masculinity arises because the interplay between racial and socioeconomic prejudices prevents black males, as individuals, from reaping the full benefits of male class privilege.
The legal system, moreover, has served as a primary instrument of oppression, carving out the racialized sphere of subordinated masculinity. According to critical race theorists, the law does not merely reflect,
mediate, and arbitrate preexisting race relations. Rather, the law "constitutes, constructs and produces races and race relations in a way that supports" white racial power and subordination. The transformative powerof law in shaping race relations is reflected in a 1697 Pennsylvania statute that imposed the penalty of castration for a black man who attempted
- ...
Today's Best Podcasts
Join Host Live Chats
There are no live chats in progress


text:
Is your resume in need of a little disruption?
Startups are all about
making things work better.
Our resume writing service is all about doing that for your CV.
StartUpHire is the place to find Director-of-Performance jobs in and near . Review the list below of Director-of-Performance jobs. These Director-of-Performance jobs represent some of the most rewarding careers available. You can refine your job search for Director-of-Performance jobs in by adding a job title, company name, or keyword to the search box above … or click the "Advanced Search" link for more options. Don't see the employment opportunity you're looking for? Check back soon for more Director-of-Performance jobs in , or sign up for an email alert (link in upper left) to receive periodic updates on new Director-of-Performance jobs in , .
JOB SUMMARY As a Sr. Firmware Developer, you will be a vital part of our fast growing company. You will be a key member of our Smart Lighting development team based in our San Francisco office and will report directly to the Engineering Director of Smart Lighting. The ideal9/24/2015 San Francisco, CA Save This Job
(Multiple states) Full Time Product Development Mid Level Founded in 1998, Legacy.com is an innovative technology company that is the digital leader in the memorial and obituary market. Legacy.com collaborates with more than 1,500 newspaper and funeral home clients in9/23/2015 Calabasas, CA Save This Job
Location: Portland, OR, United StatesAPPLY ABOUT JIVE SOFTWARE Jive Software is the leading provider of modern communication and collaboration solutions for business. Our products empower people and organizations to work better together, using technology that adapts to their9/9/2015 Portland, OR Save This Job
The Sales Operations Specialist is responsible for supporting all stages of the sales cycle as a key member of the Calabrio account team. This position will be essential to the sales activities and communications between Calabrio prospects, current customers and channel partners8/29/2015 Minneapolis, MN Save This Job
Company Overview Relypsa, Inc. is a late clinical-stage pharmaceutical company leading the discovery and development of novel non-absorbed polymeric drugs for important applications in cardiovascular and renal diseases. Relypsa's lead product candidate is patiromer, a9/9/2015 Redwood City, CA Save This Job
Human Resources San Francisco, CA, United States About Indiegogo Indiegogo was started with the simple mission to democratize the way ideas get funded and become a reality. Our founders-- Danae Ringelmann, Eric Schell, and Slava Rubin--struggled to find funding for projects9/7/2015 San Francisco, CA Save This Job
About the Job Join a team of ?Grade-A? players who thrive in a dynamic environment of entrepreneurship, creativity, professionalism and a passion to change the landscape of one of the largest industries - online recruitment advertising. RealMatch is looking to hire an Account8/19/2015 New York, NY Save This Job
Description MarkMonitor, the world leader in enterprise brand protection and a Thomson Reuters Intellectual Property & Science business, uses a SaaS delivery model to provide advanced technology and expertise that protects the revenues and reputations of the world's leading10/6/2015 San Francisco, CA Save This Job
At Good Technology, we secure enterprise information and applications on today?s most popular mobile devices?such as iPhone, iPad, and Android?protecting both company and personal data. An exceptional user experience enhances productivity, whenever and wherever people want or8/15/2015 Sunnyvale, CA Save This Job
Job Locations US-NY US-NY Category Product Management Travel .. Level .. More information about this job: Overview: MediaMath is seeking a highly qualified and passionate Product Manager to support our technology platform. This individual will work alongside engineering,9/11/2015 New York, NY Save This Job
Manage and support all project management activities for new and ongoing projects that will lead to the successful delivery of products and services offered by UnitedLex. Initiate and facilitate timely and thorough communication throughout the lifecycle of each project.8/11/2015 Boston, MA Save This Job
Manta?s Product Marketing Analyst reports to the Director of Subscription Products. The successful candidate will have a strong, direct, visible impact on growing Manta?s small business oriented products and services. Come join us in our Columbus, OH8/11/2015 Columbus, OH Save This Job
Hauppauge, New York Job Description All Covered, a division of Konica Minolta Business Solutions is focused on enabling the success of small and medium-sized businesses through the effective use and management of technology. The company is the US industry leader in the provision9/8/2015 Hauppauge, NY Save This Job
ID 2015-3110 Job Locations US-MD-Landover Category Admissions More information about this job: What We're Looking For: The Senior Manager, Enrollment Services will identify and improve the efficiency, effectiveness and productivity of the enrollment services team. This role8/5/2015 Landover, MD Save This Job
At Pandora, we're a unique collection of engineers, musicians, designers, marketers, and world-class sellers with a common goal: to enrich lives by delivering effortless personalized music enjoyment and discovery. Peopleâthe listeners, the artists, and our employeesâare8/26/2015 Oakland, CA Save This Job
JOB SUMMARY As a Sr. Electrical Engineer, you will be a vital part of our fast growing company. You will be a key member of our Smart Lighting development team based in our San Francisco office and will report to the Engineering Director of Smart Lighting. The ideal candidate8/25/2015 San Francisco, CA Save This Job
ID 2015-3053 Job Locations US-MD-Landover Category Operations/Management More information about this job: What We're Looking For: The PES Operations Analyst is responsible for helping analyze and improve the educational outcomes and retention of 2U?s students and the8/5/2015 Landover, MD Save This Job
Alkami is searching for several Senior QA Engineers with a passion for delivering quality releases for our Online Banking Suite and Mobile iOS and Android application suite. As a successful candidate, you will report into Alkami?s Director of QA and be part of a versatile, high9/9/2015 Plano, TX Save This Job
Reports to: Sr. Director of Professional Services Description: As a highly visible and customer-facing position, the Professional Services Solution Engineer works with an exceptional professional services organization, including Implementation Managers, product owners, process8/17/2015 Chicago, IL Save This Job
Job Locations US-IL-Chicago Category Engineering Travel None Level .. More information about this job: Overview: MediaMath's Platform API team builds the services that make up our TerminalOne platform, the technology platform on which our clients build highly successful media8/12/2015 Chicago, IL Save This Job


text:
Healthcare organizations have historically focused their disaster recovery efforts on protecting against unlikely but major events such as fires, floods and natural disasters. But with 24/7 hospital operations, even the smallest of interruptions can have serious consequences to delivering patient care.
As these health systems shift attention to population health management, they are bolstering their leadership ranks with roles such as Chief Population Health Officer (CPHO) and Chief Accountable Care Officer (CACO). With so much at stake, these new executive roles underscore the importance of getting it right and doing it well.
The world of healthcare IT is rapidly changing and CIOs are finding that traditional governance models are not sustainable in the fee for value world.
With HIPAA-compliant hosting available, does running an app from a HIPAA compliant hosting environment make the app itself compliant? Not necessarily. Lets explore why.
The fall is fast approaching and with that brings the reminder that the new ICD-10 compliance deadline of October 1, 2015 is a little over a year away.
Is this medication covered by my patients health insurance plan? This frustrating question is faced many times each day by physicians.
With the transition date finally here, the industry makes last-second preparations.


text:
Emergency Room Coordinator
- Indianapolis, IN
Job Description
We have worked with leading healthcare organizations across the country and throughout the world. We are continually moved and inspired by the clients that we serve. This year alone our clients collectively touched the lives of over 20 million patients, provided over $2.5 billion in charity care, and trained over 18,000 residents. Many are also at the vanguard of basic, translational and clinical research. Our clients invest in hundreds of community health outreach programs, continuously push to raise the level of clinical quality, and strive to be the employer of choice in the communities that they serve. Their steadfast dedication to those they serve encapsulates the reason we feel such great privilege and reward in the work we do. Responsibilities
This position is a skilled individual who has successfully completed the Emergency Department's Course, which validates their areas of responsibilities and patient care duties. The individual is directed by the RN or Physician. Their role is assistive with Emergency Department Services. High School Diploma/GED equivalent required. Experience in an Emergency Department preferred. A minimum of one year experience as Nurses Aide. Shifts include evenings 3:00 pm - 11:15 pm and overnights 11:00 pm - 7:15 am.
Our team is composed of senior professionals with deep and unique healthcare consulting and management experience and includes leaders from management consulting, clinical practice, provider management and industry service. Our consultants are bound together by unparalleled expertise in strategic planning, organizational alignment and operations management, a superior knowledge of process and analytics, and an unrelenting commitment to provide each and every client with meaningful results.