text:
A key problem in understanding transcriptional regulatory networks is deciphering what cis regulatory logic is encoded in gene promoter sequences and how this sequence information maps to expression. A typical computational approach to this problem involves clustering genes by their expression profiles and then searching for overrepresented motifs in the promoter sequences of genes in a cluster. However, genes with similar expression profiles may be controlled by distinct regulatory programs. Moreover, if many gene expression profiles in a data set are highly correlated, as in the case of whole organism developmental time series, it may be difficult to resolve fine-grained clusters in the first place. We present a predictive framework for modeling the natural flow of information, from promoter sequence to expression, to learn cis regulatory motifs and characterize gene expression patterns in developmental time courses. We introduce a cluster-free algorithm based on a graph-regularized version of partial least squares (PLS) regression to learn sequence patterns—represented by graphs of k-mers, or "graph-mers"—that predict gene expression trajectories. Applying the approach to wildtype germline development in Caenorhabditis elegans, we found that the first and second latent PLS factors mapped to expression profiles for oocyte and sperm genes, respectively. We extracted both known and novel motifs from the graph-mers associated to these germline-specific patterns, including novel CG-rich motifs specific to oocyte genes. We found evidence supporting the functional relevance of these putative regulatory elements through analysis of positional bias, motif conservation and in situ gene expression. This study demonstrates that our regression model can learn biologically meaningful latent structure and identify potentially functional motifs from subtle developmental time course expression data.
Author Summary
A major challenge in functional genomics is to decipher the gene regulatory networks operating in multi-cellular organisms, such as the nematode C. elegans. The expression level of a gene is controlled, to a great extent, by regulatory proteins called transcription factors that bind short motifs in the gene's promoter (regulatory region in the non-coding DNA). In a temporal regulatory process, for example in development, the "regulatory logic" of DNA motifs in the promoter largely determines the gene's expression trajectory, as the gene responds over time to changing concentrations of the transcription factors that control it. This study addresses the problem of learning DNA motifs that predict temporal expression profiles, using genomewide expression data from developmental time series in C. elegans. We developed a novel algorithm based on techniques from multivariate regression that sets up a correspondence between sequence patterns and expression trajectories. Sequence motifs are represented as graphs of sequence-similar k-length subsequences called "graph-mers". By applying the method to germline development in C. elegans, we found both known and novel DNA motifs associated with oocyte and sperm genes.
Citation: Li X, Panea C, Wiggins CH, Reinke V, Leslie C (2010) Learning "graph-mer" Motifs that Predict Gene Expression Trajectories in Development. PLoS Comput Biol 6(4): e1000761. https://doi.org/10.1371/journal.pcbi.1000761
Editor: Aviv Regev, Broad Institute of MIT and Harvard, United States of America
Received: June 22, 2009; Accepted: March 24, 2010; Published: April 29, 2010
Copyright: © 2010 Li et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Funding: We would like to acknowledge the funding from NSF grant IIS-0705580 and NIH NCBC grant U54-CA121852 to Columbia University. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
The mRNA expression level of a gene is regulated by multiple input signals that are integrated by the cis regulatory logic encoded in the gene's promoter. Genes whose regulatory sequences contain similar DNA motifs are likely to have correlated expression profiles across a given set of experimental conditions. The converse, however, is not necessarily true. That is, genes can have correlated expression profiles without being coregulated, since multiple regulatory programs may lead to similar patterns of differential expression. This is particularly evident in developmental time series data, in which the genes exhibit only a few distinct expression patterns. Nevertheless, computational approaches for deciphering gene regulatory networks from gene expression and promoter sequence data often do assume that correlation implies coregulation. For example, a typical computational strategy is to cluster genes by their expression profiles and then apply motif discovery algorithms to the promoter sequences for each cluster. The cluster-first motif discovery approach is indeed so prevalent that the best-known benchmarking study of motif discovery algorithms [1] defines the problem in precisely this way – namely, given a cluster of genes, find the overrepresented motif(s) in the promoter sequences – and compares numerous such algorithms. It is clear, however, that assigning genes to static clusters that are assumed to be coregulated oversimplifies the biology of transcriptional regulation. Moreover, in a setting where there are few experiments probing the conditions of interest or where many genes have synchronized expression profiles, such as in a time course, clustering may fail to resolve meaningful gene sets for subsequent motif analysis.
In the current work, we present an algorithm that models the natural flow of information, from sequence to expression, to learn cis regulatory motifs and to characterize gene expression patterns. Our algorithm learns motifs that help to predict the full expression profiles of genes over a set of experiments, with no clustering. More precisely, we use a novel algorithm based on partial least squares (PLS) regression to learn a mapping from the set of -mers in a promoter to the expression profile of the gene across experiments; in time series, we learn -mers that help to predict the full expression time course for genes. PLS combines dimensionality reduction and regression; it iteratively finds latent factors in the input space with maximal covariance with projections in the output space. We introduce a graph-regularized version of the PLS algorithm to enable motif discovery by imposing two constraints: a lasso [2] constraint for sparsity and a graph Laplacian constraint for smoothness over sequence-similar motifs. Our novel graph-regularized PLS algorithm can be used in any situation where the input features are related by a graph structure. Here, the graph structure is defined on the feature space of -mers, with edges connecting pairs of similar -mers. Our approach is motivated by recent machine learning work that uses the graph Laplacian to exploit graph structure in various ways, for example, by defining a graph over training examples in semi-supervised classification (Laplacian SVM [3]) and clustering (spectral clustering [4]) as well as imposing graph smoothness on features of an SVM classifier [5].
Our focus in this study is discovering regulatory elements and deciphering transcriptional regulation in the nematode Caenorhabditis elegans, a key model organism in developmental biology. In particular, we are interested in using mRNA profiling experiments from developmental time courses, where the high global level of correlation presents a challenge to clustering. Dissection of gene regulatory logic is not as advanced in C. elegans as it is in D. melanogaster, for example. There are few motif discovery programs designed specifically for worms, and while worm biologists do use generic programs such as MEME [6], traditionally they have relied on experimental strategies to define binding motifs and then performed genome-wide motif searches and validation with transgene reporters. One goal of our work is to advance this area of inquiry by defining novel elements and providing new opportunities for directed experimental validation.
As a demonstration of our method, we applied our graph-regularized PLS algorithm to an expression time course for wildtype germline development in C. elegans [7]. We found that the first and second PLS latent factors mapped to expression profiles for oocyte and sperm genes, respectively. In each iteration of our approach, we learn sequence information in the form of a "graph-mer", i.e. a graph where vertices are -mers, weighted by their contribution to the latent factor, and edges join -mers that are close in Hamming distance. To parse the motif graphs into component motifs, we applied a graph module discovery algorithm followed by hierarchical agglomeration to produce position specific scoring matrices (PSSMs) from the weighted -mers. Applying this procedure to the significant latent factors generated a collection of known and novel oocyte- and sperm-specific motifs, including novel CG-rich motifs associated with oocyte expression trajectories. One graph-mer derived sperm motif was a bHLH binding site motif and exhibited spatial bias in the promoters of sperm genes but not non-sperm genes. The functional relevance of the CG-rich motifs was supported by strong conservation between C. elegans and C. briggsae and was associated with germline-specific in situ expression patterns. This study gives an interesting proof of principle for using PLS regression models for transcriptional regulation in developmental time series.
Learning graph-mer motifs and corresponding expression trajectories
In order to learn the correspondence between (sets of) regulatory motifs in the promoter sequences of genes and gene expression trajectories over a time course, we posed a regression problem: using a training set of genes, learn a linear mapping from the vector of counts of -mer occurrences in a gene's promoter to the gene's time course expression profile. This model can then be used to predict expression from sequence on held-out genes, and -mer features that are highly weighted in the model should represent important regulatory motifs. Here we have a very high-dimensional input space of motifs (-mers) as well as a multivariate output space, both of which rule out use of ordinary least squares regression. Instead, our algorithm makes use of a partial least squares (PLS) regression strategy. PLS is a well-known statistical technique for fitting linear models when the input space is high dimensional [8] and has both univariate and multivariate formulations.
Standard PLS represents the input data as a motif matrix X (dimension , where is the number of -mers), representing -mer counts for each gene's promoter, and the gene expression matrix by Y (dimension , where is the number of experiments), and then it performs two basic steps (see Methods for more details):
- Construct weight vectors in and corresponding latent factors in , where the weight vectors are chosen so that the latent factors have maximal covariance with directions in Y. The latent factors define a reduced dimensional representation of the promoter sequence data.
- Regress Y against the latent factors using ordinary least squares (or ridge) regression. The latent factor dimensionality reduction followed by linear mapping to Y yields the final mapping from sequence to expression.
PLS algorithms typically work iteratively, so that each round generates a new latent factor, and the number of rounds is chosen by cross-validation to minimize the square loss function in the regression problem.
Here, we are most interested in what PLS tells us about the covariance structure between X and Y and how to interpret this information in terms of sequence motifs and expression patterns. In particular, along with weight vectors in the input motif space, PLS determines corresponding vectors in the output expression space, defined so that is maximal (Figure 1). Intuitively, each weight vector corresponds to a set of motifs (-mers) that helps explain expression patterns in the direction . The components of the vector that have large positive weights are the -mers that most strongly predict the expression pattern .
At each iteration of the modified PLS algorithm, , weight vectors and are derived by finding latent factors and with maximal covariance. For clarity, subscripts are omitted in the diagram and in the rest of the description. Each weight vector w is a vector in , where is the number of -mers used as input to the algorithm. Due to graph-regularization, each weight vector is sparse, i.e. most -mers have weight , and smooth over a graph connecting sequence-similar -mers, i.e. similar -mers get assigned similar weights. Therefore, we can visualize the weight vector as a "graph-mer", a graph where nodes correspond to -mers with high positive weights and edges connect sequence-similar -mers (bottom left). At each iteration, the PLS procedure sets up a correspondence between the motif weight vector w and a weight vector over expression experiments represented by vector c. In our setting, the series of expression experiments is a time course, and the vector c can be viewed as an expression pattern or trajectory (bottom right). Intuitively, we can think of the set of -mers shown in the graph-mer as driving the expression pattern c. Roughly speaking, the model predicts that genes containing these -mers will have expression patterns that correlate with c; more precisely, the full regression model predicts gene expression patterns using all latent factors.
To obtain a more interpretable model, we mathematically imposed two additional requirements on the PLS solution. First, we wanted the weight vectors to be sparse, i.e. we wanted relatively few -mers to have non-zero components, so that the algorithm produces a small number of hopefully functional motifs. Second, for each weight vector , we wanted sequence-similar -mers to have similar weights, since such -mers may represent variants of the same binding site and potentially should contribute in the same way to the linear model. We achieved the first goal by adding a lasso constraint to the PLS optimization problem (see Methods, equation (4)). For the second goal, we defined a graph on the set of -mers, joining two -mers by an edge exactly when they are close in Hamming distance, and imposed a graph Laplacian constraint to obtain smoothness over the graph (see Methods, equation (7)). Incorporating these constraints into a multivariate PLS approach yields a new algorithm that we call graph-regularized PLS.
With these additional constraints, we can view the motif vectors as "graph-mers" – weighted graphs over -mers, where highly weighted dense clusters in the graphs correspond to important sequence-similar -mer sets, or motifs. Figure 1 illustrates the mapping between motif weight vectors, interpreted as graph-mers, and corresponding expression patterns, arising from the latent factors found in graph-regularized PLS. Intuitively, we can think of each vector as the expression pattern driven by the positively weighted -mers in , that is, the common expression trajectory displayed by genes containing these motifs. This correspondence will be important for interpreting regulatory motifs in worm germline development below.
Graph-mer modeling for germline development in worm
We applied our graph-regularized PLS regression algorithm to time series gene expression data for wild-type germline development in worm C. elegans [7]. This data set consists of a time course beginning in the middle of the third larval stage (L3) and extending through adulthood. During this time, the major developmental changes occur in the germ line. Some germ cells undergo constant proliferation, while others initiate developmental events, including entry into meiosis followed by differentiation into sperm, which occurs in the fourth larval stage, or differentiation into oocytes, which occurs in young adults. By the end of the timecourse, animals have produced mature gametes and launched embryogenesis. Twelve samples were collected at 3-hour intervals with 3 replicates for each sample. Basic microarray data normalization was performed in the original study, and we used the normalized gene expression levels as reported (Gene Expression Omnibus, http://www.ncbi.nlm.nih.gov/geo/, accession numbers GSE726-GSE737). We averaged expression levels over replicates for 20,000 genes and calculated the 5% and 95% quantile of all expression values. We filtered out genes with baseline expression (defined here as having expression values between the 5% and 95% quantiles at all time points) and also ones that exhibit little variance in expression over time (). After further removing genes without upstream sequences from WormMart, we obtained the gene expression matrix for 9,000 genes and 12 time points.
We downloaded promoter sequences spanning 500 bp upstream of transcription start sites from WormMart. For genes whose upstream intergenic sequence is shorter than 500 bps, we used the intergenic sequences instead of 500 bps upstream. We scanned the promoter sequences for candidate 6-mers and 7-mers, and filtered -mers based on expected counts in background sequences (see Methods).
Regularized PLS predicts held-out gene expression
We performed 10-fold cross-validation experiments, randomly splitting genes into test and training sets with 10% of the data assigned to test data. Figure 2A illustrates the normalized mean squared error (see Methods, equation (1)) on the cross-validation test sets versus number of latent factors for both standard and graph-regularized PLS. Here, the mean squared error obtained with zero latent factors (i.e. the variance of the test data) is normalized to 1, so that cross-validation errors below 1 indicate that the model is explaining part of the variance of the held-out data. Figure 2A shows the average mean squared error across the cross-validation folds with the standard deviation over folds indicated with error bars. The minimal cross-validation error with standard PLS is obtained with four latent factors. Graph-regularized PLS appears to be more resistant to overfitting, with slightly lower cross-validation error at four latent factors and no substantial increase in error as the number of latent factors increases. Again, cross-validation error suggests that four latent factors should be used in the model. As a negative control, we randomly paired promoter sequences with expression profiles, so that we used real expression data and promoter sequences but lost the correspondence between sequence and expression, and we performed standard PLS and graph-regularized PLS . As can be seen from Figure 2A, both standard PLS and graph-regularized PLS on randomized data overfit with the very first latent factor, indicating that the performance obtained on the real data is meaningful.
(A) Normalized mean squared error versus number of latent factors for standard PLS and graph-regularized PLS on real and randomized data. The mean squared error obtained with zero latent factor is normalized to 1. Computed standard deviations of squared error across cross-validation sets are plotted as error bars. For the real cross-validation data, standard PLS overfits after the 4th factor; graph-regularized PLS is more resistant to overfitting than standard PLS. As expected, when trained and tested on randomized data, both standard and graph-regularized PLS overfit with the very first factor. (B) Normalized mean squared error of sperm and oocyte gene sets for graph-regularized PLS. The first and second factors dominate oocyte and sperm genes respectively in terms of largest chi-square reduction.
Latent factors map to germline-specific expression trajectories
By analyzing separate microarray expression data from germline mutants, the previous study also identified two gene sets consisting of sperm and oocyte genes [7], which we used in our analysis of the wild type developmental gene expression profiles. First, we estimated the prediction error on each gene set as shown in Figure 2B. Clearly, the first and second latent factors account for the largest loss reduction for oocyte and sperm genes, respectively. To show that the first two factors dominate these two gene sets, we first examined the expression profiles of the two gene sets. In PLS, each weight vector gives the weights over time points and can be interpreted as an expression pattern, and genes significantly influenced by the latent factor tend to follow this expression pattern. We plot the oocyte gene expression profiles together with and sperm gene expression profiles with in Figure 3A and 3B. The gene expression profiles are strongly correlated with the corresponding weight vectors, indicating that the first two factors are able to retrieve the expression patterns of these two gene sets, respectively. Furthermore, we used functional enrichment analysis to confirm that the genes identified based on correlation with weight vector by these two factors are indeed enriched for oocyte and sperm genes, respectively (Figure S1(A,B)).
Oocyte and sperm gene expression patterns are strongly correlated with and , respectively. (A) Oocyte gene expression versus . (B) Sperm gene expression versus .
Interpretation of motif weight vectors
In PLS, each weight vector corresponds to a set of motifs (-mers) that help to explain expression patterns in the direction . The -mers with largest coefficients in are the most important variables for predicting the projection of the expression patterns of genes onto . To identify motifs relevant for sperm and oocyte gene sets, we selected the top 50 -mers ranked by and examined the -mer graphs corresponding to the first two latent factors. Clusters in the graph that are identified by MCODE [9] represent motif patterns and hierarchical sequence clustering is performed to generate corresponding PSSMs. Figures 4A and 5A show the graph-mer representation of the top 50 -mers, motif patterns and PSSMs for the first two factors.
(A) Sperm motifs extracted from graph-mer output. The graph-mer consisting of the top 50 -mers ranked by . Graph motif patterns identified in the form of -mer clusters using the MCODE plug-in [9] in Cytoscape are shown in different colors, with each subgraph summarized by a PSSM generated through hierarchical sequence agglomeration of the corresponding -mers. Both the ELT-1 motif GATAA and the bHLH motif ACGTG are found in this way. (B) Distribution of distance of motif ACGTG to TSS (measured in base pairs) in sperm genes versus non-sperm genes. Motif ACGTG occurs more frequently within 200bp upstream of the TSS in sperm genes relative to non-sperm genes, giving us more confidence in its contribution to sperm gene expression.
(A) Top 50 -mers ranked by the weight vector , depicted as a graph-mer, which are associated by the PLS procedure to the expression pattern of oocyte genes. Graph motif patterns were identified in the form of -mer clusters using the MCODE plug-in in Cytoscape. PSSMs generated through hierarchical sequence agglomeration of the corresponding -mer sets are indicated, revealing several CG-rich motifs. (B) Analysis of oocyte -mer conservation using the motif conservation score (MCS). The plot shows the distribution of (oocyte MCSnon-oocyte MCS) for top 50 -mers versus remaining -mers in . The score distribution for the top 50 -mers has a heavy right tail, showing that as a distribution, the top 50 -mers have higher oocyte-specific conservation scores as compared to other -mers (e-13 by a one-sided KS statistic). Significantly conserved -mers are annotated, including CG-rich -mers for oocyte genes. (C) Distribution of (sperm MCSnon-sperm MCS) for top 50 -mers versus remaining -mers in . The score distribution for the top 50 -mers has a heavy right tail, showing that the top 50 -mers have higher distribution of sperm-spefic conservation scores than other -mers (e-5, one-sided KS statistic). Significantly conserved -mers are annotated, including ACGTG motif for sperm genes.
From the second factor, we successfully found the ELT-1 ('erythrocyte-like transcription factor') motif GATAA and bHLH ('basic helix-loop-helix') motif ACGTG, as shown in Figure 4A. The ELT-1 protein is a transcriptional activator that can recognize the GATA motif, is highly expressed in the germ line, and has as potential targets a number of genes encoding major sperm proteins [10]. The bHLH proteins act through E-box elements with consensus CANNTG; the canonical E-box is CACGTG. bHLH proteins have been found to act at the E-box and influence hormone-induced promoter activation in mammalian Sertoli cells, which are required to maintain the process of spermatogenesis [11]; however, this motif has not previously been associated with spermatogenesis in C. elegans.
For the first latent factor, the top ranked motifs are CG-rich sequences as shown in Figure 5A, which are highly enriched in oocyte gene promoters (Figure S2), suggesting a potential role in oogenesis or regulation of oocyte gene expression. We found further evidence supporting the functional relevance of learned motifs for the first two latent factors by performing gene set enrichment analysis, which showed that oocyte and sperm gene sets are enriched in the corresponding -mer hits (Figure S1(C,D)).
Positional bias and conservation of motifs
Since functional motifs sometimes exhibit a spatial bias in the promoter region – for example, overrepresentation close to the transcription start site (TSS) – we performed positional analysis of top ranked motifs by examining their distance to the TSS in sperm genes versus non-sperm genes. We observed that the sequence element ACGTG displayed strong positional bias towards the TSS of sperm genes. Figure 4B plots the distribution of distance of ACGTG to TSS in sperm genes versus non-sperm genes, showing that ACGTG is found far more frequently within 200bp upstream of the TSS of sperm genes but displays a fairly uniform distribution relative to TSS in non-sperm genes. This result indicates that motif ACGTG was significantly overrepresented immediately upstream of sperm genes, giving us additional confidence in the motif's contribution to sperm gene expression.
To look for evidence of the functional roles of CG-rich and other highly weighted motifs, we considered conservation patterns of these sequences. Caenorhabditis briggsae is closely related to C. elegans and is frequently used in comparative genomics studies in worm. One expects that motifs responsible for a biological function that is shared by the two species, such as oogenesis, would be under evolutionary pressure and therefore conserved in the promoter regions of orthologous genes contributing to this function. We studied the conservation of all -mers between the two species and found that highly ranked -mers, where rankings are induced by the 1st and 2nd factor, tended to be more conserved in the oocyte genes and sperm genes, respectively. Specifically, we computed the motif conservation score (MCS) [12] of each -mer by comparing its conservation rate to its expected rate , estimated using 500 random -mers of the same length. A conserved occurrence of a -mer is an instance of the -mer in the C. elegans genome, for which it is also present in the C. briggsae ortholog. We reported MCS as a Z-score () measuring the significance of observing conserved occurrences out of total occurrences. To assess the significance of inferred -mers for oocyte and sperm gene sets, we focused on motif conservation in sperm and oocyte genes relative to non-sperm and non-oocyte genes. To do this, we computed the MCS of each -mer in both oocyte genes and non-oocyte genes, and we plotted the distribution of the difference of these two MCS scores for top 50 ranked -mers in the versus remaining -mers, as shown in Figure 5B, bottom panel; similarly, Figure 5C shows the difference of the MCS scores for sperm genes and non-sperm genes for the top 50 ranked -mers in versus the remaining -mers. For both oocyte and sperm gene sets, the score distribution for the top 50 k-mers has a heavy right tail relative to other -mers, showing that the top -mers have higher oocyte- and sperm-specific conservation. To confirm the significance of this observation, we performed a one-sided Kolmogorov-Smirnov (KS) test and found that the rightward shift was highly significant in both cases (-13 and e-5 for oocyte and sperm -mers, respectively). The -mers that are most significantly conserved in oocyte and sperm genes, relative to non-oocyte and non-sperm genes, are also annotated in Figure 5B and 5C; these include the ACGTG motif for sperm genes and CG-rich -mers for oocyte genes.
Targets of CG-rich motifs are expressed in the germline
Relatively little is known about transcriptional regulation of oocyte genes. To gain additional evidence supporting a functional role for learned motifs, we examined the in situ expression patterns of genes enriched with those motifs. We searched for a subset of EST (expressed sequence tag) clones known as YK clones of each gene in WormBase (http://www.wormbase.org) and looked at in situ expression patterns at the L4-adult stage associated with each YK clone in the Nematode Expression Pattern Database (NEXTDB http://nematode.lab.nig.ac.jp/db2/index.php).
The in situ analysis provides direct evidence about where the genes are expressed, and we expect that genes highly ranked by motif hits are more likely to be germline expressed. To obtain a ranked gene list for each of the three motifs in Figure 5A, we first defined the gene group associated with the first factor based on values (see Methods). For each motif, we ranked genes within the gene group by counts of -mers of that motif and came up with a list consisting of top 80 genes. Table 1 summarizes the in situ expression patterns of genes associated with motif 1 (GGCGC), motif 2 (GCGCG) and motif 3 (ACCGTA). We split each gene list into two groups, those already known to be oocyte genes, and genes with high motif scores not already defined as oocyte genes. For each group, Table 1 shows number of genes examined; the number of genes with an in situ pattern; and percentage of genes expressed in germline tissues only, in both germline and somatic tissues, and somatic tissues only.
Over all three motifs, 7% of the genes have detectable in situ staining. Of those, an average of 78% stain only in the germ line, and with more than 80% of genes previously identified as oocyte genes staining in the germ line.
More than 70% of genes that had not previously been identified as oocyte genes (based on mutant expression profiling) were also dominantly expressed in germline tissues rather than somatic tissues. In the study that defined the oocyte and sperm gene sets [7], about 20% of genes that were not identified as oocyte or sperm had the germline expression by in situ analysis. Table 1 shows that for the genes that were associated with oocyte motifs 1, 2 and 3 via latent factor analysis – but had not previously been identified as oocyte genes – 37/52, 43/55, and 38/62 showed germline expression. All these proportions are very significantly higher than the background percentage of 20% (e-16 for all motifs by a proportions test). These results provide additional evidence that we are learning functional motifs that contribute to germline expression.
Comparison with principal component analysis
Principal component analysis (PCA) is a widely used dimensionality reduction technique that extracts from the data matrix a sequence of orthogonal vectors, or principal components, that capture the directions of maximal variance in the input data. PCA is frequently used on either rows (genes) or columns (experiments) of a gene expression matrix for visualization or preprocessing prior to other kinds of analysis [13]. By contrast, PLS is a supervised method that, in our context, determines weight vectors as directions in gene expression space having maximal covariance with latent factors in motif space. Both PCA components and PLS weight vectors are interpreted as gene expression patterns. However, principal components are learned from gene expression data only, while weight vectors are found based on a linear mapping from motif space to gene expression space.
We were interested in comparing our (graph-regularized) PLS results with standard PCA in order to assess the value added by the motif information and supervised learning formulation. We anticipated some concordance of results, since directions that capture little variance in the expression data will also fail to have significant covariance with motif latent factors. Figure 6A and 6B plot the first four PCA components versus PLS weight vectors. The first and second PCA components indeed bear some similarity to the first and second PLS weight vectors and to some extent resemble the oocyte and sperm gene expression patterns, respectively. Since these two gene sets are fairly large and follow distinct expression patterns, they account for a significant portion of gene expression variance, and so it is not surprising that the first PCs show correlation with these patterns. However, all the principal components are less smooth, as expression trajectories, than their corresponding PLS weight vectors, and the smoothness of the PCs deteriorates more rapidly than in PLS as the number of principal components/latent factors increases. It therefore appears that PLS uses motif information to provide some degree of regularization on the weight vectors, leading to smoother expression patterns corresponding to latent factors.
The first and second principal components bear some similarity to corresponding PLS weight vectors , , but all principal components are less smooth than in PLS. (A) PCA identifies the first four directions (PC1, PC2, PC3 and PC4) that have maximal variance in gene expression space. Principal components are plotted v.s. time. (B) Graph-regularized PLS learns weight vectors (, , and ) based on a linear mapping from motif space to gene expression space. Weight vectors are plotted vs. time.
To confirm that the PLS-derived motifs could not be determined from analysis of the first and second principal components (PC1 and PC2), we performed the following motif discovery procedure: we identified the sets of genes that are highly correlated with PC1 and PC2, and ran the AlignACE motif discovery program on the promoters of these genes, yielding 58 and 89 motifs, respectively (see Text S1). In both cases, the top-ranked motifs were dominated by AA-rich and GG-rich motifs that likely come from low complexity regions (Figure S5). A few CG-rich motifs appear in the AlignACE list for PC1, but with relatively low MAP scores; only one motif from the list for PC2 matches any of the PLS-derived sperm motifs, and it occurs low in the ranking (rank = 33) with relatively weak MAP score. We conclude that analysis of the principle components does not retrieve the full motif information discovered by the PLS latent factors. This result underscores the importance of our predictive framework, mapping sequence to expression, rather than relying on correlation with expression and performing motif analysis after the fact.
Since the third and fourth PLS latent factors represent much smoother and quite different expression patterns than their PCS counterparts, we examined whether the genes associated to these factors based on motif and expression similarity (see Methods) may have common functions. While there were few genes associated to the fourth PLS factor (18 genes) showed no enrichment for GO terms, the gene set for the third PLS factor was significantly enriched for 54 GO terms (using a threshold of e-4, uncorrected hypergeometric people), of which the majority involved metabolism (32/54) and almost half of these were specific to amino acid metabolism (15/54). These genes are not enriched for germline expression, suggesting that our analysis has uncovered an independent co-regulation of a set of gene functions that might have been swamped out by the stronger germline information using other techniques.
Comparison with clustering
Finally, we compared our results with standard cluster-first analysis, using hierarchical clustering to identify 5 distinct gene clusters and applying the AlignACE motif discovery program to the promoters of each cluster in order to find over-represented motifs (Text S1). We identified two clusters (Clusters 1, 2) with subtly different expression patterns both resembling the expression signature of oocyte genes and one cluster (Cluster 3) similar to the sperm gene expression signature (Figure S6(A,B,C)). AlignACE returned lists of 47, 53 and 36 motifs for these three clusters, and as in the principal component analysis, the top ranked motifs in all cases were dominated by low-complexity AA-rich and GG-rich motifs (Figure S6(D,E,F)). A handful of low-ranked motifs with relatively poor MAP scores for Clusters 1 and 2 resembled two of the CG-rich -mers identified through the first PLS latent factor; for Cluster 3, none of the AlignACE motifs were similar to the sperm-specific -mers identified by the second PLS latent factor (Text S1). We conclude first that PLS avoids many presumably spurious motifs from low complexity regions while finding true germline-specific motifs that are missed through standard cluster-based analysis.
There have still been relatively few methods that integrate mRNA expression and promoter sequence data beyond "cluster-first" motif discovery. Beer and Tavazoie [14] similarly sought to reverse the information flow implied by clustering, to see how well motif content could predict expression patterns; in their case, however, expression patterns were identified with static clusters, motifs were discovered based on these clusters, and the learning task was the prediction of cluster membership rather than vector-valued expression profile. Ernst et al. [15] proposed a time-ordered hierarchical model for integrating motif and time series expression data, where motifs were associated with up/down bifurcations of expression profiles at particular time points; this method used static motif data rather than learning motifs. Segal et al. [16] combined promoter sequence and expression data within a probabilistic relational models framework to learn "modules" supported by both data sources; rather than learning motifs de novo, the algorithm was seeded with database motifs which could then be refined during expectation-maximization iterations. In our own previous work on the MEDUSA algorithm [17], we discretized expression data and used a boosting-based algorithm to discover motifs and assemble a regulatory program that predicts up/down expression of target genes. MEDUSA is well-suited to perturbation experiments and performs well even for small perturbation data sets [18]. In the current setting, where expression levels in consecutive time points are highly correlated and expression trajectories are smooth over time, discretizing the expression levels incurs a significant loss of signal, which we avoid by moving to a regression framework.
There have been several other regression based motif discovery approaches related to our work. For example, REDUCE [19] was the original method to use correlation between -mers and differential expression for motif discovery. REDUCE, however, uses each experiment independently, where we use multivariate PLS to treat full expression trajectories as the output space. To weight the benefits of regression with a multivariate output, we also tried fitting a separate graph-regularized univariate PLS model on each time point separately. We found that multivariate PLS outperforms univariate PLS (Figure S3), suggesting that correlating motifs with full expression patterns is more statistically accurate than performing regression one experiment at a time, at least in the case of correlated experiments such as time series data. Moreover, there was substantial overlap in the motif information inferred from nearby time points (see Text S1), showing that fitting a separate model for each time point entails a good deal of redundancy.
More recently, Zhang et al. [20] used PCA to define a basis of univariate response variables in the output space and then performed a REDUCE-like regression onto each variable to collect a set of motifs. In our work, by doing multivariate regression, we retain more structure in the solution, for example, a stratification of the output space by images of latent factors, each one corresponding to a characteristic time expression profile. We also note that lasso regression has been used elsewhere for learning regulatory networks in bacteria using time course expression data [21], and standard PLS has been used with a collection of known motifs in linear modeling of expression data in yeast and bacteria [22]. Finally, graph-based motif representations have been used previously by other groups, for example Naughton et al. [23], but this work again falls into the "cluster-first" category in that it seeks to find overrepresented motifs for a predefined gene set. By contrast, we learn motifs via a global regression problem, and the graph structure is encoded as a constraint on the solution.
A number of recent studies have expanded beyond the linear regression framework by introducing various kinds of non-linearity. First, various authors have extended standard linear models by proposing that certain sets of motifs have synergistic effects. For example, a synergistic pair of TFs can be modeled by including a term in the regression model for each of the individual motif counts as well as a third term for the product of the counts, as recently reviewed [24]. However, introducing too many of these additional non-linear terms greatly increases the risk of overfitting; for a typical pair of TFs, the count of co-occurrences is simply too sparse to estimate the synergistic parameter. These models require careful feature selection strategies; moreover, they mostly assume that the motifs are known and fixed, whereas we are performing de novo motif discovery. Second, motivated by biochemical models, several studies propose that the relationship between motif counts or TF occupancy scores (in the case of PSSMs) and log expression change is not linear and make use of a non-linear transfer function. Recent work using a probabilistic framework to predict the 1D anterior-posterior positioning of expression "stripes" in the early Drosophila embryo from cis regulatory module (CRM) sequences can be seen as an elegant example of this idea [25]. In this case, a logistic transfer function converts occupancy scores, computed from the space of configurations of TFs in the CRM, into sharp stripe boundaries. In our setting, however, we are learning from microarray expression data, which gives average (and noisy) measurements over a large population of cells with large underlying variation of expression levels. It is unclear whether mRNA expression data allows us to observe and model biochemically-expected non-linearity in this situation. Third, when confronted with a multi-variate response, such as in time series expression profiles, some authors have used a model where each motif count/occupancy score contributes linearly to the expression pattern at each time point (as we do) but the time points are connected by use of non-linear basis functions such as splines [26]. However, we find that the smoothness of the PLS-derived expression patterns comes for free as a result of the regularization choices in our method, so in our hands the smoothness prior did not seem to be statistically necessary.
Finally, our method can be applied to even more sparsely sampled time series covering a broader range of developmental stages. As a proof-of-principle, we applied graph-regularized PLS to a full life cycle C. elegans developmental time course consisting whole-animal gene expression profiles from egg to adult [27] (see Text S1). In this setting, the first latent factor contained germline-specific motifs similar to the ones found in the analysis of our main data set, while the next second and third latent factors were associated with more diverse biological functions (Figure S4). These results suggest that our approach can discover the structure of gene regulatory programs, in the form of latent factors corresponding to sequence patterns and expression trajectories, at a range of developmental time scales.
Materials and Methods
Standard partial least squares regression
Since our algorithm builds on ideas from PLS regression, we first describe how to use standard PLS to iteratively learn a linear mapping from the promoter sequences of genes, as represented by their -mer counts, and their mRNA expression profiles. Formally, using a training set of genes, PLS takes a motif matrix X (dimension , where is the number of -mers), representing the individual -mer counts for each gene, and a gene expression matrix by Y (dimension , where is the number of experiments). Here, the columns of X represent the independent variables (features) and the columns of Y are the response variables; we also call X the input matrix and Y the output matrix. PLS then performs the following steps:
- Scale X and Y so that each column of the input and output matrices has zero mean and unit variance.
- Perform dimensionality reduction by construction of latent factors : Construct weight vectors, placed as column vectors in W (dimension ), and corresponding latent factors, placed as column vectors in T (dimension ), where the weight vectors are chosen so that the latent factors have maximal covariance with directions in the multivariate response Y.
- Use the latent factors T to predict Y: Regress Y against the latent factors using ordinary least squares (or ridge) regression,
- Obtain the matrix B of regression coefficients:
We split genes into test and training sets for cross validation experiments. Training data including motif matrix X and gene expression matrix Y were used to learn matrix of regression coefficients B. And we assessed predictive power of PLS on test data and by normalized mean squared error (NMSE):(1)where denotes the expected value and .
PLS not only provides a solution to the regression problem, but it also describes the covariance structure between X and Y. It constructs weight vectors in the input space and corresponding vectors in the output space , where is maximal. Intuitively, each weight vector corresponds to a set of motifs (-mers) that helps explain expression patterns in the direction . The -mers with largest coefficients in are the most important variables for predicting the projection of the expression patterns of genes onto .
SIMPLS algorithm
There are a number of variants of PLS, each of which defines and solves an optimization problem for constructing the weight matrix W. We use the SIMPLS (Statistically Inspired Modification of PLS) algorithm [28], which optimizes an objective function defined on the matrix . The latent factors in T are sequentially built by estimating weight vectors as follows:
For :
- Maximize the covariance between and Y:(2)where is a unit vector.
- Impose orthogonality constraints for all , by deflating :(3)where (i) If , .
(ii) If , .
Regularized partial least squares regression
We now modify the PLS algorithm with the dual goals of (1) making the solution more interpretable and (2) regularizing the optimization problem, to reduce overfitting. We impose two constraints to achieve these goals. First, we use a lasso () constraint [2] to promote sparsity in the weight vectors , that is, drive the weights for many -mers to zero. Sparsity is clearly attractive since fewer -mers contribute to the solution, making it easier to identify the most important motifs. The lasso constraint over coordinates of weight vector takes the form:(4)
For the second constraint, we want sequence-similar -mers to have similar coefficients in the weight vectors, so that a group of similar -mers are more likely to act as a single motif pattern in the regression problem. We define a graph structure on the -mers where we place an edge if the Hamming distance between the pair of -mers and is less than threshold . Since -mers represent potential binding sites in double-stranded DNA, here we take the distance between two -mers and to be the minimum of the Hamming distances and , where is the reverse complement of . We then impose a smoothness constraint in the form of the graph Laplacian [29], as described below. The Laplacian matrix for an unweighted graph is defined as(5)where denotes the degree of -mer , the number of edges that connect -mer with other -mers. If we write as a column vector and view it as a function on the graph – i.e. a function that assigns a weight to each vertex – then we can use the graph Laplacian to compute a quadratic form on that satisfies the relationship [30]:(6)Equation (6) shows that this quadratic form measures the smoothness of with respect to the graph: the quadratic form is small when the function's values vary smoothly over adjacent nodes, so that the weights for sequence-similar -mers are close in value. Therefore, the second constraint that we impose is precisely on the size of the quadratic form, enforcing smoothness on the weight vector :(7)
A pseudocode description of the graph-regularized PLS algorithm is given in Figure 7.
A pseudocode description of the iterative PLS procedure, enforcing sparsity and Laplacian constraints on motif weight vectors.
Filtering k-mer features
-mer features with very sparse genome-wide counts are unlikely to improve the loss function – since they only only in a handful of promoters – and can contribute to overfitting. In order to eliminate -mers with infrequent counts prior to training, we filtered the -mer feature set based on expected counts on background sequences. We constructed the background sequences by shuffling exon sequences 100 times and ranked -mers by the -score [31]: , where is the number of the -mer in all promoter sequences, N is the length of all shuffled exon sequences, and is number of the -mer in all shuffled exon sequences divided by N. (Note that shuffled intergenic sequences could also be used to generate the random model.) We kept the top -mers and built the motif matrix containing counts of -mers in promoter sequences. We found that this filtering step significantly improved cross-validation performance.
Hierarchical sequence agglomeration
For each latent factor , we rank -mers by their components in the corresponding weight vector and perform motif analysis on the top 50 -mers. Those -mers are first displayed in the form of a motif graph via Cytoscape [32], in which an edge between two -mer nodes indicates similarity. We used the MCODE Cytoscape Plugin [9] to find -mer clusters (highly interconnected sets of sequence-similar -mers) in the graph. Each -mer cluster represents a motif pattern consisting of slightly different -mers.
Finally we perform a hierarchical sequence agglomeration algorithm to generate position-specific scoring matrices (PSSMs) for -mer clusters. Within each -mer cluster, each -mer is treated as a seed PSSM (using background nucleotide probabilities for smoothing), and then the algorithm iteratively merges similar PSSMs until a single PSSM is learned as the binding site model.
A position-specific scoring matrix (PSSM) is represented by a probability distribution over sequences , where . The emission probabilities are assumed to be independent at every position such that .
When comparing two PSSMs and , we allow offsets between their starting positions. We pad either the left or right ends with the background distribution and then define a distance measure as the minimum over all possible position offsets of the JS entropy.(8)where is the Kullback-Leibler divergence. Given that the position-specific probabilities are independent, one can easily show that . The relative weights of the two PSSMs, and , are here defined as , where are the numbers of target genes for the given PSSM. The initial PSSMs are -mers and the number of target genes are the number of promoter sequences with the -mer occurrence. The number of target genes for the newly merged PSSM will be the number of target genes combined for the two old PSSMs.
Assigning genes to latent factors
To extract biological information from the algorithm output, we analyzed latent factors for potential gene groups and corresponding biological functions. To do that, we assigned each gene to the gene group associated with a factor based on values. Here, the matrix (respectively, ) is formed by placing vectors (respectively, ) for latent factors as column vectors (Figure 1). The value indicates how well captures the -mer profile of gene , and the value measures the similarity between and expression profile of gene . In contrast to traditional clustering, which only relies on gene expression to group genes, we integrate both sequence and gene expression information in learning potentially functional gene sets. For each gene , we computed across all factors and chose factor with the maximum value:
Since we suspected that only large values indicated strong association of a gene with factor , we assigned gene to factor only when was in the top 20% of all values. Although we use latent factors in our model, here we compute the representation with five factors, reasoning that if a gene is assigned to the 5th factor, it should not be included in our main analysis.
Supporting Information
Figure S1.
Correspondence between first and second latent factors and sperm and oocyte genes. (A,B) The set of all genes is split into oocyte and non-oocyte genes, or sperm and non-sperm genes, and the empirical cumulative distribution of correlation with ci, i = 1,2 is plotted. Oocyte and sperm genes are enriched towards the top of the gene expression correlation distribution. (C,D) The set of all genes is split into oocyte and non-oocyte genes, or sperm and non-sperm genes, and the corresponding empirical cumulative distributions of hits of top 50 k-mers in wi, i = 1,2 are plotted. Oocyte and sperm genes are enriched in k-mer hits corresponding to the 1st and 2nd weight vectors.
Figure S2.
Correlation of weights with significance of enrichment in oocyte and sperm genes for the k-mers from 1st and 2nd graph-mer respectively. We plot the weights of k-mers in the first motif weight vector versus the −log10(p-value) for the enrichment of these k-mers in oocyte and sperm genes, as computed by the hypergeometric distribution. (A) For oocyte genes, −log10(p-value) is moderately correlated with w1 (Pearson coefficient = 0.65), and k-mers highly ranked by w1 had p-values between 10−16 and 10−4. This enrichment supports the functional relevance of PLS-derived k-mers from the first factor in oocyte genes. (B) For sperm genes, −log10(p-value) is somewhat correlated with w2 (Pearson coefficient = 0.35), though the correlation is weaker than that of oocyte genes.
Figure S3.
Normalized mean squared prediction error on cross-validation test data. (A) Normalized mean squared error versus number of PLS iterations for standard univariate and multivariate PLS. At each iteration, standard univariate PLS learns twelve latent factors, corresponding to the twelve individual time points, while multivariate PLS learns one latent factor for all time points. Univariate PLS yielded a slightly lower test error than that of standard multivariate PLS after the 1st iteration; however, after one iteration, the univariate PLS corresponds to a collection of motif sets, each predicting a single experiment's gene expression changes, while multivariate PLS uses a single motif set to predict full gene expression trajectories. (B) Normalized mean squared error on test data by time point after the 1st univariate PLS iteration. Normalized mean squared error versus time point on all genes, oocyte and sperm gene sets. Univariate PLS reaches lowest prediction error on oocyte gene set at late time points when oocyte gene expression peaks. Similarly, prediction error on sperm gene set is small at middle time points when sperm gene expression peaks. Each time-specific univariate PLS models the motif-expression correspondence for the gene set differentially expressed at the given time point.
Figure S4.
Latent factor analysis reveals graph-mers, expression patterns and significant associations of gene annotations. For each latent factor (i = 1…3), an associated mini graph-mer, extracted motif patterns and gene group are shown; annotations that are significantly enriched in each gene group are listed at the right (p<.001, uncorrected hypergeometric p-value), with p-values and number of genes associated with each annotation.
Figure S5.
Motifs found by AlignACE in genes correlated with PC1 and PC2. (A) Top 40 AlignACE motifs in genes correlated with PC1 sorted by MAP score. Top ranked AA-rich and GG-rich motifs may result from low complexity regions, and several PCA motifs with relatively low MAP scores (e.g. MAP = 147.05, 90.77, 80.93) are similar to PLS 1st factor motifs. (B) Top 40 AlignACE motifs in genes correlated with PC2. Only one motif (MAP score = 101.03) is similar to our PLS sperm gene motif ACGTG from 2nd weight vector. None of the other PCA motifs matched any of the PLS 2nd factor motifs.
Figure S6.
Motifs found by AlignACE in different gene clusters. (A) Expression patterns of genes in Cluster 1. (B) Expression patterns of genes in Cluster 2. (C) Expression patterns of genes in Cluster 3. (D) Top 40 AlignACE motifs found in Cluster 1 genes. (E) Top 40 AlignACE motifs found in Cluster 2 genes. (F) All 35 AlignACE motifs found in Cluster 3 genes.
Text S1.
Supplementary results
Author Contributions
Conceived and designed the experiments: XL CP CHW VR CL. Performed the experiments: XL CP. Analyzed the data: XL CP CHW VR CL. Contributed reagents/materials/analysis tools: XL CP CHW VR CL. Wrote the paper: XL VR CL.
- 1. Tompa M, Li N, Bailey TL, Church GM, De Moor BD, et al. (2005) Assessing computational tools for the discovery of transcription factor binding sites. Nat Biotechnol 23: 137–144.M. TompaN. LiTL BaileyGM ChurchBD De Moor2005Assessing computational tools for the discovery of transcription factor binding sites.Nat Biotechnol23137144
- 2. Tibshirani R (1996) Regression shrinkage and selection via the lasso. J R Stat Soc Series B 58: 267–288.R. Tibshirani1996Regression shrinkage and selection via the lasso.J R Stat Soc Series B58267288
- 3. Belkin M, Niyogi P, Sindhwani V (2006) Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. JMLR 7: 2399–2434.M. BelkinP. NiyogiV. Sindhwani2006Manifold regularization: A geometric framework for learning from labeled and unlabeled examples.JMLR723992434
- 4.
Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: Analysis and an algorithm. Advances in Neural Information Processing Systems 14. MIT Press. pp. 849–856.AY NgMI JordanY. Weiss2001On spectral clustering: Analysis and an algorithm.Advances in Neural Information Processing Systems 14MIT Press849856
- 5. Rapaport F, Zinovyev A, Dutreix M, Barillot E, Vert JP (2007) Classification of microarray data using gene networks. BMC Bioinformatics 8: F. RapaportA. ZinovyevM. DutreixE. BarillotJP Vert2007Classification of microarray data using gene networks.BMC Bioinformatics8
- 6.
Bailey TL, Elkan C (1994) Fitting a mixture model by expectation maximization to discover motifs in biopolymers. Proceedings of the Second International Conference on Intelligent Systems for Molecular Biology. AAAI Press. pp. 28–36.TL BaileyC. Elkan1994Fitting a mixture model by expectation maximization to discover motifs in biopolymers.Proceedings of the Second International Conference on Intelligent Systems for Molecular BiologyAAAI Press2836
- 7. Reinke V, Gil IS, Ward S, Kazmer K (2004) Genome-wide germline-enriched and sex-biased expression profiles in Caenorhabditis elegans. Development 131: 311–323.V. ReinkeIS GilS. WardK. Kazmer2004Genome-wide germline-enriched and sex-biased expression profiles in Caenorhabditis elegans.Development131311323
- 8. Boulesteix AL, Strimmer K (2007) Partial least squares: a versatile tool for the analysis of high-dimensional genomic data. Brief Bioinform 8: 32–44.AL BoulesteixK. Strimmer2007Partial least squares: a versatile tool for the analysis of high-dimensional genomic data.Brief Bioinform83244
- 9. Bader GD, Hogue CWV (2003) An automated method for finding molecular complexes in large protein interaction networks. BMC Bioinformatics 4: 2.GD BaderCWV Hogue2003An automated method for finding molecular complexes in large protein interaction networks.BMC Bioinformatics42
- 10. Shim Y (1999) elt-1, a gene encoding a caenorhabditis elegans GATA transcription factor, is highly expressed in the germ lines with msp genes as the potential targets. Mol Cells 9: 535–541.Y. Shim1999elt-1, a gene encoding a caenorhabditis elegans GATA transcription factor, is highly expressed in the germ lines with msp genes as the potential targets.Mol Cells9535541
- 11. J C, K SM (1999) Basic helix-loop-helix proteins can act at the e-box within the serum response element of the c-fos promoter to influence hormone-induced promoter activation in sertoli cells. Mol Endocrinol 13: 774–86.C. JSM K1999Basic helix-loop-helix proteins can act at the e-box within the serum response element of the c-fos promoter to influence hormone-induced promoter activation in sertoli cells.Mol Endocrinol1377486
- 12. Waterston R, Lindblad-Toh K, Birney E, Rogers J, Abril J, et al. (2002) Initial sequencing and comparative analysis of the mouse genome. Nature 420: 520–562.R. WaterstonK. Lindblad-TohE. BirneyJ. RogersJ. Abril2002Initial sequencing and comparative analysis of the mouse genome.Nature420520562
- 13.
Raychaudhuri S, Stuart J, Altman R (2000) Principal components analysis to summarize microarray experiments: application to sporulation time series. Pac Symp Biocomput. volume 5. pp. 455–466.S. RaychaudhuriJ. StuartR. Altman2000Principal components analysis to summarize microarray experiments: application to sporulation time series.Pac Symp Biocomput. volume 5455466
- 14. Beer MA, Tavazoie S (2004) Predicting gene expression from sequence. Cell 117: 185–98.MA BeerS. Tavazoie2004Predicting gene expression from sequence.Cell11718598
- 15. Ernst J, Vainas O, Harbison CT, Simon I, Bar-Joseph Z (2007) Reconstructing dynamic regulatory maps. Mol Syst Biol 3: 74.J. ErnstO. VainasCT HarbisonI. SimonZ. Bar-Joseph2007Reconstructing dynamic regulatory maps.Mol Syst Biol374
- 16. Segal E, Shapira M, Regev A, Pe'er D, Botstein D, et al. (2003) Module networks: Identifying regulatory modules and their condition specific regulators from gene expression data. Nat Genet 34: 166–176.E. SegalM. ShapiraA. RegevD. Pe'erD. Botstein2003Module networks: Identifying regulatory modules and their condition specific regulators from gene expression data.Nat Genet34166176
- 17.
Middendorf M, Kundaje A, Shah M, Freund Y, Wiggins C, et al. (2005) Motif discovery through predictive modeling of gene regulation. In: Miyano S ea Mesirov J, editor. RECOMB. Cambridge, MA: Springer. pp. 538–552.M. MiddendorfA. KundajeM. ShahY. FreundC. Wiggins2005Motif discovery through predictive modeling of gene regulation.J. Miyano S ea MesirovRECOMBCambridge, MASpringer538552
- 18. Kundaje A, Xin X, Lan C, Lianoglou S, Zhou M, et al. (2008) A predictive model of the oxygen and heme regulatory network in yeast. PLoS Comput Biol 4: A. KundajeX. XinC. LanS. LianoglouM. Zhou2008A predictive model of the oxygen and heme regulatory network in yeast.PLoS Comput Biol4
- 19. Bussemaker HJ, Li H, Siggia ED (2001) Regulatory element detection using correlation with expression. Nat Genet 27: 167–171.HJ BussemakerH. LiED Siggia2001Regulatory element detection using correlation with expression.Nat Genet27167171
- 20. Zhang NR, Wildermuth MC, Speed TP (2008) Transcription factor binding site prediction with multivariate gene expression data. Ann Appl Stat 2: 332–365.NR ZhangMC WildermuthTP Speed2008Transcription factor binding site prediction with multivariate gene expression data.Ann Appl Stat2332365
- 21. Bonneau R, Reiss D, Shannon P, Facciotti M, Hood L, et al. (2006) The inferelator: an algorithm for learning parsimonious regulatory networks from systems-biology data sets de novo. Genome Biol 7: R36.R. BonneauD. ReissP. ShannonM. FacciottiL. Hood2006The inferelator: an algorithm for learning parsimonious regulatory networks from systems-biology data sets de novo.Genome Biol7R36
- 22. Brilli M, Fani R, Lió P (2006) MotifScorer: using a compendium of microarrays to identify regulatory motifs. Bioinformatics 23: 493–495.M. BrilliR. FaniP. Lió2006MotifScorer: using a compendium of microarrays to identify regulatory motifs.Bioinformatics23493495
- 23. Naughton BT, Fratkin E, Batzoglou S, Brutlag DL (2006) A graph-based motif detection algorithm models complex nucleotide dependencies in transcription factor binding sites. Nucleic Acids Res 34: 5730–5739.BT NaughtonE. FratkinS. BatzoglouDL Brutlag2006A graph-based motif detection algorithm models complex nucleotide dependencies in transcription factor binding sites.Nucleic Acids Res3457305739
- 24. Das D, Pellegrini M, Gray JW (2009) A primer on regression methods for decoding cis-regulatory logic. PLoS Comput Biol 5: e1000269.D. DasM. PellegriniJW Gray2009A primer on regression methods for decoding cis-regulatory logic.PLoS Comput Biol5e1000269
- 25. Segal E, Raveh-Sadka T, Schroeder M, Unnerstall U, Gaul U (2008) Predicting expression patterns from regulatory sequence in drosophila segmentation. Nature 451: 535–540.E. SegalT. Raveh-SadkaM. SchroederU. UnnerstallU. Gaul2008Predicting expression patterns from regulatory sequence in drosophila segmentation.Nature451535540
- 26. Wang L, Chen G, Li H (2007) Group SCAD regression analysis for microarray time course gene expression data. Bioinformatics 23: 1486–1494.L. WangG. ChenH. Li2007Group SCAD regression analysis for microarray time course gene expression data.Bioinformatics2314861494
- 27. Hill A, Hunter C, Tsung B, Tucker-Kellogg G, Brown E (2000) Genomic analysis of gene expression in C. elegans. Science 290: 809–812.A. HillC. HunterB. TsungG. Tucker-KelloggE. Brown2000Genomic analysis of gene expression in C. elegans.Science290809812
- 28. Jong S (1993) SIMPLS: An alternative approach to partial least squares regression. Chemom Intell Lab Syst 18: 251–263.S. Jong1993SIMPLS: An alternative approach to partial least squares regression.Chemom Intell Lab Syst18251263
- 29.
Weinberger KQ, Sha F, Zhu Q, Saul LK (2007) Graph laplacian regularization for large-scale semidefinite programming. In: Schölkopf B, Platt J, Hoffman T, editors. Advances in Neural Information Processing Systems 19. Cambridge, MA: MIT Press. pp. 1489–1496.KQ WeinbergerF. ShaQ. ZhuLK Saul2007Graph laplacian regularization for large-scale semidefinite programming.B. SchölkopfJ. PlattT. HoffmanAdvances in Neural Information Processing Systems 19Cambridge, MAMIT Press14891496
- 30. Chung FRK (1997) Spectral Graph Theory (CBMS Regional Conference Series in Mathematics, No. 92) (Cbms Regional Conference Series in Mathematics). American Mathematical Society. FRK Chung1997Spectral Graph Theory (CBMS Regional Conference Series in Mathematics, No. 92) (Cbms Regional Conference Series in Mathematics).American Mathematical Society
- 31.
Eskin E, Gelfand M, Pevzner P (2002) Genome wide analysis of bacterial promoter regions. Biocomputing 2003: Proceedings of the Pacific Symposium Hawaii, USA 3–7 January 2003. World Scientific Pub Co Inc. 29 p.E. EskinM. GelfandP. Pevzner2002Genome wide analysis of bacterial promoter regions.Biocomputing 2003: Proceedings of the Pacific Symposium Hawaii, USA 3–7 January 2003World Scientific Pub Co Inc29
- 32. Shannon P, Markiel A, Ozier O, Baliga NS, Wang JT, et al. (2003) Cytoscape: a software environment for integrated models of biomolecular interaction networks. Genome Res 13: 2498–2504.P. ShannonA. MarkielO. OzierNS BaligaJT Wang2003Cytoscape: a software environment for integrated models of biomolecular interaction networks.Genome Res1324982504


text:
Dismiss Notice
Welcome To CK5!
Registering is free and easy! Hope to see you on the forums soon.
Score a FREE Membership sticker when you sign up for a Premium Membership and choose the recurring plan.
Lemme tell a story, I'm sure I've told before....
Discussion in 'The Lounge' started by chevyfumes, Apr 24, 2007.

subset: commoncrawl


text:
Tap Apply Coupons
This Item qualifies for FREE Shipping*
Your delivery dates will be verified at checkout.
Your delivery dates will be verified at checkout.
Product Information
HP 72 Cyan ink cartridges offer the finest quality and consistency for all your documents, projects, and presentations HP 72 130ml Cyan ink cartridge is designed to deliver the best image quality and reliable printing. Cartridge is compatible with Designjet T610/T620/T770/T790 T1100/T1120/T1200/T1300 and T2300 series printers to provide high quality documents.
- Ink volume: 130 ml
- Fade-resistant color provides superior results and brilliant, true-to-life images that last for generations
- The unmatched reliability of original HP ink cartridges means consistent convenience and better value
- High-quality cartridges print everything the way you wanted and are perfect for everyday printing applications
- Match the size of the ink cartridge to the needs of the project choose from 69 ml and 130-ml color ink cartridge sizes designed to let you optimize your productivity
- Packaging: Each
- Compatible with: HP Designjet printers: T1100, T1100 MFP, T1100ps 24-in, T1100ps 44-in, T1120HD, T1120SD, T1200, T1200 PostScript, T2300, T610, T620, T770
- Operating temperature range: 15 - 35 deg C, storage temperature range: 15 - 35 deg C, storage humidity range: 20 - 80% RH
- Warranty: 6 months limited
- Safety Data Sheet
- Product Information
Yield Type :High Yield
Cartridge Series :HP 72
Page Yield (Up to) :Unknown
Ink/Toner Color :Cyan
Ink/Toner Pack Size :1/Pack
Ink/Toner Cartridge Type :Original
Supply Type :Ink
Wide Format Compatibility :Yes
- Item Information
High Yield :Yes
Multipack :No
- Miscellaneous
Brand :HP
Quill Plus :Yes
- Weight
Weight :0.54 lbs. per Each
Product Reviews
Compatible Printers
- This product is compatible with these printers:
HPDesignJet T1100 24in. - 44in.DesignJet T610 24in. - 44 in.DesignJet T1100 ps 24in. - 44in.DesignJet T1200DesignJet T620DesignJet T770DesignJet T1200 HDDesignJet T1120 SD MFP 44in.DesignJet T1120 ps 24in.DesignJet T1120 ps 44in.DesignJet T1120DesignJet T2300 eMFPDesignJet T2300 PostScript eMFPDesignJet T1300 ePrinterDesignJet T790 ePrinter 24InDesignJet T790 ePrinter 44InDesignJet T790 PostScript ePrinter 44InDesignJet SD Pro


text:
5.0mm Cushion-Cut Amethyst and 0.11 CT. T.W. Diamond Frame Ring in 10K Rose Gold
Item# 19997634
The Peoples Difference
Why shop Peoples?
Orig. $479.00
Now $407.15
Celebrate her February birthday with a brilliant expression of your love. Created in warm 10K rose gold, this vintage-inspired ring features a 5.0mm cushion-cut bright purple amethyst centre stone. A glittering frame of dazzling diamonds surrounds the centre stone, while looping adornments flank each side. The diamond-lined shank completes this magical look. Radiant with 0.11 ct. t.w. of diamonds, this captivating ring is finished with a bright polished shine. View product details.
Special Offers
The amethysts pictured may have been treated or enhanced by heating. Typographic errors are subject to correction. Merchandise enlarged to show detail and may not always be exactly as pictured. Promotions, prices, styles and availability may vary by store and online.


text: I'm not sure Radio Shack has the culture to do anything remotely close to what I expect Best Buy to do and what Yahoo! did. In fact, there's no life left. It would take a miracle, quite possibly something more extraordinary than what Ford needs, for Radio Shack to subsist, even close to what it is, for much longer. Don't take a chance on Ford just yet. Never take one on RSH. Consider a speculative bet with ZNGA. But, really, tread carefully with low-priced stocks. Stick with large, dividend-paying growth stocks unless you have a time-tested, well-oiled plan to speculate on beaten-down, sub-$10 stocks. At the time of publication, the author was long FB and ZNGA.


text:
"What is one supposed to stuff it with? Cotton batting or something?"
This is my third moroccan pouf I bought from Overstock! Love,love,love!!!!...
Leather Silver Pouf Ottoman (Morocco)
ITEM#: 13449091This handmade Moroccan leather pouf ottoman has been beautifully hand-stitched by skilled artisans in Morocco. Taking about seven individual artisans to create, this pouf also features a stylish silver color.
- Colors: Silver
- Finish: Hand-stitched
- Upholstery materials: Goat-skin leather
- Upholstery color: Silver
- Dimensions: 14 inches high x 20 inches in diameter
Story Behind the Art:
This pouf was handmade by skilled men and women artisans from different regions of Marrakesh in Morocco. The process of making this pouf has been passed on from generation to generation. Also as these artisans work closely with designers; they add contemporary twist to their work which makes their products unique and more sophisticated. Some artisans work in a workshop and some work from their homes, especially women who are stay-at-home moms that have skills and like to contribute to the economy while taking care of their children and homes. Making these poufs is helps make a living for these artisans and the more they sell, the more they better the lives of their families.
What is Worldstock?
The handcrafted touch of artisan skill creates variations in color, size and design. If buying two of the same item, slight differences should be expected. Note: Color discrepancies may occur between this product and your computer screen.
Ships Carbon Neutral*
Please note: Due to the materials used, the pouf may have a slight odor when it arrives. Allow the pouf to air for a few days to remove the odor.
|Worldstock Country
|Product Features
|See Details
|Model Number
|Country of Origin
Similar Items
Handmade Contemporary Leather Ottoman Pouf (Morocco)
Sale: $202.49
Handmade Leather Contemporary Round Ottoman Pouf (Morocco)
Sale: $116.99 - $148.94
Handmade Round Mosaic Black Leather Ottoman Pouf (Made in Morocco)
Sale: $121.94
Handmade Moroccan Contemporary Leather Ottoman Pouf (Morocco)
Sale: $142.64
Handmade Moroccan Neutral Contemporary Leather Ottoman Pouf (Morocco)
Sale: $188.99
Handmade Moroccan Contemporary Leather Ottoman Pouf Green (Morocco)
Sale: $142.64
Customer Reviews
Most Helpful
Love it!
It was the perfect size for in front of a sofa that didnt have an ottomon. It is heavy enough that it doesnt move around and is light enough to move when cleaning the floor. People think it cost a lot more that around $XXXRead More
Functional Beauty
High end pouf. Beautiful way to add modern, elegant touch to a traditional or transitional decor. Color is a light silver but not super shiny thankfully high ore of a matte finish with a bit of the shine. High end quality with no dents as in some of the cheapos being sold. Nice goat skin leather. Perfect as a foot rest or additional seating. Great price compared to ******.Read More
Most Recent
smaller than indicated
Very pretty color and nice workmanship but smaller than on the website description. I will keep it but not sure if I could recommend it to another person unless they want something very small and lopsided.Read More
lovely accent piece!
This is my third moroccan pouf I bought from Overstock! Love,love,love!!!!Read More
Cheap-looking silver color
I have been looking for an ottoman like this for almost a year now and thought I found the perfect silver colored ottoman - this is not it! The color is cheap silver metallic and is almost neon it's so bright. I'd go to Jonathan Adler and take a look at theirs first - price is very close and the color is much richer.Read More
Questions & Answers
Hello kmac, unfortunately this item is out of stock. We currently do not have inventory to check, and cannot guarantee the item will be restocked. Thank you for shopping with us.
"Does the pouf come stuffed or with filling?"
Moroccan Leather Pouf, Silver color unstuffed with stitching. Thank you for your question.
Shopping Tips & Inspiration
Living Room Ideas
Get inspired by the hottest trends in living room furniture and decor. Shop our huge selection of living room furniture and decor at Overstock.com.
How to Use a Storage Ottoman
Take a look at some of the many ways you can use storage ottomans in your own home. Shop and save money on our great selection of ottomans.
Coastal Furniture and Decor Ideas
Find Coastal decorating ideas for your home. Shop our huge selection of Coastal furniture and decor at Overstock.com.
Boho Chic Furniture and Decor Ideas
Find Boho Chic decorating ideas for your home. Shop our huge selection of Boho Chic furniture and decor at Overstock.com.
Industrial Furniture and Decor Ideas
Find industrial decorating ideas for your home. Shop our huge selection of industrial furniture and decor at Overstock.com.
Shabby Chic Furniture and Decor Ideas
Find shabby chic decorating ideas for your home. Shop our huge selection of shabby chic furniture and decor at Overstock.com.
Mid-Century Modern Furniture and Decor Ideas
Shop our huge selection and save on mid-century modern furniture and decor. Find mid-century modern decorating ideas for your home at Overstock.com.
How to Clean an Ottoman
How to Clean an Ottoman from Overstock.com. Because ottomans are the object of much use by feet and shoes, they need an occasional deep cleaning. Here's how to clean your ottoman easily and effectively.
How to Accessorize an Ottoman
How to Accessorize an Ottoman from Overstock.com. Our guides provide customers with information about how to accessorize an ottoman.
Tips on Coordinating an Ottoman with Your Living Room
Tips on Coordinating an Ottoman with Your Living Room from Overstock.com. Our guides provide customers with information about Tips on Coordinating an Ottoman with Your Living Room.
Spirit of the Southwest
Capture the enchanting aura cast by red rock and turquise stone in home furnishings and decorative accessories inspired by the American Southwest.
Mixed Material Home
Reclaimed Wood and Cement. Marble and Metal. Leather and Wood. These materials didn't used to be mixed but now they've come together for a winning combination.
Shipping & Returns
Contact Information
Shipping Information:
Our standard shipping method is ground shipping. We also offer expedited shipping for most items. Your final
shipping cost and estimated delivery timeframe will be available at checkout.
Standard shipping details
Expedited shipping details
Standard Return Policy:
Items must be returned in new or unused condition and contain all original materials included with the shipment. More Details
FINAL SALE EXCLUSION: Items marked as FINAL SALE are not returnable unless the problem you experience is the result of our error.
For your protection, all orders are screened for security purposes. If your order is selected for review, our Loss Prevention Team may contact you by phone or email. There may be a two business day delay to process your order.
** Most Oversize orders are delivered within 1-4 weeks. Some orders may take 6 weeks to be delivered.


text:
There is a lot of good news to share since my last message. First, we are happy to announce that we have become an affiliated association of the Academy of Management. We were welcomed as the latest affiliate by President Deborah Shapiro at the annual meeting held in Anaheim, California in August.
While all affiliates operate autonomously, there will be opportunities for us to collaborate with AOM on specific projects. Another benefit of our affiliate status is being able to provide opportunities for our members to engage with one another at annual meetings of AOM.
I would like to thank all of the members of the Executive Committee who have worked hard since the inception of AFAM for the progress we have made. In our application for affiliate status, we were able to point to a number of accomplishments that support our mission to become the premiere association for academics who focus their teaching and research on management in Africa. We also want to thank each of you for being a faithful member of AFAM. Your continued support is important to our future success.


text:
Reviews - Hotel El Faro Marbella3
C/ Virgen del Pilar, 11, Marbella, 29602
Customer Reviews
Our reviews are from genuine LateRooms.com customers. We verify that the customer has stayed before leaving a review.
50% would recommend it to a friend
Customer service
Food quality
Room quality
Good location, rooms dated
Hotel was in a superb location, a few minutes from beach and main drinking / eating areas in marbella, fifteen minute drive to peurta banus.
No parking however and rooms are basic and dated. No kettle or microwave in kitchenette. Air conditioning didn't work either. Hotel staff friendly and helpful. Fairly cheap compared to other hotels.
Read More Less
Quick break away
Having booked a 5 night stay a few days before departure the EL Faro was a very economical choice for our stay.
The hotel is a little dated but very clean with comfortable beds and good showers. Breakfast is continental but very fresh with buffet being re-stocked regularly.
The hotel is convenient for sea front and Old Town both being only minutes away. One downside of the stay was the noise, we were placed at the back of the hotel overlooking the pool but along with this we were also overlooking a school playground and construction work, both started around 8.00am also noise from adjoining rooms. The hotel staff are very friendly and helpful. We would stay again but it would not be our first choice due to the noise.
Read More Less
- < previous
- 1
- page 1 of 1
- > next


text:
Academy members work to ensure that all Americans have access to a healthy, safe food supply by leading efforts to reduce food deserts, increasing participation in nutrition programs and working with industry to help develop nutritious food products that are widely available to the public. Federal nutrition programs, like the Supplemental Nutrition Assistance Program, National School Lunch and Breakfast Program and the Special Supplemental Women, Infants and Children Program, provided a bridge for citizens during the recent economic downturn. Nevertheless, the percentage of those that are food insecure still remains high.
We recognize that many of the populations served are vulnerable to food insecurity and to food poisoning and waterborne illnesses, and that counseling and other services must cover these topics to be effective. Academy members use a variety of engaging and effective nutrition education interventions that include social marketing initiatives to empower consumers to make safe, healthy food choices, as directed in the National Prevention and Health Promotion Strategy. It is important that consumers be given accurate and easy-to-understand information to make these food choices.


text:
Rhett Run
During the fall semester, the Terrier Connection invites students to compete in a fun run with a style of its own!
The Rhett Run is an event involving dozens of students who all come together to meet and race across campus. Students can trade in t-shirts from their old school (college or high school) in exchange for a BU t-shirt. Walk with friends, sprint to the finish line, or just take in the expanse of BU's campus during this fun event that kicks off the school year.
Join the Terrier Connection to get more information about this event.


text:
The WOD Life
Mobility Compression Floss Band - Standard
$29.95 $19.95
All you need is:
1) An Australlian visa or master card; 2) To be over 18 year of age; 3) To live in australia To see Afterpay's complete terms, Visit https://www.afterpay.com.au/terms
How Much Is Shipping?
Add products to your cart
to calculate shipping costs
Easy returns
100 Day return policy
Same Business Day Dispatch
on orders before 2PM AEST
Fast Shipping
on all orders
Have questions?
9AM to 5PM AEST MON-FRIDAY
Product description
Not just a voodoo band but a Mobility Compression Floss Band!
Flossing everyday keeps the doctor away!
The Mobility Compression Band is an indispensable performance tool for every athlete that is aiming to mobilise in as close to the actual movement you are performing as possible. Compression bands can aid in improving range, restore joint mechanics and unglue matted down, previously injured tissue. By compressing the swell of tissues and joints, a compression tack can work to perfuse tissues that have grown to be cold and stiff after injury.
It's difficult to find techniques that are effective and can actually be applied to a working movement. Because compression band techniques can be done while essentially performing the movement the athlete is trying to change, it's result on sliding surface and restoration and tissue mobilisation is without comparison. The Mobility Compression Band is an essential performance tool in injury prevention and recovery. Compression tack and flossing works on many levels.
Because the Mobility Compression Band can be used while actually performing the movement the athlete is trying to change, its effect on sliding surface and restoration and tissue mobilisation is unmatched.
Length: 7'(2.13m)
Width: 2" (5cm)
Material: Natural Latex Rubber – will stretch 150% of length.


text:
My name's Andy Baker – Hove is a long standing nickname from school. I'm now a middle-aged Dad trying to achieve what I did as a teenager. Back then I built a turtle controlled by a BBC Micro via a very long piece of ribbon cable. All I have left to show of that is a photocopy of an article in Everyday Electronics, and a couple of aluminium wheels.
Now, 25 years on, the Raspberry Pi offered the opportunity to do this again, but better. This blog starts with the building of another turtle. It then moves on to cover the building of my quadcopters – a process which is still in progress despite having spent over four years already. It's been interesting, challenging, and often very frustrating, but while the net progress is positive, I'm going to keep on going.
I'm one of the founders of the Cheltenham based Cotswold Jam, a volunteer-run event, promoting kids' education in ICT / STEM in an interactive and fun way. Details of the next event are available here. Tickets are free, and open to all, with a bias towards kids.
By all means, get in touch with me via e-mail here
P.S. The site logo is my son's and daughter's favourite bed-time cuddle friends, Jack and Tedda, as named by them.
P.P.S. My favourite colour is amber; can't you tell from the site and my projects!


text:
Group Music Classes
Learn a new musical instrument or brush up on your skills in a social setting with one of our Denver group music classes! Offered at our Yale, Lowry and Highlands locations. We offer group classes for every age, interest, and skill level. Next 8-week session begins the week of June 26. Join in on the fun!
Beginner / Core
CORE Programs
A great place for those just starting out. Our level 1-A classes start with how to hold your instrument and tune it…for the very beginner!
Start your search for Core Classes
What's an Ensemble class?
Play with a group! We have ensembles for everything from Cajun to the Beatles.
View All Ensemble Classes
What's a Specialty class?
Focus on a specific skill (fingerstyle guitar) or genre/artist (African American Spirituals).
View All Specialty Classes
Early Childhood
6mo - 4yr
Popular Programs View All EC Classes