text:
Protein tertiary structure prediction is a fundamental problem in computational biology and identifying the most native-like model from a set of predicted models is a key sub-problem. Consensus methods work well when the redundant models in the set are the most native-like, but fail when the most native-like model is unique. In contrast, structure-based methods score models independently and can be applied to model sets of any size and redundancy level. Additionally, structure-based methods have a variety of important applications including analogous fold recognition, refinement of sequence-structure alignments, and de novo prediction. The purpose of this work was to develop a structure-based model selection method based on predicted structural features that could be applied successfully to any set of models.
Here we introduce SELECTpro, a novel structure-based model selection method derived from an energy function comprising physical, statistical, and predicted structural terms. Novel and unique energy terms include predicted secondary structure, predicted solvent accessibility, predicted contact map, β-strand pairing, and side-chain hydrogen bonding.
SELECTpro participated in the new model quality assessment (QA) category in CASP7, submitting predictions for all 95 targets and achieved top results. The average difference in GDT-TS between models ranked first by SELECTpro and the most native-like model was 5.07. This GDT-TS difference was less than 1% of the GDT-TS of the most native-like model for 18 targets, and less than 10% for 66 targets. SELECTpro also ranked the single most native-like first for 15 targets, in the top five for 39 targets, and in the top ten for 53 targets, more often than any other method. Because the ranking metric is skewed by model redundancy and ignores poor models with a better ranking than the most native-like model, the BLUNDER metric is introduced to overcome these limitations. SELECTpro is also evaluated on a recent benchmark set of 16 small proteins with large decoy sets of 12500 to 20000 models for each protein, where it outperforms the benchmarked method (I-TASSER).
SELECTpro is an effective model selection method that scores models independently and is appropriate for use on any model set. SELECTpro is available for download as a stand alone application at: http://www.igb.uci.edu/~baldig/selectpro.html webcite. SELECTpro is also available as a public server at the same site.
Selecting the most native-like model from a set of possible models is a crucial task in protein structure prediction. A variety of Model Quality Assessment Programs (MQAPs) have been developed that assign numeric scores to models in a set, and then use the scores to rank the models and ultimately select a single model. MQAP methods can be divided roughly into three categories based on the type of information they use: evolutionary methods use sequence or profile similarity between target sequence and template, consensus methods use similarity between models, and structure-based methods use model coordinates [1]. Each category of methods has inherent strengths and weaknesses.
Evolutionary methods can provide quality scores that have been shown to correlate with structural similarity to native [2]. However, for lower confidence alignments the scores do not correlate well with structural similarity. Furthermore, identification of the best template and specific alignment can be difficult. In addition, models built from multiple templates or template-free methods cannot be scored appropriately by evolutionary methods alone.
Consensus methods take advantage of the observation that similar models produced by different predictors tend to be more accurate than those that are structural outliers. In practice, consensus methods outperform the methods they draw from, and they rarely pick a very poor model. The disadvantage, however, is that when the best model is a structural outlier it will be overlooked for lack of popularity [1]. Also, consensus methods are not appropriate for selecting from small sets of structurally diverse models, especially in the extreme case of a two-model set.
While consensus methods depend on similarity between models, structure-based methods calculate scores on each model independently. For this reason, structure-based methods can be applied to model sets of any size and diversity, and will produce the same score for a model regardless of the other models in the set. Structure-based methods can also be used for template-free modeling [3-6] and model refinement procedures [7,8]. One weakness of high resolution structure-based methods, including protein free energy approximation functions [9-12] and physics based approaches [13,14], is their sensitivity to local structural irregularities such as steric clashes and chain breaks, which can significantly bias scores on otherwise accurate models. Even slight differences in model backbones can produce significantly different scores [15]. Lower resolution structure-based methods, such as statistical potentials [6,16,17], are more robust to backbone variation, but are sensitive to extended low contact-order regions in the models.
Here we describe SELECTpro, a novel structure-based MQAP that combines high and low resolution energy terms into a model selection method that is effective on model sets of variable size, diversity, and target difficulty. Most of our assessment is calculated from the CASP7 model quality assessment category (QA) results published online [18]. The QA category provides a framework for the unbiased evaluation of MQAPs on ensembles of models produced by diverse automated prediction methods.
Results and discussion
We analyze the CASP7 quality assessment category predictions with a focus on the quality of the model ranked first by each predictor and the recovery of the most native-like model in the set. Only SetAll is used in the assessment of the quality of the model ranked first by each group (Table 1). The results are very similar when using SetComplete (data not shown) because QA groups rarely rank an incomplete model first.
Table 1. Quality of Model Ranked First (MQA1) Relative to Most Native-Like Model (Mmax)
The assessment of the recovery of the most native-like model, is performed on both SetAll and SetComplete (Table 2) because the few cases where an incomplete model is the most native-like have a significant effect on the average recovery metrics of all QA groups. Incomplete and irregular models are especially challenging for structure-based methods. A comparison of the average Pearson Correlation on SetAll and SetComplete, highlights these issues (Table 3). The frequency of recovering the most native-like model is calculated on SetComplete (Figure 1).
Table 2. Recovery of Top GDT-TS Model (Mmax)
Table 3. Correlation of Selected Groups
Figure 1. Recovery of Mmax using SetComplete. (A) number of targets where Mmax is ranked first (top of dark blue bar), in the top five (top of gray bar), and in the top ten (top of white bar). (B) number of targets where ΔGDTBLUNDER% is less than 10% (top of light blue bar) and less than 20% (top of purple bar). Only the first ten groups are shown in both graphs.
The utility of SELECTpro for selecting the best model from a small set is demonstrated by selecting from the five models submitted for each target by the top automated predictors. These small set selection results are calculated using SetAll (Figure 2). SELECTpro is also evaluated on a recent benchmark set of 16 small proteins with large decoy sets of 12500 to 20000 models for each protein and compared to I-TASSER (Figure 3).
Figure 2. Reranking models from top servers. Each server predictor submitted five models per target, with the highest confidence model ranked first. (A) the number of targets where each server's highest GDT-TS model is ranked first is shown with gray bars, and black bars when the models are reranked with SELECTpro. (B) shows the change in average GDT-TS for each group when SELECTpro is used to select model 1. P-values of paired t-tests are shown above the horizontal axis when SELECTpro demonstrates improved model selection and statistically significant improvements (p < .05) are in bold.
Figure 3. Large Decoy Set Model Selection. Large decoy set model selection with SELECTpro on I-TASSER benchmark set. This set of 16 small proteins was used as one of the benchmark sets for evaluating the I-TASSER method [19]. The complete decoy sets can be downloaded from [20]. Each protein has from 12500 to 20000 decoy models. For each protein different symbols are used to indicate the GDT-TS of Mmax (□), SELECTpro's MQA1 (×), and I-TASSER's MQA1 (+).
To make fair comparisons to groups participating on only a subset of targets, common subset comparisons between SELECTpro and each of these groups are included in Tables 1 and 2. Only groups participating on at least half of the targets are included, and for groups with multiple submissions only the best one is shown. In the results tables any value that is better than SELECTpro is underlined.
For multiple domain targets, the sum of GDT-TS over all domains is used as the GDT-TS of the model. Since the QA predictions correspond to the entire structures, it is impossible to fairly assess the domains independently.
To assess the significance of the summary statistics compared in Table 1, Table 2, and Figure 2, we performed paired t-tests between SELECTpro each other group on common subsets of targets (or targets and models when appropriate). All p-values from the tests appear in the tables and figure, but only statistically significant p-values (p < .05) are shown in bold.
The following notations are used throughout the results section:
• Mmax: The model with the highest GDT-TS among all server models.
• MQA1: The model with the highest QA score.
• NT: The number of targets a group made valid predictions on.
• ND: The number of domains a group made valid predictions on.
The recovery of Mmax by a QA predictor can only be evaluated if Mmaxwas scored by the predictor. In most cases QA predictors did not provide scores for all available server models, and frequently there is no score for Mmax. For example, predictor 016_1 (AMBER/PB) made submissions on 86 targets, but Mmax is only scored for 53 of these targets – so only these targets (NT = 53) can be evaluated for this predictor.
Quality of Model Ranked First (MQA1) Relative to Most Native-Like Model (Mmax)
In this section on the assessment of the model ranked first, and the corresponding Table 1, we use the following three metrics:
• ΔGDTQA1 = GDT-TS(Mmax) - GDT-TS(MQA1) : The GDT-TS difference between Mmax and MQA1 measures how much is lost by selecting MQA1 rather than Mmax for a single target.
• = ΣΔGDTQA1/ND : The average ΔGDTQA1 is a simple way of assessing the quality of MQA1 over all targets.
• ΔGDTQA1% = ΔGDTQA1/GDT-TS(Mmax) : The GDT-TS difference percentage allows for comparison across targets with different numbers of domains and difficulty levels.
The columns of Table 1 are: (1) group number; (2) number of targets the group made predictions on; (3) number of targets such that ΔGDTQA1 = 0; (4) number of targets such that ΔGDTQA1% < 1%; (5) number of targets such that ΔGDTQA1% < 10%; and (6) . The common subset results section has an additional column for the p-value of the paired t-test using ΔGDTQA1. The rows are sorted first by the number of targets and then by . Of the groups participating on all 95 targets, SELECTpro has the lowest average ΔGDTQA1, with a value of 5.07, followed closely by group 713_1 (Circle-QA), with a value of 5.44. Predictor 038_1 (GeneSilico) has an average ΔGDTQA1 of 5.75, with predictions on 85 targets. In common subset comparisons with these two groups SELECTpro is not significantly better, with p-values of .25 and .12 respectively. In common subset comparisons with all remaining groups SELECTpro is significantly better.
Another way to assess the quality of MQA1 over many targets is to count the number of targets such that MQA1 is the best model, or nearly the best, in the set. A method that performs very well on most targets, but very poorly on a few, would still be recognized by this criteria. SELECTpro recovers the best model for 12 targets, selects a model with ΔGDTQA1% < 1% for 18 targets, and selects a model with ΔGDTQA1% < 10% for 66 targets. Group 091_1 (Ma-OPUS) also performs well, with 11, 18, and 61 targets in the respective categories. Only the 60 targets with ΔGDTQA1% < 10% of predictor 038_1 (GeneSilico) on its 85 target subset are better than SELECTpro in common subset comparison (58 for SELECTpro).
The BLUNDER Measure Recovery of Mmax
How well does a QA predictor recover Mmax? The traditional metric to assess Mmax recovery is the rank of Mmax, and the average rank over many targets (). While rank captures some important information, it ignores the redundancy of models and the quality of models ranked better than Mmax. Consider the following hypothetical situation: group A ranks Mmax 10th and all nine models ranked above it are redundant with ΔGDT of ~2.0, group B ranks Mmax 5th and the four models ranked above it are diverse with a ΔGDT between 10.0 and 20.0. Which group has done a better job of recovering Mmax? In this example, the rank metric favors group B, although group A ranks only a single redundant model above Mmax. In addition, the models ranked better than Mmax by group A have only slightly lower GDT-TS than Mmax, while the models ranked better than Mmax by group B are significantly worse than Mmax. To address these weaknesses of the rank metric, we introduce the BLUNDER metric, which focuses on the worst model ranked better than Mmax (the most embarrassing blunder). This measure is not affected by model redundancy and measures the quality of models ranked above Mmax. The BLUNDER metric is defined using the following notation, and used in the assessment of the recovery of Mmax and the corresponding Table 2 and Figure 1:
• MBLUNDER: The model with the minimum GDT-TS among models ranked better than Mmax.
• ΔGDTBLUNDER = GDT-TS(Mmax) - GDT-TS(MBLUNDER) : The GDT-TS difference between Mmax and MBLUNDER measures the size of the worst blunder.
• = ΣΔGDTBLUNDER/ND : The average ΔGDTBLUNDER measures how well a method robustly recovers Mmax over many targets.
• ΔGDTBLUNDER% = ΔGDTBLUNDER/GDT-TS(Mmax) : The ΔGDTBLUNDER percentage allows for comparison across targets with different numbers of domains and difficulty levels.
Figure 1 contains graphs of the frequency of recovering Mmax using the rank (A) and ΔGDTBLUNDER% (B) measures on SetComplete. SELECTpro ranks Mmax first for 15 targets, in the top five for 39 targets, and in the top ten for 53 targets. SELECTpro's ΔGDTBLUNDER% values are less than 10% of GDT-TS(Mmax) for 40 targets and less than 20% for 63 targets. These results are best among all QA participants. The average Mmax recovery results are summarized in Table 2. The results columns are (1) average rank () and (2) average ΔGDTBLUNDER () on SetAll and SetComplete. The common subset results section also includes a column for the p-value of a paired t-test using ΔGDTBLUNDER (p-value). Rows are sorted separately for each dataset by NT first and then . On SetComplete SELECTpro has a of 10.4. In common subset comparisons one group has a lower : group 091_1 (Ma-OPUS) with of 16.8 on 94 targets compared to 17.4 for SELECTpro. On SetAll SELECTpro did not submit a score for Mmax of target T0356 (HHpred2_TS1) due to a processing error. In order to make complete common subset comparisons when possible we added in the SELECTpro score for HHpred2_TS1. SELECTpro ranks it 86th and ΔGDTBLUNDER = 50.0. Both results are significantly worse than the SELECTpro averages.
Pearson Correlation for Individual Proteins
The assessor evaluation of the quality assessment category [18] focused on the Pearson Correlation between the QA scores and GDT-TS. Here we use the Pearson Correlation only to highlight some of the difficulties for structure-based methods in dealing with incomplete models, as well as basic non-protein like structural features. Approximately half of the models in SetAll are incomplete, with backbone coordinates missing for one or more residues.
Incomplete models present a challenge to SELECTpro and other structure-based methods because the scores for each model are only comparable when calculated on coordinates for the same set of residues. Another issue is that some complete models have severe chain-breaks, severe steric clashes, or significant portions modeled only as extended chains. These local problems can overwhelm the energy of what may otherwise be a good model. Consensus methods do not suffer from these local structure problems. Given this rationale, one would expect structure-based methods to see the most improvement in terms of average Pearson Correlation on SetComplete relative to SetAll. Table 3 shows the average Pearson Correlation of five selected groups. Predictors 713_1 (Circle-QA), 633_1 (ProQ), and SELECTpro are structure-based MQAPs, while 634_1 (Pcons) is a consensus method and 556_1 (LEE) scored structures based on the GDT-TS similarity to their human Model 1 CASP7 prediction [18]. As expected, the structure-based MQAPs improve more than the structural similarity-based methods. The even greater increase in Pearson Correlation for SELECTpro can be accounted for by the failure to generate appropriate complete models for some of the incomplete models resulting in QA scores calculated on extended chains.
Reranking Top Server Group Models
Predictors in CASP may submit up to five models, but CASP evaluation focuses on the model designated as Model 1. Clearly, the selection of Model 1 is critical in the CASP setting and for protein structure prediction in general. Figure 2 contains the results when SELECTpro is used to rerank the five models submitted by each of the top ten servers from CASP7, compared to each server's results. In the following assessment Mmax-g is the model with the highest GDT-TS of the five models submitted by a server. Figure 2 (A) shows that SELECTpro recovers Mmax-g more frequently than 8 of the top 10 server groups; in addition, when SELECTpro is used to select Model 1 the average GDT-TS increases for 7 of 10 sever groups; however, the increase is only statistically significant for 3 groups. SELECTpro improves using both criteria for the top 3 server groups (Zhang-Server, Pmodeller6, and ROBETTA). These results highlight the utility of SELECTpro for the task of model selection. The comparisons made here are fair because structure-based methods can be applied in the server setting to any number of models.
Large Decoy Set Model Selection
Here we analyze SELECTpro's model selection capability on the large decoy sets for 16 small proteins from a recent I-TASSER benchmark set [19]. The I-TASSER prediction method generates 12500 to 20000 different backbone conformations. The complete decoy sets can be downloaded from [20]. The consensus method SPICKER [21] is used to cluster the models and a centroid model is built from the first cluster. A second round of simulation resolves the steric clashes in the centroid model and results in the final predicted model. The centroid model and final model are not part of the decoy set. In order to make a fair model selection comparison the decoy model closest to the centroid is used as I-TASSER's MQA1.
On the benchmark set SELECTpro has an average GDT-TS of 63.7, while I-TASSER has an average GDT-TS of 62.1. SELECTpro's average ΔGDTQA1 is 9.2 and I-TASSER's ΔGDTQA1 is 10.7. Figure 3 displays the GDT-TS results for the individual proteins in the benchmark set. Different symbols are used to indicate the GDT-TS of Mmax (□), the GDT-TS of SELECTpro's MQA1 (×), and the GDT-TS of I-TASSER's MQA1 (+) for each protein. A paired t-test of the hypothesis that SELECTpro and I-TASSER's mean performance are equal produces a p-value of .19, which is not statistically significant, but does give some evidence that SELECTpro can select a very good model from a large set of decoys at least well as an established method that utilizes consensus methods.
A MQAP that can select the most native-like model from a set of possibilities has a variety of applications in protein structure prediction. The new quality assessment category introduced in CASP7 allows for the unbiased assessment of MQAPs on the models produced by automated predictors. This category allows researchers to focus on the model scoring aspect of protein structure prediction.
The results presented in this work demonstrate that SELECTpro, a structure-based model selection method, consistently selects one of the best models from the large diverse sets of models produced by automated predictors, across all levels of target difficulty. On these large diverse sets of models, SELECTpro also recovers the single most native-like model well compared to other methods. On the small sets of five models submitted for each target by the top automated predictors, in most cases SELECTpro selects better models than the predictors themselves.
Since SELECTpro and other structure-based methods score models independently, they can be incorporated into the model selection pipelines of individual protein structure prediction servers. For this reason, it may help predictors if the CASP organizers distinguished methods that score models independently from those that do not.
Consensus and structure-based methods can be combined to achieve improved results. For example, the meta-server method Pmodeller [22] combines consensus (Pcons [23]) and structure-based methods (ProQ [24]) to predict protein structures more accurately than either method in isolation. The assessment of the QA category by CASP assessors recognized the consensus method Pcons (group 634_1) for the high Pearson Correlation between their scores and model GDT-TS on most targets [18]. In their own assessment the authors of Pcons recognized that while consensus methods perform well in most cases, "when most of the models are incorrect and the few correct models are outliers a consensus based approach cannot be expected to make an optimal choice." [1] For instance, they identified three particular targets in CASP7 where their consensus method failed: T0283, T0350, and T0351 [1]. The Pcons average ΔGDTQA1 on these three targets is 30.8. The same research group's structure-based method ProQ (group 633_1) has an average ΔGDTQA1 of 17.2. In contrast, on these three targets SELECTpro has an average ΔGDTQA1 of only 7.1. This example highlights the potential of combining SELECTpro with existing model selection methods.
SELECTpro has been made publicly available as a server, where users may submit from 2 to 100 models for evaluation. In addition to the global confidence scores, the scores of individual energy terms are also returned to the user by email for each model submitted. SELECTpro is one of several protein structure tools in the SCRATCH suite of predictors [25], and is available through: http://www.igb.uci.edu/~baldig/selectpro.html webcite.
All of the comparative analysis in this work is performed on the server models and quality assessment predictions submitted in the CASP7 [26] experiment. The CASP QA experiment is particularly relevant for the evaluation of model selection methods for several reasons: (1) the QA predictors were blind to the true structures at the time of prediction making it impossible for methods to be tuned to improve results; (2) the set of proteins is diverse: the 95 targets range in size from 68 to 530 amino acids, come from a variety of organisms, and span the full range of prediction difficulty; (3) each target has more than 200 predicted models that contain the types of errors that occur in automated structure prediction; (4) the protein set is not selected by any of the participating QA groups; (5) the models are scored by a variety of methods and the results are publicly available. We perform analysis on the set of all models (SetAll) and a subset of models (SetComplete) that are complete and free of gross structural irregularities, as described below. All of the ABIpro models and some of the 3Dpro models were optimized using the exact energy function of SELECTpro. These models are removed because of the obvious bias towards these models. In recent CASP experiments the GDT-TS [27] has been used as the primary automatic structural similarity measure. The published GDT-TS values from the CASP7 website are the only structural similarity measure used in this work.
The SetAll dataset consists of the server models with a GDT-TS value published on the CASP7 website, a total of 23,423 models. To calculate a score on a protein model SELECTpro requires the backbone coordinates (N, Cα, C) for all model residues as input. A total of 8,812 models in SetAll have only a Cα trace or have no coordinates for one or more residues. Modeller8v1 [28-30] was used to generate complete models from the incomplete ones, and then the complete models were scored by SELECTpro. In most cases the complete models were built appropriately from the incomplete models; however, in some cases the final model was a fully extended chain due to an error in our application of Modeller. We failed to identify this problem until after the completion of the CASP7 competition. The SELECTpro scores versus GDT-TS scores for all models of target T0305 are displayed in plot A of Figure 4. The circled outliers with very low confidence scores and high GDT-TS scores are models that were incomplete and the complete models generated by Modeller were fully extended chains. The Pearson correlation on the set of all models for T0305 is .641. The SELECTpro scores versus GDT-TS scores for complete models only are displayed in plot B of Figure 4, and the Pearson correlation is .966.
Figure 4. SetAll versus SetComplete. Plots of SELECTpro scores versus GDT-TS scores for T0305 models from SetAll (A) and SetComplete (B). The Pearson correlation is .641 for SetAll and .996 for SetComplete. This large difference is mainly due to the extended chain models (circled in plot A) scored by SELECTpro due to an error in our use of Modeller to generate complete models from incomplete ones.
The scores produced by SELECTpro are comparable on complete models of the same sequence. There is no standard for the handling of incomplete models and we assume that participating groups took a variety of approaches. Using only complete models ensures that the MQAP scores are calculated from the same coordinates. Thus, the models retained in SetComplete are screened first for completeness. Models missing backbone coordinates for one or more residues are removed. This leaves 14,611 models.
Structure-based MQAPs are susceptible to local structural irregularities in models, and will tend to score such models poorly. This is why methods developed to select near-native models from sets of decoys remove such models from consideration [31]. We apply additional filters (described below) for Cα-Cα clashes, Cα-Cα chain breaks, and expanded termini to remove an additional 1,217 models leaving 13,494 more plausible models in SetComplete.
The Cα-Cα clash model filter enforces a squared difference penalty for Cα-Cα distances less than 3.6 Ǻ. The distance between the Cα atoms of residue i and j is denoted by ri,Cα,j,Cα and N is the protein length. The constant 13.52 in the threshold below corresponds to two severe clashes where ri,Cα,j,Cα = 1.0 Ǻ. Models with a sum of squared differences greater than 13.52 per 100 residues are filtered out.
The Cα-Cα chain break model filter enforces a squared difference penalty for ri,Cα,i+1,Cα distances greater than 4.0 Ǻ. The constant 16.0 in the threshold below corresponds to a single chain break where ri,Cα,i+1,Cα = 8.0 Ǻ. Models with a sum of squared differences greater than 16.0 per 100 residues are filtered out.
The expanded termini filter removes models where a large portion of the structure is modeled as expanded chain with no non-local interactions. The screening procedure is: scan from the N-terminus until three consecutive residues have a contact number of at least 10, and repeat from the C-terminus. The contact number of a residue is defined here as the number of other Cβ atoms within 10 Ǻ of the residue's Cβ [3]. If the sum of low contact number termini residues is at least 20% of N, the model is filtered out.
Model Representations
Reduced representation
In the reduced representation the heavy backbone atoms, carbonyl oxygen, amide hydrogen (N, Cα, C, O, H), and Cβ are represented explicitly. For glycine residues a pseudo Cβ is calculated. The side-chain atoms are represented by a single united point (centroid) [32,33]. The centroid is calculated as the mean of the position of the heavy side-chain atoms. For glycine and alanine the centroid (CT) is set to the Cβ atom. Only the heavy backbone atoms (N, Cα, C) are used as input to SELECTpro and the positions of additional atoms and centroids are calculated from these.
All heavy-atom representation
In the all heavy-atom representation the centroid is removed and the heavy side chain atoms are represented explicitly. The side-chains are initially placed onto the backbone of the reduced representation in their most likely conformation according to the SCWRL backbone-dependent rotamer library [34]. The side-chain placements are then optimized using the SELECTpro all-atom energy terms (described below) in conjunction with the rotamer library.
Energy Functions Overview
EREDUCED is the combined energy calculated from the reduced representation. EREDUCED is a linear combination of predicted (EPRED-SS, EPRED-SA, EPRED-CM), physical (EVDW-REP), and statistical (ECT-REP, ESTAT-ENV, ESTAT-PW-CI, ESTAT-PW-CD, EROG) terms:
EALL-ATOM consists of the energy terms that depend on the all heavy-atom representation. EALL-ATOM is a linear combination of the following physical terms:
EFINAL is the sum of EREDUCED and EALL-ATOM, and is used for the final scoring of models by SELECTpro. The individual energy terms are outlined briefly below and the detailed description of the novel terms follow in the remainder of this section. Underlined terms are adapted from previously described energy terms their details are included in the Appendix.
Parameter Weights
The parameter weights were determined by repeatedly varying individual weights and maximizing the sum of the GDT-TS of the lowest EFINAL models on a training set built from CASP6 protein domains. For each CASP6 protein domain a set of 500 decoy models was generated using fragment assembly with the RMSD to native as the dominant term in the objective function [3].
EPRED-SS: predicted secondary structure
EPRED-ACC: predicted solvent accessibility
EPRED-CM: predicted contact map
EBETA: sheet formation
BB-REP: backbone repulsion
CT-REP: centroid repulsion
STAT-ENV: residue environment potential [3]
STAT-PW-CI: context independent pair-wise potential [3,16]
STAT-PW-CD: context dependent pair-wise potential [6]
ROG: compactness
ESC-HB: side-chain hydrogen bonding
LEN-JONES: van der Waals forces [10]
SOLVATION: solvation effects [35]
ELECTRO: electrostatic interactions
Throughout this work the convention of all capital letters referring to global energy and all lower case referring to local energy is used. For instance, EPRED-CM refers to the global contact map energy and Epred-cm(i,j) refers to the contact map energy between residues i and j.
Parameter notation used in energy equations
Model variables
ri,x,j,y: distance between atom x of residue i and atom y of residue j
rx,y: distance between atom x and atom y
vi,x,j,y: vector from atom x of residue i to atom y of residue j
ui,x,j,y: unit vector calculated from vi,x,j,y
Ni: number of residues in contact with residue i, with contact defined as ri,Cβ,j,Cβ < 10 Ǻ
phii: Phi angle of residue i
psii: Psi angle of residue i
Protein specific input parameters
aai: amino acid type of residue i
ssi: predicted secondary structure of residue i (H,E,C)
acci: predicted solvent accessibility of residue i ('e': exposed, '-', buried)
cmapi,j: predicted contact/non-contact between residues i and j, with contact defined as ri,Cα,j,Cα < 12 Ǻ
Protein independent parameters
Ivalue: ideal parameter value for a given calculation
σvalue: standard deviation value for a given calculation
vdwx: van der Waals radius of atom x
vdwx+y: vdwx + vdwy
Ωstat-env: pre-calculated statistics for use in ESTAT-ENV
Ωstat-pw-oi: pre-calculated statistics for use in ESTAT-PW-CI
Ωstat-pw-od: pre-calculated statistics for use in ESTAT-PW-CD
Dmin,pw-od: minimum interaction distance for centroid pairs used in ESTAT-PW-CD
Dmax,pw-od: maximum interaction distance for centroid pairs used in ESTAT-PW-CD
Dmin-CT: minimum distances between centroids of amino acid pairs observed in pdb_select25 [36].
Reduced Representation Energy Term Details
The details of how the novel reduced representation energy terms are calculated are presented in this section. The predicted structural terms EPRED-SS, EPRED-ACC, and EPRED-CM and the β-strand pairing term, EBETA, are novel and unique to SELECTpro. Additional reduced representation terms are adapted from previously published work and their details are included in the Appendix.
Predicted structural features overview
The predicted structural feature predictions used in EPRED-SS, EPRED-ACC, and EPRED-CM come from the SCRATCH suite of predictors [25]. Each predictor is trained in a supervised fashion using curated non-redundant datasets extracted from the PDB [37]. The secondary structure (SSpro [38]) and solvent accessibility (ACCpro [39]) predictors use ensembles of 1D-RNN (one dimensional-recursive neural network) architectures [40]. The contact map predictor (CMAPpro [41]) uses ensembles of 2D-RNN architectures [40].
EPRED-SS: predicted secondary structure
The predicted secondary structure term EPRED-SS penalizes deviation of the torsion angles from the torsion angle parameters for helices and strands predicted by SSpro. There is no penalty for predicted coils. The parameter values for helix residues are: IHφ = -65.3, σHφ = 11.9, IHψ = -39.4, σHψ = 11.3. The parameter values for strand residues are: IEφ = -135.0, σEφ = 15.0, IEψ = 135.0, σEψ = 15.0. Only torsion angles that are more than two standard deviations from the ideal are penalized, with the penalty defined as follows:
The definition of Epred-strand(j) is equivalent to Epred-helix(i), but with IEφ, σEφ, IEψ and σEψ in place of the corresponding helical values.
EPRED-ACC: predicted solvent accessibility
The solvent accessibility predictor ACCpro predicts the percent of solvent accessibility in 5% increments for each residue. Using 25% exposure as a binary threshold the accuracy of the predictor is ~77% [39]. The binary exposure ('e')/burial ('-') prediction is used as the predicted solvent accessibility for EPRED-ACC. In the reduced representation the solvent accessibility of residue i is estimated by its contact number (Ni), where Ni > 16 is considered buried [3]. If the predicted status of a residue is not realized in the model, the penalty is calculated as:
EPRED-CM: predicted contact map
The contact map predictor CMAPpro predicts the probability of contact or non-contact between Cα atoms, with a contact threshold of 12 Å. The strategy utilized to infer predicted contacts from the probability matrix [41] results in maps that are sparse when compared to those of real proteins; thus, unrealized contacts are penalized while non-contacts are not. The constant 1.0 is added to the penalty to ensure that all unrealized contacts make a significant contribution to EPRED-CM.
The predicted contact map can help identify the highest GDT-TS models in the set, even when they are not highly similar to native. A good example of this is CASP7 target T0304 is a 122 residue α/β protein where the highest GDT-TS model in the set is Zhang-Server_TS1 (GDT-TS = 45.55). Most secondary structure predictors (including SSpro) failed to predict the first two strands making this target especially difficult. No QA method ranked the highest GDT-TS model first; however, SELECTpro ranked it second and the model ranked first by SELECTpro (T0304.Zhang-Server_TS4) has the second highest GDT-TS. These models have the lowest EPRED-CM of any models in the set, but the native structure has an even lower EPRED-CM. Figure 5 compares the native and predicted contact maps for target T0304.
Figure 5. Contact map comparison. True contact map of target T0304 in lower left, predicted contacts upper right. Contact is defined as Cα atoms within 12 Å. For predicted contacts with a sequence separation of at least six, 651 of 915 (71%) are correct.
EBETA: strand pairing
The formation of hydrogen bonds between the residues of β-strand partners is a major determinant of the tertiary structure of β and α/β proteins. The β hydrogen bonding treatment described here favors realistic strand pairing and sheet formation. The treatment also efficiently accommodates bulges in strands because it does not force the register between two paired strands. EBETA is the global strand pairing energy that penalizes the hydrogen bonding of β residues between strand pairs. Ebeta-sp(βk→βw) is the strand pairing energy of strand βk to strand βw. Ebeta-sp is only commutative if the two strands have the same length. Ebeta-hb(i,j) is the hydrogen bonding penalty between residues i and j.
Ebeta-sp is calculated for all possible strand pairings, but only the two lowest energies from each strand are used in EBETA. Other strand-strand interactions are ignored. In the equations below S is the set of all strands in the protein, βm1 is the strand with the minimum pairing energy from βk, and βm2 is the strand with the next lowest pairing energy from βk. If the strand count is less than six at least two of the strands must be edge strands. This is accounted for by only considering the single best strand partner for two strands.
In the equations for Ebeta-sp below, Sk is the set of all residues in strand βk. Each time Ebeta-hb is calculated the pair (i,j) is chosen with i from Sk and j from Sw, such that Ebeta-hb is minimized. Then residue i is removed from Sk, and residue j is removed from Sw. Ebeta-hb is calculated once for each residue in Sk. If Sk has more residues than Sw each unpaired residue is given maximum penalty of Ebeta-hb.
Between two anti-parallel strand partners, only every other pair of residues is hydrogen bonded. For the pairs that are not hydrogen bonded, a pseudo-bonding calculation is used. The hydrogen bonding energy and pseudo-bonding energy are both calculated and the minimum of the two is used in Ebeta-hb(i,j).
If residues i and j are paired in parallel strands, either i forms hydrogen bonds with j-1 and j+1, or j forms hydrogen bonds with i-1 and i+1. No hydrogen bonds are formed between the atoms of residues i and j. The hydrogen bonding energy is calculated for both possible conformations and only the minimum of the two is used in Ebeta-hb(i,j).
Φ(a→d) is the directional energy calculation for a single hydrogen bond where a is the index of the acceptor residue and d is the index of the donor residue. Three geometrical measures are used to estimate the strength of hydrogen bonds: the distance between the acceptor and the hydrogen atoms (ra,O,d,H), the angle at the acceptor atom (ua,C,a,O · ua,O,d,H), and the angle between the acceptor and donor atom vectors (ua,C,a,O · ud,N,d,H). The distance and acceptor atom angle parameters are motivated by the orientation-dependent hydrogen bonding potential described in [42]. The following parameters were set based on idealized hydrogen bonding between β residues, with standard deviation values set such that two standard deviations approximate the cut-off in true hydrogen bonds. The ideal distance from hydrogen atom to accepting oxygen is Ihb-dist = 1.9 Ǻ, with standard deviation σhb-dist = 0.5 Ǻ. The ideal angle at the acceptor atom is 0°, so the ideal (ua,C,a,O · ua,O,d,H) is Iacc-dp = 1.0, with standard deviation σacc-dp = 0.11. The ideal angle between the acceptor and donor atom vectors is 180°, so the ideal (ua,C,a,O · ud,N,d,H) is Iacc-don-dp = -1.0, with standard deviation σacc-dp = 0.15. The parameters for pseudo-bonded residues are as follows: the ideal distance for ra,O,d,H is Ips-hb-dist = 7.9 Ǻ, Ips-acc-dp = -1.0, and Ips-acc-don-dp = -1.0. The standard deviations from the corresponding hydrogen bonding parameters above are used in Φps(a→d).
The penalty for the observed value (x) increases up to 6 standard deviations from the ideal value (μ).
All-Atom Energy Term Details
The all-atom energy terms depend on atom-atom interactions when all heavy atoms are included in the model. In the all-atoms energy equations x and y refer to atoms in the model and the residue positions are not referenced. The van der Waals radii and well-depths (εx, used in ELEN-JONES) come from the CHARMM19 parameter set [43]. The side-chain hydrogen bonding term, ESC-HB, is described in detail here because it is unique to SELECTpro. The details of ELEN-JONES, ESOLVATION, and EELECTRO are provided in the Appendix.
ESC-HB: side-chain hydrogen bonding
ESC-HB penalizes unsatisfied hydrogen bond donor and acceptor atoms that are at least partially buried. There is no penalty for fully exposed donor or acceptor atoms. Exposure percent ( %) is calculated as . The definitions of and are provided in the description of ESOLVATION in the Appendix. Atoms at least 75% exposed are considered fully exposed and atoms less than 25% exposed are considered fully buried. For 25% <% < 75% the penalty weight is reduced linearly from 1.0 at 25% to 0 at 75%. The ideal distance from the acceptor atom to donor atom is Ihb-da-dist = 2.9 Ǻ. In the equations below donors is the set of all side-chain hydrogen donor atoms and acceptors is the set of all side-chain hydrogen acceptor atoms.
In the interest of completeness and reproducibility we include the details of the energy terms that are adapted from previous work.
Reduced Representation Energy Term Details
EBB-REP: backbone repulsion
This term penalizes steric clashes between non-bonded atoms explicitly represented in the reduced representation. The penalty for overlapping atoms is the overlap distance squared as defined here:
ECT-REP: centroid repulsion
A centroid-centroid repulsive term is used to reduce the overcrowding of side-chains in the reduced representation. The minimum distance between two centroids in the calculation is the minimum observed for each pair of residue types – DCT-min(aai,aaj) – in pdb_select25. The penalty for centroid-centroid overlaps is defined as the overlap distance squared:
ESTAT-ENV: residue environment potential
The motivation for this term is to model the hydrophobic effect. The level of burial for each residue in the model is estimated by the number of other Cβ atoms within 10 Ǻ (the contact number Ni) [3]. The values in the table Ωstat-env reflect the likelihood of observing a particular Ni for each residue type. For model residues near both termini the contact number is artificially increased to account for the missing neighbors along the chain.
ESTAT-PW-CI: context independent pair-wise interactions
This context independent pair-wise potential comes from Equation 6 of [3]. The potential considers the likelihood of observing the pair of centroids in a given distance bin relative to the background, with distance bins of < 5, 5–7, 7–10, 10–12, and > 12 Å. The advantage of a context independent pair-wise potential is that it is less vulnerable to over-fitting by a conformational search because of its generality.
ESTAT-PW-CD: context dependent pair-wise potential
This context specific pair-wise potential is from [6]. This pair-wise potential depends on the local structure and relative orientation of both amino acids in the interaction. The statistics are calculated independently for each combination of local structures and relative orientations. At each position the local structure is considered either compact or open and the relative orientation is determined by the dot product of the Cα to Cβ unit vectors of each residue and divided into three classes: parallel, anti-parallel, and intermediate.
EROG: compactness
The radius of gyration is a simple measure of the global compactness of a domain. EROG penalizes models that are less compact than expected according to [44]. If the radius of gyration of the model (λ) is less than the expected value (2.2N.38), there is no penalty. If it is greater, then the penalty is the squared difference between observed and expected. In the equation below ri,mean is the distance between the Cα of residue i and the mean of all Cαs in the model.
All-Atom Energy Term Details
ELEN-JONES: van der Waals forces
A fundamental characteristic of native globular protein structures is their efficient steric packing of atoms in the protein core. A Lennard-Jones 12-6 potential with damped repulsion (ELEN-JONES) is used to measure the quality of steric packing. ELEN-JONES is the sum of local energy calculations Elen-jones(x,y) performed on all pairs of non-bonded atoms. Since the repulsive portion of the standard Lennard-Jones 12-6 potential will overwhelm the entire energy function with a single significant atom-atom clash – repulsion is handled by a linear ramp from 0 to 10 as shown in the equation below [10]. Since Elen-jones = 0 when (vdwx,y/rx,y) = independent of atom types, the switch to a linear ramp occurs when (vdwx,y/rx,y) > .
ESOLVATION: solvation effects
Solvation energy is calculated using the implicit solvation model described in [35] with the following adjustment: for overlapping atoms, the sum of their van der Waals radii is used in the calculation in place of the observed atom-atom distance in the model. This restricts the amount a single atom can contribute to the burial of another atom. Without this adjustment overlapping atoms will bias the calculation to indicate an atom is more buried than it would be otherwise. In the solvation model is the observed solvation free energy of atom x in the model, calculated as the free energy of the fully exposed atom () minus the reduction in solvation caused by the surrounding atoms. was determined empirically by setting it equal to and increasing its magnitude until of deeply buried atoms became zero. λx is the correlation length of atom x. Vy is the volume neighboring atom y. The values of these parameters come from [3535], with the exception of [45]. The equation for below is the combination of Equations 5,6, and 7 of [35], with the atom overlap adjustment.
EELECTRO: electrostatics
Electrostatic interactions between charged atoms are treated by simple repulsion and attraction according to inverse distance squared. The use of distance squared rather than linear distance encourages the formation of salt bridges in the models. There is a correction for atom-atom distance below the minimum realistic value. The ideal distance between oppositely charged atoms is Ihb-da-dist = 2.75 Ǻ. In the equations below pos is the set of all positively charged atoms and neg is the set of all negatively charged atoms.
Availability and requirements
• Project home page: http://www.igb.uci.edu/~baldig/selectpro.html webcite
• Operating system: linux for stand alone version, server is platform independent
• Programming language: C++ and Perl
• Software requirements: Perl
• Disk space requirements: 1.6 Gb for full version, 13 Mb without feature predictors
Authors' contributions
AR and PB designed the novel energy terms. AR implemented the methods and carried out the experiments. AR and PB authored the manuscript. Both authors approved the manuscript.
Work supported by NIH grant LM-07443-01, NSF grants EIA-0321390 and IIS-0513376, and a Microsoft Faculty Research Award to PFB.
Wallner B, Elofsson A: Prediction of global and local model quality in CASP7 using Pcons and ProQ.
Proteins 2007, 69(Suppl 8):184-193. PubMed Abstract | Publisher Full Text
Cozzetto D, Tramontano A: Relationship between multiple sequence alignments and quality of protein comparative models.
Proteins 2005, 58:151-157. PubMed Abstract | Publisher Full Text
Simons KT, Kooperberg C, Huang E, Baker D: Assembly of protein tertiary structures from fragments with similar local sequences using simulated annealing and Bayesian scoring functions.
J Mol Biol 1997, 268:209-225. PubMed Abstract | Publisher Full Text
Kihara D, Lu H, Kolinski A, Skolnick J: TOUCHSTONE: An ab initio protein structure prediction method that uses threading-based tertiary restraints.
Proc Natl Acad Sci USA 2001, 98:10125-10130. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Boniecki M, Rotkiewicz P, Skolnick J, Kolinski A: Protein fragment reconstruction using various modeling techniques.
J Comput Aided Mol Des 2003, 17:725-738. PubMed Abstract | Publisher Full Text
Kolinski A: Protein modeling and structure prediction with a reduced representation.
Acta Biochim Pol 2004, 51:349-371. PubMed Abstract | Publisher Full Text
Sanchez R, Sali A: Comparative protein structure modeling. Introduction and practical examples with modeller.
Methods Mol Biol 2000, 143:97-129. PubMed Abstract
Qian B, Ortiz A, Baker D: Improvement of comparative model accuracy by free-energy optimization along principal components of natural structural variation.
Proc Natl Acad Sci USA 2004, 101:15346-15351. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Lazaridis T, Karplus M: Discrimination of the native from misfolded protein models with an energy function including implicit solvation.
J Mol Biol 1999, 288:477-487. PubMed Abstract | Publisher Full Text
Kuhlman B, Baker D: Native protein sequences are close to optimal for their structures.
Proc Natl Acad Sci USA 2000, 97:10383-10388. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Vorobjev Y, Hermans J: Free energies of protein decoys provide insight into determinants of protein stability.
Protein Sci 2001, 10:2498-2506. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Felts A, Gallicchio E, Wallqvist A, Levy R: Distinguishing native conformations of proteins from decoys with an effective free energy estimator based on the OPLS all-atom force field and the Surface Generalized Born solvent model.
Proteins 2002, 48:404-422. PubMed Abstract | Publisher Full Text
Dominy B, Brooks C: Identifying native-like protein structures using physics-based potentials.
J Comput Chem 2002, 23:147-160. PubMed Abstract | Publisher Full Text
Oldziej S, Czaplewski C, Liwo A, Chinchio M, Nanias M, Vila JA, Khalili M, Arnautova YA, Jagielska A, Makowski M, Schafroth HD, Kazmierkiewicz R, Ripoll DR, Pillardy J, Saunders JA, Kang YK, Gibson KD, Scheraga HA: Physics-based protein-structure prediction using a hierarchical protocol based on the UNRES force field: Assessment in two blind tests.
Proc Natl Acad Sci USA 2005, 102:7547-7552. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Shortle D, Simons KT, Baker D: Clustering of low-energy conformations near the native structures of small proteins.
Proc Natl Acad Sci USA 1998, 95:11158-11162. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Simons KT, Ruczinski I, Kooperberg C, Fox BA, Bystroff C, Baker D: Improved recognition of native-like protein structures using a combination of sequence-dependent and sequence-independent features of proteins.
Proteins 1999, 34:82-95. PubMed Abstract | Publisher Full Text
Vendruscolo M, Najmanovich R, Domany E: Can a pairwise contact potential stabilize native protein folds against decoys obtained by threading?
Proteins 2000, 38:134-148. PubMed Abstract | Publisher Full Text
Cozzetto D, Kryshtafovych A, Ceriani M, Tramontano A: Assessment of predictions in the model quality assessment category.
Proteins 2007, 69:175-183. PubMed Abstract | Publisher Full Text
Wu S, Skolnick J, Zhang Y: Ab initio modeling of small proteins by iterative TASSER simulations.
BMC Biol 2007, 5:17. PubMed Abstract | BioMed Central Full Text | PubMed Central Full Text
Zhang 2007 Decoy Sets [http://zhang.bioinformatics.ku.edu/I-TASSER/decoys/] webcite
Zhang Y, Skolnick J: SPICKER: A clustering approach to identify near-native protein folds.
J Comput Chem 2004, 25:865-871. PubMed Abstract | Publisher Full Text
Wallner B, Fang H, Elofsson A: Automatic consensus-based fold recognition using Pcons, ProQ, and Pmodeller.
Proteins 2003, 53(Suppl 6):534-541. PubMed Abstract | Publisher Full Text
Lundstrom J, Rychlewski L, Bujnicki J, Elofsson A: Pcons: a neural-network-based consensus predictor that improves fold recognition.
Protein Sci 2001, 10:2354-2362. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Wallner B, Elofsson A: Can correct protein models be identified?
Protein Sci 2003, 12:1073-1086. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
SCRATCH Cheng J, Randall AZ, Sweredoski M, Baldi P: SCRATCH: a protein structure and structural feature prediction server.
Nucleic Acids Res 2005, (33 Web Server):W72-W76. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Moult J, Fidelis K, Kryshtafovych A, Rost B, Hubbard T, Tramontano A: Critical assessment of methods of protein structure prediction-Round VII.
Proteins 2007, 69(Suppl 8):3-9. PubMed Abstract | Publisher Full Text
Zemla A, Veclovas C, Moult J, Fidelis K: Processing and analysis of CASP3 protein structure predictions.
Proteins 1999, 37(Suppl 3):22-29. Publisher Full Text
Sali A, Blundell TL: Comparative protein modeling by satisfaction of spatial restraints.
J Mol Biol 1993, 234:779-815. PubMed Abstract | Publisher Full Text
Martin-Renom MA, Stuart A, Fiser A, Sanchez R, Melo F, Sali A: Comparative protein structure modeling of genes and genomes.
Annu Rev Biophys Biomol Struct 2000, 29:291-325. PubMed Abstract | Publisher Full Text
Fiser A, Do RK, Sali A: Modeling of loops in protein structures.
Protein Sci 2000, 9:1753-1773. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Tsai J, Bonneau R, Morozov AV, Kuhlman B, Rohl CA, Baker D: An Improved Protein Decoy Set for Testing Energy Functions for Protein Structure Prediction.
Proteins 2003, 53:76-87. PubMed Abstract | Publisher Full Text
Baker D, Bystroff C, Fletterick RJ, Agard DA: PRISM: topologically constrained phased refinement for macromolecular crystallography.
Acta Crystallogr D Biol Crystallogr 1993, 49:429-39. PubMed Abstract | Publisher Full Text
Sun S: Reduced representation approach to protein tertiary structure prediction: statistical potential and simulated annealing.
J Theor Biol 1995, 172:13-32. PubMed Abstract | Publisher Full Text
Canutescu AA, Shelenkov AA, Dunbrack RL: A graph-theory algorithm for rapid protein side-chain prediction.
Protein Sci 2003, 12:2001-2014. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Lazaridis T, Karplus M: Effective Energy Function for Proteins in Solution.
Proteins 1999, 35:133-152. PubMed Abstract | Publisher Full Text
Hobohm U, Sander C: Enlarged representative set of protein structures.
Protein Sci 1994, 3:522-524. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, Weissig H, Shindyalov IN, Bourne PE: The Protein Data Bank.
Nucleic Acids Res 2000, 28:235-242. PubMed Abstract | Publisher Full Text | PubMed Central Full Text
Pollastri G, Przybylski D, Rost B, Baldi P: Improving the prediction of protein secondary structure in three and eight classes using recurrent neural networks and profiles.
Proteins 2002, 47:228-235. PubMed Abstract | Publisher Full Text
Pollastri G, Baldi P, Fariselli P, Casadio R: Prediction of coordination number and relative solvent accessibility in proteins.
Proteins 2002, 47:142-153. PubMed Abstract | Publisher Full Text
Baldi PF, Pollastri G: The principled design of large-scale recursive neural network architectures–DAG-RNNs and the protein structure prediction problem.
J Mach Learn Res 2003, 4:575-602. Publisher Full Text
Pollastri G, Baldi P: Prediction of contact maps by GIOHMMs and recurrent neural networks using lateral propagation from all four cardinal corners.
Bioinformatics 2002, 18:S62-S70. PubMed Abstract | Publisher Full Text
Kortemme T, Morozov AV, Baker D: An orientation-dependent hydrogen bonding potential improves prediction of specificity and structure for proteins and protein-protein complexes.
J Mol Biol 2003, 326:1239-1259. PubMed Abstract | Publisher Full Text
Neria E, Fischer S, Karplus M: Simulation of activation free energies in molecular systems.
J Chem Phys 1996, 105:1902-1921. Publisher Full Text
Skolnick J, Kolinski A, Ortiz AR: MONSSTER: A method for folding globular proteins with a small number of distance restraints.
J Mol Biol 1997, 265:217-241. PubMed Abstract | Publisher Full Text
Privalov PL, Makhatadze GI: Contribution of hydration to protein folding thermodynamics II. The entropy and Gibbs energy of hydration.
J Mol Biol 1993, 232:660-679. PubMed Abstract | Publisher Full Text


text:
Monday, September 13, 2004
© Copyright 2015 Clayton News Daily
By Justin BoronDawn Robinson transformed from a meetings planner at Stanley Morgan to a missionary for God after the hi-jacked jetliner's tremendous impact hurled her against the stairwell wall in the World Trade Center. As she descended from the 61st floor on Sept. 11, 2001, she blanketed those around her with the support of her prayer, converting atheists from her office that had previously rejected her strong faith. "Even amid all of that fear, I felt the peace of God," she said. Robinson told the congregation of the Ole Country Church in McDonough Sunday how her spiritual communion with God helped appease the hysteria pervading the two buildings on that passion-filled day. Her speech sewed up a weekend of dedications, prayers, and solemn remembrances that aimed to lend credence to the phrase spawned by three-year old tragedy: "We will never forget." American flags flew over commemorative ceremonies around the nation, much like the one hoisted up from a Henry County fire engine over Ga. Highway 155, reminding passers-by of the day when the overwhelming shock of violence changed the country forever. For many people, the day not only served as a remembrance of those who died, but as an occasion to bestow gratitude on those who endow the community with protection each day. Pastor Neil Anderson of the Ole Country Church thanked Henry County's police and fire departments, calling up chiefs individually to give them leather-bound bibles inscribed with their name. "The people that watch over us, that take care of our children should be at the top of the pay scale," he said. Chief H.E. Smith of Hampton Police Department was one of the people honored at the church and gave his assurances that public safety workers will always be present in times of tragedy. "What you can always depend on is that these peple will work hand-in-hand to protect the community," he said. While inspirational hymns illuminated the atmosphere at the Ole Country Church, the cheers of supportive fans graced the Clayton County Police and Fire Departments at Morrow High School. A charity game between the two departments commemorated the day with gracious speeches from local basketball coaches and the Sheriff-elect Victor Hill. "There are many nights that I turn on the TV and see police officers and firefighters doing deeds that no one else will," said James Livingston, the head basketball coach at Morrow High School. As the wind rippled through the flag over the Ole Country Church Sunday, Lt. Kevin Gilham stood with members of the Henry County Fire Dept. from stations three, six, and seven, embracing the fellowship that has grown between public safety workers since so many of them lost their lives in New York three years ago. "Even though they don't work down here in Georgia, they are still our brothers and sisters," he said.
More like this story
- Funeral home remembers 9/11victims, heroes ( September 12, 2009 )
- Honoring Service, Sacrifice Military personnel, first responders lauded ( September 13, 2011 )
- Victims of 2001 attacks to be remembered ( September 11, 2003 )
- Clayton residents pause to pray ( May 2, 2003 )
- Sheriff moves plaque, creates tension ( August 19, 2005 )

subset: commoncrawl


text:
Articles tagged "Mike Mularkey"
Pat Shurmur's Winless Browns One Of NFL's Best Bets
Pat Shurmur's Browns are 0-5 on the field but still one of the NFL's best bets: Browns and the under has won money for you 65 percent of the time since the start of 2011.
Naughty Nurse: Jacksonville Jaguars need brain transplant
The Jacksonville Jaguars fielded the league's rushing champion last season, but it produced only a 5-11 record because the organization is too dumb to realize NFL games have always been won with superior passing attacks.
Quick Outs
- Quality Stats Super Bowl Breakdown: Patriots vs Seahawks
- Super Bowl XLIX Prop Bets: Full List
- Super Bowl 49 Game Capsule: Patriots @ Seahawks
- Awesome: Must-see Super Bowl Hall of Shame Infographic
- Referee Bill Vinovich Gets The Nod For Super Bowl 49 -- Balls
- 2014 NFL Divisional Playoffs Most-Watched Evah!
- CHFF's Pat Imig Talks Stan Kroenke & The Rams on Sportsnet 590 Toronto
- NFL 2014 Playoff Seeds and Schedule
- Cold Hard Football Facts Quality Stats: Perfect 4-0 In Wild Card Weekend
- Vegas Win Totals For Each NFL Team
Team Pages


text:
Revit, BIM, Fabrication, Plant, CAD, Misc Blog
Simple video regarding pipe fitting annotation scale within Revit MEP and benefits of turning the check box off vs. leaving the box on. Video HERE
wow! Really like this article. keep it up!
Thanks for sharing the information with all the required details using the video.
Post a Comment


text:
Home page
Site map
Complaints / Compliments
contact center:(02) 3 296 800
In the period 12 – 15 May 2011, in Ohrid, the event "Banks' Sports Days" was organized for the 13th time in a row by the Union of Financial Organizations of Macedonia (UFOM). 12 financial institutions (banks and insurance companies) – members of UFOM took part in the event. Our Bank was also part of this event and was represented by its employees – members of the Union.
Competing with maximum engagement in 13 sport disciplines, Bank's representatives were honored and had the pleasure to compete in the name of the Bank against other financial institutions.
As it was the case before, the competition was rather strong. However, the incitement and the wish for victory made Bank's representatives end on the third place with total 26 points. NLB Tutunska Banka was second with total 27 and Stopanska Banka first with total 34 points.
The first place won by the Bank was in handball – women and chess – men.
The big prize winning game of Komercijalna Banka and MasterCard® titled "Buy, give gifts from the heart and win valuable prizes" begins! On the occasion of 20 years from issuance of the first MasterCard® card of Komercijalna Banka, which, at the same time was the first MasterCard® card in the Republic of Macedonia, our clients will have the opportunity to win amazing prizes and at the same time help in humanitarian purposes.
Participation is very simple. In the period from 21.11. to 31.12.2015 with each transaction with your Maestro® and MasterCard® cards from Komercijalna Banka of at least MKD 300 at points of sale in our country you will gain one chance to participate, and with each contactless transaction of MKD 300 you will gain two chances.
The major prize is the amount of EUR 20.000 in denar equivalent. Consolation prizes comprise 20 (twenty) gift cards for the "Skopje City Mall" shopping center in the amount of MKD 5.000, 20 (twenty) gift cards in the amount of MKD 3.000 and 200 (two hundred) gift cards in the amount of MKD 1.000.
We shall mark this jubilee with humanity as well. Komercijalna Banka and MasterCard® shall donate for the needs of the Red Cross of the Republic of Macedonia from each transaction executed by you in the period of the prize winning game, thus, together with you, helping the ones who need it most. The more purchases with cards, the bigger chances to win a prize and the more donations.
Withdrawal of prizes shall be carried out on 12.01.2016, and the list of winners shall be announced on the Bank's web site.
Rules of the Prize winning game "Buy, give gifts from the heart and win valuable prizes!"
The MNT city branche was closed for renovation and on 02.11.2015 it was opened.
In this city-branch customers can also be informed and apply for consumer loans intended for citizens.
Sportsmen of Komercijalna Banka AD Skopje was awarded three plates for achieved first place.
On the occasion of the Savings Day, the Union of Finance Organizations of Macedonia organized sports games in football (men), bowling (men and women) and volleyball (mixed teams). The competitions took place in the sport center Forca. 12 banks took part in this event. Teams of Komercijalna Banka AD Skopje won three plates for the first place in the following disciplines: volleyball (mixed teams) and bowling (men) and bowling (women).
In order to transfer the knowledge and experience the employees of Komercijalna Banka to the students from the high schools of economics in the Republic of Macedonia we have commenced the Project for education of students from the schools of economics called "Practical Learning for More Knowledge".
As from today, managers and employees of the Bank shall hold several lectures on actual topics from the field of economics and banking, which are compatible with the curriculum.
The lectures of the project "Practical Learning for More Knowledge" shall start in the high school of economics Vasil Antevski Dren in Skopje, and after that they will continue in other schools in the country.
By introducing innovative products and services which are fully adjusted with the needs of customers and properly realizing the policies for sustainable development, Komercijalna Banka has remained on the course which in the past 60 years resulted in creation of real and permanent values for society as a whole. We will take care for our successors neither to start from zero nor to have any liabilities to serve any under debts, but to have a dignified task to exceed our successes. Full text
Dear competitors in the quiz,
We believe that in the period of 15 days you managed to find new and interesting information on our web site. Our aim was to make you more familiar with products and services of the Bank. We hope that we have succeeded in it. If the persistence and knowledge is your guide and condition for victory, then you have done the right thing.
Congratulations for all competitors who were with us in this period and hope that they will stay with us in future too.
Congratulations to the winners of the quiz. You can find your name and surname, nickname, number of answered questions and the time you spent for answers here.
Additionally you will be informed by phone where and when to take your prizes.
- no charges for approving and administration of all newly approved consumption loans with EUR Clause within the promotion period from 01.11.2015 until 31.01.2016. The consumption loans intended for refinancing of existing consumption loan shall be subject to charging fees and commissions in accordance with the Bank's acts. More information on the conditions related to the loan, here.
On the occasion of the savings day - 31 October, Komercijalna Banka AD Skopje shall open passbooks – Open child savings "Ververicka" (Squirrel) in the amount of MKD 5.000,00 and money saving boxes to all children that will be born on 31 October from 00:00 to 24:00 o'clock.
This will be realized in Skopje and in other cities in the country the Bank has branches or city-branches in.
Parents of the newborn babies can take the prize from the Bank against presentation of birth certificate for the newborn and copy of identity card (of one of the parents – legal custodians) documents at the Bank's counters in Skopje and throughout the country until 31.12.2015.


text:
Skip to content
Main Navigation: Current Page: /paintworks/ | /paintworks/products/bytype.htm
Search the site
Additional Site Resources
Site 'Breadcrumb' Location (Navigation)
We have plenty of kits and product accessories to choose from. Find everything
you need to get started here.
You are now leaving the site.
View All EK Success Brands