Effects of olive oil and its minor phenolic constituents on obesity-induced cardiac metabolic changes

Geovana Mx Ebaid
Fábio Rf Seiva
Katiucha Khr Rocha
Gisele A Souza
Ethel Lb Novelli
Effects of olive oil and its minor phenolic constituents on obesity-induced cardiac metabolic changes
R E S E A R C H Open Access
Background: Olive oil and its minor constituents have been recommended as important dietary therapeutic interventions in preventive medicine. However, a question remains to be addressed: what are the effects of olive oil and its phenolic compounds on obesity-induced cardiac metabolic changes? Methods: Male Wistar rats were divided into two groups (n = 24/group): (C) receiving standard-chow; (Ob) receiving hypercaloric-chow. After 21 days C and Ob groups were divided into four subgroups (n = 6/group):(C) standard-chow and saline; (C-Olive)standard-chow and olive-oil (3.0 g/kg.day); (C-Oleuropein)standard-chow and oleuropein (0.023 mg/kg/day); (C-Cafeic) standard-chow and cafeic-acid (2.66 mg/kg/day); (Ob)receiving hypercaloric-chow and saline;(Ob-Olive) hypercaloric-chow and olive-oil;(Ob-Oleuropein) hypercaloric-chow and oleuropein;(Ob-Cafeic) hypercaloric-chow and cafeic-acid. Treatments were given twice a week during 21 days. Results: After 42 days, obesity was evidenced in Ob rats from enhanced body-weight, surface-area, and bodymass-index. Energy-expenditure, oxygen consumption(VO 2 ) and fat-oxidation were lower in Ob-group than in C. Despite no morphometric changes, Ob-Olive, Ob-Oleuropein and Ob-Cafeic groups had higher VO 2 , fat-oxidation, myocardial beta-hydroxyacyl coenzyme-A dehydrogenase and lower respiratory-quotient than Ob. Citrate-synthase was highest in Ob-Olive group. Myocardial lipid-hydroperoxide(LH) and antioxidant enzymes were unaffected by olive-oil and its compounds in obesity condition, whereas LH was lower and total-antioxidant-substances were higher in C-Olive and C-Oleuropein than in C.Conclusions:The present study demonstrated for the first time that olive-oil, oleuropein and cafeic-acid enhanced fat-oxidation and optimized cardiac energy metabolism in obesity conditions. Olive oil and its phenolic compounds improved myocardial oxidative stress in standard-fed conditions.

Background

Olive oil consumption was claimed as the basic food supply responsible for the low incidence of cardiovascular diseases in some societies [1]. Recent research has suggested that olive oil minor constituents might have more effects on health than once believed [1][2][3][4][5].

On this concern, a group of olive oil phenolic compounds, such as oleuropein and cafeic acid have attracted considerable attention because of their anti-diabetic [6,7], anti-atherosclerotic [8] and anti-inflammatory [9] properties. Olive oil minor components have been also associated with the antioxidant activity of olive oil [9,10], but little information is available on cardiac oxidative stress and energy metabolism in obesity conditions. Cardiac muscle utilizes a variety of substrates to produce energy, and the heart can shift from one substrate to another depending on food intake and pathophysiological state [11]. There is a growing awareness that during energy metabolism, the mitochondrial respiratory chain represents a major intracellular source of reactive oxygen species (ROS). Alterations in food constituents or substrate for energy generation, as found in hypercaloric diet [12] may result in higher ROS, thus inducing oxidative stress, an imbalance between oxidants and antioxidants systems in favor of the former [13,14]. Recent research in our laboratory demonstrated that enhanced energy intake reduced the mitochondrial membrane fluidity, increasing ROS generation [12,15]. Furthermore, it has been shown that blocking mitochondrial oxidative phosphorylation system increases lipid accumulation in adipocytes, and that in obesity there is decreased oxygen consumption, indicating impairment in mitochondrial activity [16]. Therefore, weight gain can be viewed not only as the consequence of an initial positive energy balance, but also as the mechanism by which energy balance is eventually stored. Whether obesity-related cardiac effects are due to an increase in caloric intake, a decrease in metabolic rate, or a combination of both factors is still unclear.

An excess of energy intake and/or a decrease in energy expenditure may reflect important metabolic dysfunction, for instance exacerbated fat storage and/or a deficit in oxidative metabolism [17]. However, it was not studied whether olive oil intake can alter calorimetric parameters and can improve energy metabolism in cardiac tissue of obese rats, which certainly would bring new insights on obesity-related cardiac metabolic changes control.

Thus, the aim of this study was to investigate the effects of olive oil and its minor phenolic constituents, oleuropein and cafeic acid, on calorimetric parameters, myocardial oxidative stress and energy metabolism in cardiac tissue of control and obese rats. Animal models are useful for increasing the understanding of the effects of diet and the development of obesity related abnormalities, particularly on cardiac tissue [15,18]. Therefore, we used a rat model to study the effects of olive oil and its phenolic compounds on an unresolved issue in the field of obesity and health that is the control of obesityrelated cardiac metabolic alterations.

Methods

Obesity induction and experimental procedure

All experiments and procedures were performed in accordance with the Guide for the Care and Use of Laboratory Animals published by the US National Institute of Health and were approved by the Ethics Committee of the Botucatu School of Medicine, UNESP, Botucatu, SP, Brazil. Forty eight male Wistar rats, 75 days of age, were individually housed in polypropylene cages in an environmentally controlled clean-air room with a temperature of 22 ± 3°C, a 12 h light-dark cycle and a relative humidity of 60 ± 5%. Taking into account the hormonal influences in female, we used only male rats.

Initially the animals were randomly divided into two groups (n = 24/group). The (C) group received water ad libitum and standard rodent chow (3074 SIF, Purina Ltda., Campinas, SP, Brazil), containing (by weight) 19 The hypercaloric chow was made mixing 131.01 g sucrose, 84.77 g of soy oil, 12.33 g of cholesterol and 1.23 g cholic acid with 1000 g of a previously triturated standard chow. The dietary ingredients were homogenized in 60°C warm distilled water and the homogenate was used to prepare the pellets. Therefore, both control and hypercaloric diets were given as pellets, and there was no spillage. The proportions of dietary protein, carbohydrate and fat were determined by direct analysis [19]. Food and drinking solutions consumption were measured daily at the same time (9:00-10:00 h). The body weights were determined once a week.

In order to more appropriately study the effects of olive oil and its minor constituents on obesity, and standard fed conditions, after 21 days of dietary treatments, both C and Ob groups were then randomly divided into four subgroups (n = 6/group): (C) remained to eat standard chow, receiving saline (NaCl 0,9%); (C-Olive) remained to eat standard chow, receiving olive oil supplementation (3.0 g/kg/day) [20,21]; (C-Oleuropein) remained to eat standard chow, receiving oleuropein (0.023 mg/kg/day) [22]; (C-Cafeic) remained to eat standard chow, receiving cafeic acid (2.66 mg/kg/day) [23]; (Ob) remained to eat hypercaloric chow, receiving saline (NaCl 0,9%); (Ob-Olive) remained to eat hypercaloric chow, receiving olive oil (3.0 g/kg/day); (Ob-Oleuropein) remained to eat hypercaloric chow, receiving oleuropein (0.023 mg/kg/day); (Ob-Cafeic) remained to eat hypercaloric chow, receiving cafeic acid (2.66 mg/kg/day). Treatments were given twice a week by gavage.

Extra-virgin olive oil was chosen for its high content of phenolic compounds and was purchased from the local market (Botucatu city, São Paulo State, Brazil). The olive oil administration was the adopted dose according to human olive oil consumption, and considered that the average consumption of olive oil in humans is 4% of the diet, twice a week [24]. This value corresponded to 1 g olive oil per 25 g of consumed chow, or 3 g/kg/day olive oil for rats of C-Olive and Ob-Olive groups [21]. The olive oil was analysed for oleuropein [25] and cafeic acid [26]. The oleuropein and cafeic acid doses took into account that the amount of these compounds in the olive oil utilized was 7.78 mg/l oleuropein and 887.5 mg/l cafeic acid. Since the amount of 3 g/kg/day olive oil [20,21] was given to the rats of control group, we administered 0.023 mg/kg/day oleuropein and 2.66 mg/ kg/day cafeic acid, twice a week. Oleuropein and cafeic acid were purchased from Sigma (St. Louis, MO, USA). Food intake and caloric value of chows were used to obtain total energy intake.

Indirect calorimetry

After 21 days of dietary supplementations (42 days of the experimental period), the rats were fasted overnight (12 to 14 h) and placed into metabolic chambers (airflow = 1.0 l/min) of a computer-controlled indirect calorimeter (CWE, Inc, St. Paul, USA) to determine the calorimetric parameters. Respiratory quotient (RQ) and energy expenditure, namely resting metabolic rate (RMR) were measured using a respiratory-based software program (software MMX, CWE, Inc., USA). Average oxygen consumption (VO 2 ) and average carbon dioxide production (VCO 2 ) were integrated over periods of 15 min. Carbohydrate and fat oxidation were calculated from the no protein oxygen consumption and the amount of oxygen consumed per gram of substrate oxidized. The energy balance over 24 h was determined as energy intake minus energy expenditure [27].

Morphometric and biochemical determinations

After 42 days of experimental period, rats were fasted overnight (12 to 14 h). The animals were anaesthetized (0.1 ml ip of 1% sodium barbiturate) for measurement of body length (nose-to-anus, or nose-anal length) [19]. The body weight and body length were used to determine body mass index (BMI, body weight/body length 2 ) and surface area (body weight 0.7 ). After measuring the body length, the rats were sacrificed by decapitation. The heart was rapidly removed and weighed. The left ventricle samples of 200 mg were homogenized in 5 mL of a cold 0.1 M phosphate buffer, pH7.4. Tissue homogenates were prepared in a motor-driven Teflon-glass Potter-Elvehjem, tissue homogeniser (1 min, 100 × g). The homogenate was centrifuged at 10000 × g, for 15 min, and the supernatant was assayed for total protein [28], lipid hydroperoxide (LH) [29], antioxidant capacity, or total antioxidant substances (TAS) (test-kit Randox Laboratories, Crumlin, Co., Antrim, United Kingdom), glutathione peroxidase (GSH-Px, E.C.126.96.36.199) [30], catalase (E.C.188.8.131.52.) [31] and superoxide dismutase (SOD, E.C.188.8.131.52.) [32].

Several key metabolic enzymes were studied to reveal the possible alteration of myocardial function due obesity. The cardiac energy metabolism was assessed by lactate dehydrogenase (LDH; E.C.1.1.127.), beta-hydroxyacyl coenzyme-A dehydrogenase (OHADH; E. C.22.214.171.124.) and citrate synthase (CS; E.C.184.108.40.206.) determinations [33].

Enzyme activities were performed at 25°C using a micro-plate reader (μQuant-MQX 200 with Kcjunior software to computer system control, Bio-Tec instruments, Winooski, Vermont, USA). The spectrophotometric determinations were performed in a Pharmacia Biotech spectrophotometer with temperature-controlled cuvette chamber (UV/visible Ultrospec 5,000 with Swift II applications software to computer system control, 974213, Cambridge, England, UK).

Statistical analysis

The results are presented as means ± standard deviations (S.D). Analysis of variance for two variables (Two Way-ANOVA) was used to examine the diet and supplementation treatment effects. Significant analysis of variance results were subjected to post hoc Tukey's test. Statistical significance was set at p < 0.05 (Systat Software, USA).

Results

General characteristics of rats, nutritional and calorimetric parameters

Hypercaloric diet significantly increased final body weight, BMI and surface area. Despite no changes in food consumption, the energy intake was higher in Ob than in C rats. Ob rats had depressed VO 2, VO 2 /body weight, VO 2 /surface area, RMR, RMR/body weight and fat oxidation, as well enhanced RQ, RQ/surface area, carbohydrate oxidation and energy balance, comparing with C rats (Table 1). Olive oil, oleuropein and cafeic acid in control-fed and obese-rats had no effects on final body weight, surface area, BMI, food consumption and energy intake. C-Olive and C-Cafeic rats had higher VO 2 , VCO 2 , RMR and fat oxidation than C. VO 2 /surface area, RQ and RQ/surface area were higher in C-Olive than in C. C-Oleuropein rats had lower VO 2 , RMR and fat oxidation, as well higher RQ and RQ/surface area than C-Olive. C-Cafeic rats had higher VO 2 , RMR and fat oxidation, as well lower RQ, RQ/body weight and RQ/ surface area than C-Oleuropein (Table 2). Ob-Olive, Oboleuropein and Ob-Cafeic rats had lower RQ, RQ/body weight and RQ/surface area, as well higher fat oxidation than Ob. The VO 2 and energy balance were higher, while RMR and RMR/body weight were lower in Ob-Cafeic than in Ob, Ob-Olive and Ob-Oleuropein rats ( Table 3). The carbohydrate oxidation was not detected in C, C-Olive and C-Cafeic, as well in Ob-Olive, Ob-Oleuropein and Ob-Cafeic rats (Tables 2 and 3).

Morphometric and biochemical determinations in the heart

The heart weight, heart weight/body weight ratio, total protein, catalase, GSH-Px and LDH were unaffected by the treatments in both dietary conditions (Tables 4  and 5).

Myocardial LH was lower, and TAS was higher in C-Olive and C-Oleuropein than in C. SOD was lower in C-Cafeic than in C and C-Oleuropein groups (Table 4). Myocardial LH and TAS remained unchanged in hypercaloric fed animals (Table 5).

C-cafeic rats had the lowest OHADH activity. Ob-Olive, Ob-Oleuropein and Ob-Cafeic rats had higher OHADH than Ob (Figure 1). CS was lower in C-Oleuropein than in C, as well in Ob-Olive than in Ob group ( Figure 2).

Discussion

Dietary olive oil supplementation [20], and more recently olive oil phenols [21] have been recommended as important therapeutic interventions in preventive medicine. However, a question remains to be addressed: what are the effects of olive oil and its minor phenolic compounds on obesity-induced cardiac metabolic changes?

To the best of our knowledge this is the first study that evaluated the relative potency of olive oil and its phenolic constituents, oleuropein and cafeic acid, on some markers of metabolic pathways in cardiac tissue of obese rats, as well their relationships with calorimetric parameters and oxidative stress. The present study brought new insights into the long term and low-dose intake of olive oil and its compounds on cardiac energy metabolism.

To demonstrate the efficiency of obesity induction [19,34] despite the same food intake, Ob rats showed higher final body weight, surface area and BMI than C (Table 1). Prospective studies [35] demonstrated that a relatively low RMR and a high RQ [36] are predictors of body weight gain. In fact, as previously observed [34] obesity was characterized in Ob rats from reduced RMR, oxygen consumption, VO 2 /final body weight and VO 2 /surface area, corroborating with lower fat oxidation (Table 1). There was also higher energy balance indicating that energy intake exceeded energy expenditure for Values are given as mean ± SD. C, control rats receiving saline; C-Olive, standard-fed rats receiving olive oil; C-Oleuropein, standard-fed rats receiving oleuropein; C-Cafeic, standard-fed rats receiving cafeic acid; VO 2 oxygen consumption; VCO 2 , dioxide carbon production; RQ, respiratory quotient; RMR, resting metabolic rate. The detected fasting carbohydrate oxidation in Ob group was associated with liver glycogen storage due hypercaloric diet intake in these animals. Since the influence of the meal had essentially ended in the post absorptive state, the fasting RQ value gives an indication of the glycogen storage [27].

As previously found [20,21], in both dietary conditions there were no significant alterations in morphometric parameters by olive oil and its phenolic compounds. There were no significant changes in heart weight, heart weight/body weight, as well as in cardiac protein, indicating adequate protein supply in both dietary conditions.

The lower RMR in C-Oleuropein rats, as well the higher RMR in C-Olive and C-Cafeic rats were not enough to significantly change the energy available for fat storage ( Table 2). Note that in C-Olive and C-Cafeic rats there was a better fat utilization and this was evidenced by the lower RQ [34] in these animals. The detected carbohydrate oxidation in C-Oleuropein rats was associated with the highest RQ, RQ/body weight and RQ/surface area found in these animals ( Table 2).  Values are given as mean ± SD. C, control rats receiving saline; C-Olive, standard-fed rats receiving olive oil; C-Oleuropein, standard-fed rats receiving oleuropein; C-Cafeic, standard-fed rats receiving cafeic acid; LH, lipid hydroperoxide; TAS, total antioxidant substances; SOD, superoxide dismutase; GSH-Px, glutathione peroxidase; LDH, lactate dehydrogenase. Therefore, the lower fat oxidation in C-Oleuropein animals was not surprising, since the change in fuel selection is controlled by carbohydrate intake, and when carbohydrate oxidation rises in response to intake, there is a profound counterregulatory suppression of fat oxidation, because triacylglycerol lipase enzyme from the adipose tissue is inhibited by the insulin secretion [11]. Oleuropein enhanced glucose uptake in tissues, increasing insulin response [6] and the glycogen storage [36]. It was widely accepted that oxygen consumed for carbon dioxide liberation, during substrate oxidation depends upon the oxygen amount in dietary nutrient. Because fat oxidation requires more oxygen, there was a significant reduction in the RQ, RQ/body weight and RQ/surface area in Ob-Olive, Ob-Oleuropein and Ob-Cafeic rats (Table 3).

Several mechanisms may be associated with increased fat oxidation in obese rats receiving olive oil and its phenolic components. It is well known that absorption and delivery of dietary compounds by intestinal cells are part of a complex process, which is influenced by the physiological state of enterocytes. A persistent situation of redox imbalance, due to hypercaloric diet intake has been associated with gastrointestinal alterations [37]. Under these conditions, it might be expected an imbalance in the absorption of dietary compounds and the metabolic utilization for energy generation [38]. Olive oil and its compounds modulating cellular signal [39] and the activity of peptidases [3,40] would allow the adequate balance between uptake and metabolism of dietary compounds.

Judging from our experimental results, it was evident that the beneficial effects of olive oil and its minor constituents enhancing fat oxidation were reflected in cardiac tissue of obese rats. Note that there was no antioxidant activity of olive oil and its phenolic compounds in cardiac tissue of animals fed with hypercaloric diet.  Figure 1 Beta-hydroxyacyl coenzyme-A dehydrogenase (OHADH) activity in cardiac tissue of rats. C, control rats receiving saline; C-Olive, standard-fed rats receiving olive oil; C-Oleuropein, standard-fed rats receiving oleuropein; C-Cafeic, standard-fed rats receiving cafeic acid; Ob, Obese rats receiving saline; Ob-Olive, obese rats receiving olive oil; Ob-Oleuropein, obese rats receiving oleuropein; Ob-cafeic, obese rats receiving cafeic acid. Values are given as mean ± SD. a p < 0.05 vs. respective group. b p < 0.05 vs. respective Olive supplemented group. c p < 0.05 vs. respective Oleuropein supplemented group.

The enhanced fat oxidation was demonstrated in cardiac tissue by higher OHADH and CS activities in Ob-Olive rats (Figure 1 and 2). OHADH is a key enzyme for fatty acid oxidation, and CS is the key enzyme for the control of the flux of metabolites through tricarboxylic acid cycle. A clear link between triacylglycerol accumulation and the cardiomyopathy was established in experimental models in which the rate of fatty acid uptake by the heart was increased, or the capacity for fatty acid oxidation was reduced in the mitochondria [41]. Myocardial OHADH activity was also significantly enhanced by oleuropein and cafeic acid in obese animals, despite the maintenance of CS activity in these animals. It has been shown that hypercaloric diet induces adverse effects on cardiac function through changes in fatty acid metabolism, by inappropriate activation and expression of PPARα (peroxisome-proliferator-activated receptor α) as a result of glucotoxicity. The decrease in the transcription of PPARα may be a regulatory event for the reduced use of fatty acids [42]. Thus, cafeic acid and oleuropein could allow adequate PPARα activation and fatty acid utilization, increasing the OHADH activity under obesity conditions, reducing triacylglycerol accumulation in cardiac muscle. Optimizing cardiac energy metabolism in obese conditions may be one approach to prevent and treat cardiac dysfunction [11,13].

Curiously, no significant changes were found in myocardial metabolic enzymes of C-Olive rats, whereas, CS was significantly reduced in C-Oleuropein rats, despite no changes in OHADH (Figure 1 and 2). This indicated enhanced glycolytic pathway relative to aerobic metabolism, or delayed flux of metabolites through tricarboxylic acid cycle in cardiac tissue. Glycolysis preferentially serves energy channeling to sarcolemmal membranes, were glucose transport into cells occurs, by providing this readily available substrate for glycolytic enzymes bound to sarcolemmal molecular complexes [13]. Glycolytic pathway thus represents low capacity, but high specificity modules of the integrated metabolic network of a cardiac myocite [11]. On the other hand, the reduced OHADH, and the maintenance of CS activity, clearly indicated that C-Cafeic rats had depressed fatty acid degradation, relative to aerobic metabolism in cardiac tissue, as compared to C, C-Olive and C-Oleuropein rats. Therefore, in standard fed conditions, dietary supplementation with olive oil phenolic compounds induced changes in the substrate used for energy generation in cardiac tissue. Further studies may be considered to clear this open question, and to show the importance of these changes on cardiac function.

Anyhow, the beneficial effect of olive oil in standardfed condition was evidenced from the reduced myocardial LH in these animals (Table 4). Lipids accumulation within the myocardium induces cardiac lipotoxicity [12,13]. Oxidative stress and lipotoxicity with increased LH may affect myocardial function, in a fashion that mimics reperfusion injury including persistent cellular loss of K + , depletion of energy phosphates and decreased metabolic function [43]. Note that despite the effects of cafeic acid reducing myocardial SOD, no significant changes were found in LH and TAS concentrations as compared with C rats.

Although both oleuropein and olive oil have reduced LH, the antioxidant responses in both groups occurred by different ways. The antioxidant defense system includes SOD that catalyzes the destruction of superoxide radical (O 2 _ ) by dismutation and hydrogen peroxide formation, catalase and GSH-peroxidase that catalyzes the conversion of hydrogen peroxide to water. The total antioxidant substances (TAS) include non-enzymatic, lipophilic and aqueous antioxidants [11]. Enhanced TAS was associated with direct antioxidant activity of olive oil and oleuropein, because of the free radical scavenger Figure 2 Citrate synthase (CS) activity in cardiac tissue of rats. C, control rats receiving saline; C-Olive, standard-fed rats receiving olive oil; C-Oleuropein, standard-fed rats receiving oleuropein; C-Cafeic, standard-fed rats receiving cafeic acid; Ob, Obese rats receiving saline; Ob-Olive, obese rats receiving olive oil; Ob-Oleuropein, obese rats receiving oleuropein; Ob-cafeic, obese rats receiving cafeic acid. Values are given as mean ± SD. a p < 0.05 vs. respective group. b p < 0.05 vs. respective Olive supplemented group. c p < 0.05 vs. respective Oleuropein supplemented group.

ability [44]. This property depends on the number of hydroxyl radicals in the phenolic molecule [21]. Oleuropein and hydroxytyrosol, free radical scavengers, have antioxidant activity as found in other antioxidants, such as vitamin E and butylated hydroxytoluene (BHT) [20]. On the other hand, it has been demonstrated that olive oil may interfere in iron absorption [24]. Considering the importance of iron in the oxidative stress induction, the reduced iron uptake and diminished serum concentration of iron may be alternative mechanism for reduced myocardial LH in C-Olive animals.

Conclusions

In conclusion, the present study brought new insights of olive oil effects on obesity related cardiac metabolic changes, demonstrating for the first time that olive oil enhanced fat oxidation and regulated myocardial metabolic enzymes, optimizing cardiac energy metabolism in obesity conditions. Olive oil and its minor phenolic compounds, oleuropein and cafeic acid had myocardial antioxidant activity in standard-fed conditions.

a p < 0.05 vs. C group. b p < 0.05 vs. C-Olive group. c p < 0.05 vs. C-Oleuropein group. functions (RMR), thus providing greater amount of energy to be stored as fat.

a p < 0.05 vs. C group. b p < 0.05 vs. C-Olive group. c p < 0.05 vs. C-Oleuropein group.

.80% protein, 39.25% carbohydrate, 4.41% fat, 13.25% fibre, and 2.76 kcal/g of metabolizable energy. The (Ob) group received a hypercaloric chow containing 15.25% protein, 43.34% carbohydrate, 11.86% fat, 10.20% fibre (by weight) and 3.41 kcal/g of metabolizable energy.

Table 1
1General characteristics and calorimetric parameters of rats after 42 days of standard fed (C), and hypercaloric fed (Ob) dietsValues are given as mean ± SD. C, control rats; Ob, obese rats; BMI, body mass index; VO 2 oxygen consumption; VCO 2 dioxide carbon production; RQ, respiratory quotient; RMR, resting metabolic rate.Groups

Table 2
2Calorimetric parameters of standard-fed ratsGroups

Table 3
3Calorimetric parameters of obese ratsGroups

Table 4
4Morphometric and biochemical determinations in the heart of standard fed ratsGroups

Table 5
5Morphometric and biochemical determinations in the heart of obese ratsGroups

AcknowledgementsThe authors inform that there are no conflicts of interest. The authors thank the financial support from FAPESP, Fundação de Amparo a Pesquisa do Estado de São Paulo and CNPq, Conselho Nacional de Desenvolvimento Científico e Tecnológico.Authors' contributions GMXE took part in planning the study design, sample analysis, analyzed the data and involved in drafting and writing of manuscript. FRFS contributed to the study design, sample analysis and performed the statistical analysis. KKHRR and GSA involved in drafting and sample analysis. ELBN took part in planning and supervising the study design, contributed to the analysis, writing of manuscript, edited the paper, revising it critically for important intellectual content and provided the final version. All authors read and approved the final manuscript.Competing interestsThe authors declare that they have no competing interests.
Bioavailability and antioxidant effects of olive oil phenolic compounds in humans: a review. M Fitó, R De La Torre, M F Albaladejo, O Khymenetz, J Marrugat, M I Covas, Ann ist Super Sanitá. 43Fitó M, De la Torre R, Albaladejo MF, Khymenetz O, Marrugat J, Covas MI: Bioavailability and antioxidant effects of olive oil phenolic compounds in humans: a review. Ann ist Super Sanitá 2007, 43:375-381.

Polyunsaturated fatty acids, inflammation and obesityrelated disease. L M Browning, Proceedings Nutrition Society. 62Browning LM: n-3 Polyunsaturated fatty acids, inflammation and obesity- related disease. Proceedings Nutrition Society 2003, 62:447-453.

Involvement of oleuropein in (some) digestive metabolic pathways. V Polzonetti, D Egidi, A Vita, S Vincenzetti, P Natalini, Food Chem. 88Polzonetti V, Egidi D, Vita A, Vincenzetti S, Natalini P: Involvement of oleuropein in (some) digestive metabolic pathways. Food Chem 2004, 88:11-15.

Virgin olive oil Study (VOLOS): vasoprotective potential of extra virgin olive oil in mildly dyslipidemic patients. F Visioli, D Caruso, S Grande, F Bosisio, M Villa, G Galli, C Sirtori, C Galli, Eur J Nutr. 44Visioli F, Caruso D, Grande S, Bosisio F, Villa M, Galli G, Sirtori C, Galli C: Virgin olive oil Study (VOLOS): vasoprotective potential of extra virgin olive oil in mildly dyslipidemic patients. Eur J Nutr 2005, 44:121-127.

The effect of polyphenols in olive oil on heart disease risk factors: a randomized trial. M I Covas, K Nyyssonen, H E Poulsen, J Kaikkonen, H J Zunft, H Kiesewetter, Eurolive Study Group, Ann Intern Med. 145Covas MI, Nyyssonen K, Poulsen HE, Kaikkonen J, Zunft HJ, Kiesewetter H, EUROLIVE Study Group, et al: The effect of polyphenols in olive oil on heart disease risk factors: a randomized trial. Ann Intern Med 2006, 145:333-341.

Hypoglycemic and antioxidant effect of oeluropein in alloxan-dibetic rabbits. H F Al-Azzaweie, M Saiel, S Alhamdani, Life Scienc. 78Al-Azzaweie HF, Saiel M, Alhamdani S: Hypoglycemic and antioxidant effect of oeluropein in alloxan-dibetic rabbits. Life Scienc 2006, 78:1371-1377.

Antidiabetic and antioxidant effects of hydroxytyrosol and oleuropein from olive leaves in alloxan-diabetic rats. H Jemai, A El Feki, S Sayadi, J Agric Food Chem. 57Jemai H, El Feki A, Sayadi S: Antidiabetic and antioxidant effects of hydroxytyrosol and oleuropein from olive leaves in alloxan-diabetic rats. J Agric Food Chem 2009, 57:8798-8804.

Differential anti-inflammatory effects of phenolic compounds from extra virgin olive oil identified in human whole blood cultures. E A Miles, P Zoubouli, P C Calder, Nutrition. 21Miles EA, Zoubouli P, Calder PC: Differential anti-inflammatory effects of phenolic compounds from extra virgin olive oil identified in human whole blood cultures. Nutrition 2005, 21:389-394.

Bioactive effects of olive oil phenolic compounds in humans: reduction of herat disease factors and oxidative damage. M I Covas, Inflammopharmacology. 16Covas MI: Bioactive effects of olive oil phenolic compounds in humans: reduction of herat disease factors and oxidative damage. Inflammopharmacology 2008, 16:216-218.

Active components and clinical applications of olive oil. E Waterman, B Lockwood, Alternative Med Review. 12Waterman E, Lockwood B: Active components and clinical applications of olive oil. Alternative Med Review 2007, 12:331-342.

Nutrição e Vida Saudável -estresse oxidativo e metabolismo energético. Elb Novelli, Ribeirão PretoNovelli ELB: Nutrição e Vida Saudável -estresse oxidativo e metabolismo energético. Ribeirão Preto: Tecmed 2005.

N-acetylcysteine in high sucrose induced obesity: energy expenditure. metabolic shifting for cardiac health. Elb Novelli, P Santos, H Assalin, G Souza, K Rocha, G Ebaid, F Seiva, F Mani, A Fernandes, Pharmacol Res. 59Novelli ELB, Santos P, Assalin H, Souza G, Rocha K, Ebaid G, Seiva F, Mani F, Fernandes A: N-acetylcysteine in high sucrose induced obesity: energy expenditure. metabolic shifting for cardiac health. Pharmacol Res 2009, 59:74-79.

Cardiac energy metabolism in obesity. G D Lopaschuk, Cdl Folmes, W C Stanley, Cir Res. 101Lopaschuk GD, Folmes CDL, Stanley WC: Cardiac energy metabolism in obesity. Cir Res 2007, 101:335-347.

Effects of digitonin on hyperglycaemia and dyslipidemia induced by high-sucrose intake. Gmx Ebaid, L A Faine, Y S Diniz, H G Rodrigues, Elb Novelli, Food Chem Toxicol. 44Ebaid GMX, Faine LA, Diniz YS, Rodrigues HG, Novelli ELB: Effects of digitonin on hyperglycaemia and dyslipidemia induced by high-sucrose intake. Food Chem Toxicol 2006, 44:293-299.

Diet compounds. glycemic index and obesity-related cardiac effects. Y S Diniz, R C Burneiko, Frf Seiva, F Q Almeida, C M Galhardi, Novelli Filho, Jlvb Mani, F Novelli, Elb , Int J Cardiology. 124Diniz YS, Burneiko RC, Seiva FRF, Almeida FQ, Galhardi CM, Novelli Filho JLVB, Mani F, Novelli ELB: Diet compounds. glycemic index and obesity-related cardiac effects. Int J Cardiology 2008, 124:592-599.

Mitochondrial dysfunction induces triglyceride accumulation in 3T3-L1 cells: role of fatty acid beta-oxidation and glucose. S Vankoningsloo, M Piens, C Lecocq, A Gilson, De Pauw, A , Renard P , J Lipid Res. 46Vankoningsloo S, Piens M, Lecocq C, Gilson A, De Pauw A, Renard P: Mitochondrial dysfunction induces triglyceride accumulation in 3T3-L1 cells: role of fatty acid beta-oxidation and glucose. J Lipid Res 2005, 46:1133-1149.

Western diet, but not high fat diet causes derangements of fatty acid metabolism and contractile dysfunction in the heart of Wistar rats. C R Wilson, M K Tran, K L Salazar, M E Young, H Taegtmeyer, Biochem J. 406Wilson CR, Tran MK, Salazar KL, Young ME, Taegtmeyer H: Western diet, but not high fat diet causes derangements of fatty acid metabolism and contractile dysfunction in the heart of Wistar rats. Biochem J 2007, 406:457-467.

Sucrose-induced obesity impairs novel object recognition learning in young rats. N Jurdak, R B Kanarek, Physiol Behav. Jurdak N, Kanarek RB: Sucrose-induced obesity impairs novel object recognition learning in young rats. Physiol Behav 2009, 961-5.

N-acetylcysteine na Allium plant compound improves highsucrose diet-induced obesity and related effects. G A Souza, G X Ebaid, Frf Seiva, Khr Rocha, C M Galhardi, F Mani, Elb Novelli, eCAM Advance Acess. 11Souza GA, Ebaid GX, Seiva FRF, Rocha KHR, Galhardi CM, Mani F, Novelli ELB: N-acetylcysteine na Allium plant compound improves high- sucrose diet-induced obesity and related effects. eCAM Advance Acess 2008, 11:1-7.

Synergistic action of olive oil supplementation and dietary restriction on serum lipids and cardiac antioxidant defences. Can. L A Faine, Y S Diniz, C M Galhardi, H G Rodrigues, R C Burneiko, L S Santana, A C Cicogna, Elb Novelli, J. Physiol. Pharmacol. 8211Faine LA, Diniz YS, Galhardi CM, Rodrigues HG, Burneiko RC, Santana LS, Cicogna AC, Novelli ELB: Synergistic action of olive oil supplementation and dietary restriction on serum lipids and cardiac antioxidant defences. Can. J. Physiol. Pharmacol 2004, 82(11):969-975.

Effects of olive oil and its minor constituents on serum lipids. oxidative stress. and energy metabolism in cardiac muscle. L A Faine, H G Rodrigues, C M Galhardi, Gmx Ebaid, Y S Diniz, C R Padovani, Elb Novelli, Can J Physiol Pharmacol. 84Faine LA, Rodrigues HG, Galhardi CM, Ebaid GMX, Diniz YS, Padovani CR, Novelli ELB: Effects of olive oil and its minor constituents on serum lipids. oxidative stress. and energy metabolism in cardiac muscle. Can J Physiol Pharmacol 2006, 84:239-245.

Phenolic compounds profile of cornicabra virgin olive oil. S Gómez-Alonso, M D Salvador, G Fregapane, J Agric Food Chem. 50Gómez-Alonso S, Salvador MD, Fregapane G: Phenolic compounds profile of cornicabra virgin olive oil. J Agric Food Chem 2002, 50:6812-6817.

Simple and hydrolyzable phenolic compounds in virgin olive oil: Their extraction. separation. and quantitative and semiquantitative evaluation by HPLC. G F Montedoro, M Servili, M Baldioli, E Miniati, J Agric Food Chem. 40Montedoro GF, Servili M, Baldioli M, Miniati E: Simple and hydrolyzable phenolic compounds in virgin olive oil: Their extraction. separation. and quantitative and semiquantitative evaluation by HPLC. J Agric Food Chem 1992, 40:1571-8.

Dietary supplementation with olive oil influences iron concentrations in rats. K N Chetty, R Conway, K C Harris, D Hill, S Chetty, S Jain, Nutr Res. 19Chetty KN, Conway R, Harris KC, Hill D, Chetty S, Jain S: Dietary supplementation with olive oil influences iron concentrations in rats. Nutr Res 1999, 19:1665-1670.

A model for enhanced pea seedling vigor following low pH and salicylic acid treatments. P Mccue, Z Zheng, J L Pinkham, K Shetty, Process Biochem. 35McCue P, Zheng Z, Pinkham JL, Shetty K: A model for enhanced pea seedling vigor following low pH and salicylic acid treatments. Process Biochem 2000, 35:603-613.

Enhancement of seed vigour following insecticide and phenolic elicitor treatment. A Horii, P Mccue, K Shetty, Bioresearch Technology. 98Horii A, McCue P, Shetty K: Enhancement of seed vigour following insecticide and phenolic elicitor treatment. Bioresearch Technology 2006, 98:623-32.

Nutrient oxidation and metabolic rate as affected by meals containing different proportions of carbohydrate and fat. in healthy young women. I Labayen, L Forga, J A Martinez, Eur J Nutr. 38Labayen I, Forga L, Martinez JA: Nutrient oxidation and metabolic rate as affected by meals containing different proportions of carbohydrate and fat. in healthy young women. Eur J Nutr 1999, 38:158-66.

. Ebaid, Nutrition Journal. 946Ebaid et al. Nutrition Journal 2010, 9:46

Protein measurement with the folinphenol reagent. O H Lowry, N L Rosembrough, A L Farr, J Biol Chem. 193Lowry OH, Rosembrough NL, Farr AL: Protein measurement with the folin- phenol reagent. J Biol Chem 1951, 193:265-275.

Lipid hydroperoxide measurement by oxidation of Fe2+ in the presence of xylenol orange. Comparison with TBA assay and on iodometric method. Z Y Jiang, A C Woollard, S Wolf, Lipids. 26Jiang ZY, Woollard AC, Wolf S: Lipid hydroperoxide measurement by oxidation of Fe2+ in the presence of xylenol orange. Comparison with TBA assay and on iodometric method. Lipids 1991, 26:853-56.

Purification and properties of rat liver glutathione peroxidase. W Nakamura, S Hojoda, K Hayashi, Biochem Biophys Acta. 358Nakamura W, Hojoda S, Hayashi K: Purification and properties of rat liver glutathione peroxidase. Biochem Biophys Acta 1974, 358:251-261.

Methods of enzymatic analysis. H Aebi, Bergmeyer H. V2New YorkAebi H: Methods of enzymatic analysis.Edited by: Verloy Chemia Weinheim. (New York):Bergmeyer H. V; , 2 1974:1-4.

Microplate superoxide dismutase assay employing a nonenzymatic superoxide generation. J F Ewing, Janero Dr, Anal Chem. 232Ewing JF, Janero Dr: Microplate superoxide dismutase assay employing a nonenzymatic superoxide generation. Anal Chem 1995, 232:243-248.

Metabolic differentiation of distinct muscle types at the level of enzymatic organization. A Bass, P E Brdicka, P Eyer, S Hofer, D Pette, Eur J Biochem. 10Bass A, Brdicka PE, Eyer P, Hofer S, Pette D: Metabolic differentiation of distinct muscle types at the level of enzymatic organization. Eur J Biochem 1969, 10:198-206.

Energy expenditure and oxygen consumption as novel biomarkers of obesity-induced cardiac disease in rats. Elb Novelli, G A Souza, Gmx Ebaid, Kkhr Rocha, Frf Seiva, F Mani, K E Campos, J M Sforcin, Obesity. 18Novelli ELB, Souza GA, Ebaid GMX, Rocha KKHR, Seiva FRF, Mani F, Campos KE, Sforcin JM: Energy expenditure and oxygen consumption as novel biomarkers of obesity-induced cardiac disease in rats. Obesity 2010, 18:1-8.

A low resting metabolic rate is associated to body weight gain in adult Caucasian subjects: preliminary results of a 8-10 year longitudinal study. S Buscemi, Di Magio, O Blunda, G Maneri, R Verga, S Bompiani, G D , Int J Obesity. 2275Suppl 1Buscemi S, Di Magio O, Blunda G, Maneri R, Verga S, Bompiani GD: A low resting metabolic rate is associated to body weight gain in adult Caucasian subjects: preliminary results of a 8-10 year longitudinal study. Int J Obesity 1998, 22(Suppl 1):S75.

Energy expenditure. fat oxidation. and body weight regulation: A metabolic adaptation to long-term weight change. C Weyer, R E Pratley, A D Salbe, C Bogardua, E Ravussin, P A Tataranni, J Cin Endocrinol Metab. 85Weyer C, Pratley RE, Salbe AD, Bogardua C, Ravussin E, Tataranni PA: Energy expenditure. fat oxidation. and body weight regulation: A metabolic adaptation to long-term weight change. J Cin Endocrinol Metab 2000, 85:1087-1094.

Resveratrol toxicity: effects on risk factors for atherosclerosis and hepatic oxidative stress in standard and high-fat diets. Kkhr Rocha, G A Souza, Gmx Ebaid, Frf Seiva, A C Cataneo, Elb Novelli, Food Chem Toxicol. 47Rocha KKHR, Souza GA, Ebaid GMX, Seiva FRF, Cataneo AC: Novelli ELB. Resveratrol toxicity: effects on risk factors for atherosclerosis and hepatic oxidative stress in standard and high-fat diets. Food Chem Toxicol 2009, 47:1362-1367.

Redox-dependent modulation of lipid synthesis induced by oleic acid in the human intestinal epithelial cell line caco-2. R Rivabene, M Napolitano, A Cantafora, E Bravo, Exp. Biol. Med. 2261Rivabene R, Napolitano M, Cantafora A, Bravo E: Redox-dependent modulation of lipid synthesis induced by oleic acid in the human intestinal epithelial cell line caco-2. Exp. Biol. Med 2001, 226(1):191-198.

Olive oil modulation of cell signaling in disease prevention. K W Wahle, D Caruso, J J Ochoa, J L Quiles, Lipids. 39Wahle KW, Caruso D, Ochoa JJ, Quiles JL: Olive oil modulation of cell signaling in disease prevention. Lipids 2004, 39:1223-1231.

SIRT1 Regulates hepatocyte lipid metabolism through activating AMP-activated protein kinase. X Hou, S Xu, K A Maitland, K Sato, B Jiang, Y Ido, F Lan, K Wals, M Wierzbicki, T J Verbeuren, R A Cohen, M Zang, J Biol Chem. 283Hou X, Xu S, Maitland KA, Sato K, Jiang B, Ido Y, Lan F, Wals K, Wierzbicki M, Verbeuren TJ, Cohen RA, Zang M: SIRT1 Regulates hepatocyte lipid metabolism through activating AMP-activated protein kinase. J Biol Chem 2009, 283:20015-26.

Regulation of myocardial triacylglycerol synthesis and metabolism. T M Lewin, R A Coleman, Biochim Biophys Acta. 1634Lewin TM, Coleman RA: Regulation of myocardial triacylglycerol synthesis and metabolism. Biochim Biophys Acta 2003, 1634:63-75.

Mitochondrial remodelling in adipose tissue associated with obesity and treatment with rosiglitazone. L Wilson-Fritch, S Nicoloro, M Choulnard, M A Lazar, P C Chui, J Reszyk, J Clin Invest. 114Wilson-Fritch L, Nicoloro S, Choulnard M, Lazar MA, Chui PC, Reszyk J: Mitochondrial remodelling in adipose tissue associated with obesity and treatment with rosiglitazone. J Clin Invest 2004, 114:1281-89.

Metabolic cardioprotection by pyruvate: recent progress. R T Mallet, J Sun, E M Knott, A B Sharma, Aho Yurvati, Exp Biol Med. 230Mallet RT, Sun J, Knott EM, Sharma AB, Yurvati AHO: Metabolic cardioprotection by pyruvate: recent progress. Exp Biol Med 2005, 230:435-43.

Protective and antioxidant role of caffeic acid against chloramphenicol induced hepatotoxicity in rats. L Pari, K Velmurugan, 10.1186/1475-2891-9-46Ad Phytother Res. 37Pari L, Velmurugan K: Protective and antioxidant role of caffeic acid against chloramphenicol induced hepatotoxicity in rats. Ad Phytother Res 2009, 37:79-89. doi:10.1186/1475-2891-9-46

Effects of olive oil and its minor phenolic constituents on obesity-induced cardiac metabolic changes. Ebaid, Nutrition Journal. 946Cite this article asCite this article as: Ebaid et al.: Effects of olive oil and its minor phenolic constituents on obesity-induced cardiac metabolic changes. Nutrition Journal 2010 9:46.


Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments

James H Bullard
Elizabeth Purdom
†
Kasper D Hansen
Sandrine Dudoit
Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments
R E S E A R C H A R T I C L E Open Access
Background: High-throughput sequencing technologies, such as the Illumina Genome Analyzer, are powerful new tools for investigating a wide range of biological and medical questions. Statistical and computational methods are key for drawing meaningful and accurate conclusions from the massive and complex datasets generated by the sequencers. We provide a detailed evaluation of statistical methods for normalization and differential expression (DE) analysis of Illumina transcriptome sequencing (mRNA-Seq) data. Results: We compare statistical methods for detecting genes that are significantly DE between two types of biological samples and find that there are substantial differences in how the test statistics handle low-count genes. We evaluate how DE results are affected by features of the sequencing platform, such as, varying gene lengths, base-calling calibration method (with and without phi X control lane), and flow-cell/library preparation effects. We investigate the impact of the read count normalization method on DE results and show that the standard approach of scaling by total lane counts (e.g., RPKM) can bias estimates of DE. We propose more general quantilebased normalization procedures and demonstrate an improvement in DE detection. Conclusions: Our results have significant practical and methodological implications for the design and analysis of mRNA-Seq experiments. They highlight the importance of appropriate statistical methods for normalization and DE inference, to account for features of the sequencing platform that could impact the accuracy of results. They also reveal the need for further research in the development of statistical and computational methods for mRNA-Seq.

Background

For the past decade, microarrays have been the assays of choice for high-throughput studies of gene expression. Recent improvements in the efficiency, quality, and cost of genome-wide sequencing have prompted biologists to rapidly abandon microarrays in favor of ultra highthroughput sequencing, a.k.a., second-generation or next-generation sequencing: e.g., Applied Biosystems' SOLiD, Helicos BioSciences' HeliScope, Illumina's Genome Analyzer, and Roche's 454 Life Sciences sequencing systems. These high-throughput sequencing technologies have already been applied to monitor genome-wide transcription levels (mRNA-Seq), DNA-protein interactions (ChIP-Seq), chromatin structure, and DNA methylation [1][2][3][4][5][6][7][8][9].

We evaluate statistical methods for the inference of differential expression (DE) with mRNA-Seq, using reference samples from the MicroArray Quality Control (MAQC) Project [10]. With corresponding quantitative real-time polymerase chain reaction (qRT-PCR) data on roughly one thousand genes, we compare different normalization and DE procedures and assess possible biases related to the sequencing technology. For genes that are well-expressed in both samples being compared, the examined tests (Fisher's exact test and GLM-based tests) are indistinguishable. However, substantial differences exist in their ability to give reliable DE estimates when even just one of the samples yields low read counts (e.g., ≤ 10). One inherent bias of the Illumina platform is the preferential sequencing of longer genes [11]. With the tests considered here, longer genes are more likely declared DE. We demonstrate that weighting the DE statistics by gene length can mitigate this effect.

While small "nuisance" technical effects can be observed due to differences in flow-cells/library preparations, we show that these do not impact substantially the differential expression calls for the MAQC dataset. We also find that not using the standard phi X control lane in each flow-cell, as in the base-calling calibration procedure recommended by Illumina, does not negatively impact DE detection. Moreover, auto-calibration without the phi X lane increases both quantity and quality of mapped reads. In this regard, there is no obvious benefit in using a phi X lane; doing away with such a control lane leads to more balanced and cost-effective designs.

We demonstrate that the greatest impact on DE detection is the choice of normalization procedure. As different lanes have different total read counts, i.e., sequencing depths, the usual approach is to scale gene counts within each lane by the total lane count: e.g., the now standard reads per kilobase of exon model per million mapped reads (RPKM) of [7] or the hypergeometric model of [6]. We show that this form of global normalization is heavily affected by a relatively small proportion of highly-expressed genes and, as such, can give biased estimates of DE if these few genes are differentially expressed across the conditions under comparison. We propose alternative more robust quantile-based normalization procedures that remove the bias without introducing additional noise.

Methods

MAQC datasets

This article considers two mRNA-Seq datasets related to the MicroArray Quality Control Project [10] and obtained using Illumina's Genome Analyzer II highthroughput sequencing system [12]. The experiments analyze two biological samples: Ambion's human brain reference RNA and Stratagene's human universal reference RNA, herein referred to as Brain and UHR, respectively.

In the first experiment (MAQC-2), two types of biological samples (Brain and UHR) were assayed, each using seven lanes distributed across two flow-cells. One library preparation was used for each of the two types of biological samples. Thus, biological effects are confounded with library preparation effects, i.e., some differences in mRNA-Seq measures between Brain and UHR could be due only to experimental artifacts. In the second experiment (MAQC-3), four different UHR library preparations were assayed using 14 lanes from two flow-cells; each library preparation was assayed on only one of the flow-cells. Thus, library preparation effects are nested within flow-cell effects and differences between flowcells are confounded with library preparation effects (see [Additional file 1: Supplemental Figure S1] for the experimental design). Sequencing reads from both MAQC-2 and MAQC-3 experiments have been deposited to the short-read archive under the accession number, SRA010153.1.

As part of the original MAQC Project, around one thousand genes were also chosen to be assayed by qRT-PCR [13]. We use these qRT-PCR data as a goldstandard to benchmark the gene expression values determined by mRNA-Seq. Additionally, a large number of microarray experiments were conducted. We compare the mRNA-Seq measures to those derived from a set of Affymetrix Human Genome U133 Plus 2.0 arrays (GSE5350, samples AFX_1_ [A-B] [1][2][3][4][5]; see [Additional file 2: Supplemental Sections S1.2 and S1.2] for details on qRT-PCR and array analysis).

Overview of the Illumina sequencing platform

We give a brief, non-technical overview of the steps involved in an Illumina mRNA-Seq experiment [12]. A sample of interest undergoes library preparation, a series of steps to convert the input RNA into small fragments of DNA that can be sequenced by the Illumina machine. Specifically, starting with any total RNA sample, Illumina's mRNA-Seq library preparation protocol includes poly-A RNA isolation, RNA fragmentation, reverse transcription to cDNA using random primers, adapter ligation, size-selection from a gel, and PCR enrichment [ [14], Figure six]. The resulting cDNA library is placed in one of the eight lanes of a flow-cell. Individual cDNA fragments attach to the surface of the lane and subsequently undergo an amplification step, whereby they are converted into clusters of double-stranded DNA. The flow-cell is then placed in the sequencing machine, where each cluster is sequenced in parallel. Specifically, at each cycle, the four fluorescently labeled nucleotides are added and the signals emitted at each cluster recorded. For each flow-cell, this process is repeated for a given number of cycles, e.g., 35 cycles in the MAQC experiments. The fluorescence intensities are then converted into base-calls. The number of cycles determines the length of the reads; the number of clusters determines the number of reads.

Pre-processing of sequencing data

For the two MAQC experiments, 35 base-pair-long reads were obtained using Illumina's standard Genome Analyzer pre-processing pipeline, Version 1.3 [12,15]. We used Bowtie to map reads to the genome (GRCh37 assembly) [16].

Illumina's default base-calling algorithm, Bustard, can be calibrated in two ways. The method recommended by Illumina is to reserve one lane per flow-cell for sequencing DNA (typically phi X DNA) and use data from this control lane to determine base-calls and quality scores for the other seven lanes [ [15], Supplementary Information, p. 7]. Bustard can also be run using the auto-calibration method, which scores basecalls in a manner similar to the phred base-caller [17] and does not require a control lane per flow-cell. In both MAQC experiments, one lane of each flow-cell was reserved for sequencing phi X genomic DNA. For one experiment (MAQC-2), we obtained both auto-calibrated and phi X-calibrated reads.

Except for the section discussing the impact of basecalling calibration method, we focus on phi X-calibrated, purity-filtered reads that map uniquely to the genome, with up to two mismatches. The restriction to reads mapping to the genome implies that exon-exon junction reads are excluded (~10% of the reads). Additionally, the library preparation protocol does not allow consideration of strand-specific counts, so reads mapping to the forward and reverse strands are pooled.

Definition of union-intersection genes

In our evaluation of DE, we focus on overall expression of a gene, rather than isoform-specific expression. There is no standard technique for summarizing expression levels of genes with several isoforms (see, for example, [6] and [7] for different approaches). For a given gene, we first define a constitutive exon as a set of consecutive exonic bases (i.e., portion of or entire exon) that belong to each isoform of the gene. We then define a unionintersection (UI) gene as a composite gene-level region of interest consisting of the union of constitutive exons that do not overlap with coding exons of other genes (based on Ensembl, Version 55; see [Additional file 2: Supplemental Section S2]). We retain all genes identified with chromosomes 1-22, X, and Y. In addition to including protein-coding genes, the UI genes represent a number of other classes of Ensembl annotation, such as pseudogenes and small RNAs.

Normalization

In order to derive gene expression measures and compare these measures between (groups of) lanes, one first needs to normalize read counts to adjust for varying lane sequencing depths and potentially other technical effects. All but one of the normalization methods considered here are global procedures, in the sense that only a single factor d i is used to scale the counts (perlane).

We evaluate three types of global normalizations: (1) total lane counts, as in RPKM of [7], (2) per-lane counts for a "housekeeping" gene expected to be constantly expressed across biological conditions, e.g., POLR2A, (3) per-lane upper-quartile of gene counts for genes with reads in at least one lane. In order to make the normalized expression measures comparable, the scaling factors are themselves scaled so that their sum across all lanes is equal to the sum of the total counts across all 14 lanes (see [Additional file 2: Supplemental Section S4]).

The expression quantitation problem can be framed in terms of generalized linear models (GLM),
log( [ | ]) log , , ( ) , , E X d d i j i i a i j i j = + +   (1)
where the natural logarithm of the expected value of the read count X i, j for the jth gene in the ith lane is modeled as a linear function of the gene's expression level l a(i), j for the biological condition a(i) assayed in lane i plus an offset (log d i ) and possibly other technical effects (θ i, j ).

Finally, we propose a quantile normalization procedure, inspired from the microarray normalization approach of [18] and its implementation in the R package aroma.light. Specifically, for each lane, the distribution of read counts is matched to a reference distribution defined in terms of median counts across sorted lane. The normalized data are rounded to produce integer values that can be used with the DE statistics described below.

Differential expression

We compare three types of methods for inferring DE, each of which yields one test statistic per gene: Fisher's exact test statistic, likelihood ratio statistics based on a generalized linear model as in Equation (1), and t-statistics based on estimated parameters of the same GLM. Two different t-statistics are evaluated, which use different techniques for estimating the variance of the estimated parameters. We also assess the impact of flow-cell effects, either through the addition of parameters θ i, j in the GLM or through a Mantel-Haenszel test, an extension of Fisher's exact test (see [Additional file 2: Supplemental Section S5]). All of the considered DE statistics can accommodate global normalization via an offset d i . For the GLM-based statistics, the offset is handled as in Equation (1). Fisher's exact test and the Mantel-Haenszel test compare the distribution of the counts of the jth gene to that of d.

The likelihood ratio statistics are the most general, as they can be used for comparisons of any number of biological sample types and adjust for general experimental effects as well as sample covariates, e.g., RNA quality. The t-statistics are only applicable for testing differences between two groups. The t-statistics and likelihood ratio statistics are based on maximum likelihood estimators from the same GLM, but have different performance in certain cases. Distributional properties of all of the GLM-based statistics are derived under asymptotic theory; therefore, they may have poor behavior for small numbers of input samples or low counts (though this is not what we experience). In contrast, Fisher's exact test makes no assumption about sample size; however, it only adjusts for global experimental effects and even the Mantel-Haenszel extension allows only a single genelevel experimental effect.

Likelihood ratio statistics have been used in [6] for the special case of only a global lane effect (i.e., θ i, j = 0 in Equation (1)); these authors also mentioned applying an arcsine-root transformation for variance stabilization of the per-gene read proportions within each lane. Bayesian statistics with Gamma prior for the Poisson parameter have been found to yield similar results as the above GLM-based test statistics [19]. Other test statistics considered in the recent mRNA-Seq literature include t-statistics with square root-transformed standard errors and Bayesian statistics based on the Beta-Binomial distribution [3].

Receiver operator characteristic curves using qRT-PCR gold-standard

The qRT-PCR data of [13] are used as gold-standard to determine "true" differential expression and derive receiver operator characteristic (ROC) curves for various mRNA-Seq and microarray DE methods. The qRT-PCR estimate of UHR to Brain expression log-fold-change is the difference of average expression measures for UHR and Brain across replicates (see [Additional file 2: Supplemental Section S6]).

We divide the genes assayed by qRT-PCR into three sets, "non-DE", "DE", and "no-call", based on whether their absolute expression log-fold-change is less than a, greater than b, or falls within the interval [a, b], respectively. We ignore the "no-call" genes when determining true/false positives/negatives. True positives (TP) are reported when the sequencing (or microarray) platform not only correctly declares a gene DE, but also agrees with qRT-PCR regarding the direction of DE. The true positive rate (TPR) is then defined as the total number of TPs divided by the total number of DE genes according to qRT-PCR; the false positive rate (FPR) is computed as usual. See Table 1 for a summary.

Software

In order to facilitate analysis and visualization of mRNA-Seq data, we developed two R/Bioconductor software packages, Genominator and GenomeGraphs [20]. Both packages are available from the Bioconductor Project, http://bioconductor.org/packages/release/bioc/ html/Genominator.html and http://bioconductor.org/ packages/release/bioc/html/GenomeGraphs.html, respectively.

Results and Discussion

Comparison of mRNA-Seq differential expression statistics

Lists of differentially expressed genes are typically produced by computing, for each gene, a test statistic comparing expression levels between the two types of biological samples and ranking the genes based on pvalues assessing the statistical significance of the observed differences.

We evaluate various statistics for differential expression (see description in Methods, above) and find that the main difference between test statistics is their ability to handle low counts, an issue of great importance when investigating differential expression in context of mRNA-Seq. When both samples have zero reads, clearly nothing can be said about differential expression. The more pertinent zero-count or low-count scenario occurs when a gene has zero reads for one sample and a reasonable number for the other. Around 700 genes (~1.8%) have zero reads in either Brain or UHR and 10 or more reads in the other tissue. Presumably, this represents an interesting biological phenomenon, where a gene in one tissue is completely non-expressed according to sequencing.

For genes with zero counts in either sample, the t-statistics fail: the estimated standard errors become extremely large (or infinite in the case of the delta method tstatistic) and the nominal p-values cluster around one, regardless of the number of reads in the other sample. For Fisher's exact test and the GLM-based likelihood ratio test, however, we see a continuum of p-values as desired. For genes with reasonable counts in both samples, the choice of test statistic makes little difference in the nominal p-values ([Additional file 1: Supplemental Figures S2 and S3]). Because they cannot stably handle low-count genes, the t-statistics are failing to detect many "easy" cases of DE (i.e., genes with large differences in expression between the two conditions) and, as a result, have very low sensitivity. The poor performance of the t-statistics is reflected in ROC curves ( Figure 1). Removal of genes with fewer than 20 reads in either sample completely accounts for the poor sensitivity of the t-statistics and results in equivalent ROCs for the various DE statistics, all of which are dramatically improved (Figure 1). Table 1 Definition of true and false positive rates. Synopsis of the rules for calling true/false positives and negatives, which take into account the sign of the direction of differential expression: "+" for overexpression in UHR, "-" for over-expression in Brain. As the different mRNA-Seq DE tests show similar behavior, we will from here on focus only on the results from the GLM-based likelihood ratio tests. The results do not change when different test statistics are used, except for the already noted poor performance of the t-statistics for low-count genes.

Impact of technical effects on differential expression Gene-length biases in differential expression It is expected from the mRNA-Seq assay that longer transcripts contribute more "sequencible" fragments than shorter ones expressed at the same level. There is clearly a positive association between gene counts and length, an association that is not entirely removed via scaling by gene length, as in the RPKM of [7] ([Additional file 1: Supplemental Figure S4]). This suggests either higher expression among longer genes or non-linear dependence of gene counts on length.

As noted by [11], the dependence of gene counts on length creates "gene length-related biases" in mRNA-Seq DE results: longer genes tend to have more significant DE statistics ( Figure 2). All of the mRNA-Seq DE statistics evaluated here have an inherent dependence of their estimated standard errors on read counts. This is a serious shortcoming in terms of creating "gene-lists" for differential expression, as the resulting lists could favor long genes with small underlying effects as compared to short genes with large effects. Considering only estimated fold-changes is inadequate, as this ignores the fairly large range of standard errors for a given foldchange and gene length.

One can possibly remedy the length dependence of DE statistics using a fixed number of bases from each gene; repeating the DE analysis by randomly selecting 250 bp from each gene removes the association between DE significance and length ([Additional file 1: Supplemental Figure S5]). This also indicates that the cause of the association is the length of the gene and not differences in the underlying expression levels of longer genes. However, a fixed-length analysis is unsatisfactory, as it discards large amounts of data and there is no natural choice of common length.

A weighted analysis based on gene length might constitute a reasonable compromise towards a length-independent DE filter. Indeed, scaling each t-statistic by the inverse of the square root of length provides a lengthindependent ranking ( Figure 2). However, the problem of choosing a cutoff still remains. Under the assumptions presented in [11], with the unweighted t-statistics and using the same cutoff across genes, power increases with gene length for a given level of DE. Under the same scenario, for the weighted t-statistics, both Type I error rate and power decrease with length.

Impact of base-calling calibration method

The practice of reserving one lane out of eight, in each flow-cell, for sequencing bacteriophage phi X genomic DNA has important implications for experimental design, in terms of sample size and balance. We find that more reads are mapped to the genome with autocalibration than with the standard phi X calibration, at each of three mapping stringency levels ( Figure 3). Purity-filtered perfectly matching (FPM) reads are unlikely to contain sequencing errors and can serve as proxies for perfectly accurate reads. Similarly, purity-filtered reads with either 0, 1, or 2 mismatches (FMM) are comprised of both FPM reads as well as reads that represent GLM-based likelihood ratio statistics and t-statistics, before and after removing genes with fewer than 20 reads in either Brain or UHR. In both plots, a gene was declared non-DE if its qRT-PCR absolute log-ratio was less than 0.2 and DE if its absolute log-ratio was greater than 2.0. Note that we require a true positive to be differentially expressed in the same direction according to both mRNA-Seq and qRT-PCR (see Table 1 and Methods).   sequencing errors. Then, the ratio (FMM-FPM)/FMM can be viewed as a rough estimate of the sequencing error rate, assuming no SNPs. For all lanes, the autocalibration method produces slightly lower error rates (by~5%).

The increased number of reads is spread unevenly throughout the transcriptome. A majority of the UI genes have no change in read counts between calibration methods, whereas around 25% of the genes have 4 or more additional reads when using auto-calibration. When computing an (FMM-FPM)/FMM ratio for each gene for both phi X and auto-calibration, auto-calibration yields lower error rates by about 3.8% on average.

The significance of differences in expression measures between the two calibration methods was evaluated by comparing observed differences to a permutation distribution of differences obtained by randomly swapping the auto-calibrated and phi X-calibrated sets of read counts for each of the 14 lanes. We find that in terms of absolute expression measures there are small, but statistically significant differences between the two calibration methods. However, relative expression measures, as used in DE analyses, do not appear to be significantly different (see [Additional file 2: Supplemental Section S8]).

Although our assessment is based on only two flowcells, it seems quite clear that auto-calibration is advantageous, as it yields more balanced designs, frees up one lane per flow-cell, and produces a larger number of higher quality reads per lane.

Lane, flow-cell, and library preparation effects

The Poisson distribution has been shown to provide a good fit to the distribution of gene-level counts across replicate lanes, after normalization by total lane counts [4,6]; our experience with both the MAQC data and unpublished datasets for Drosophila melanogaster supports this conclusion. The goodness-of-fit of the Poisson model across different organisms and different sequencing facilities strongly supports its validity as a model for lane variation and justifies the pooling of read counts across lanes by summation. Note, however, that the applicability of the Poisson distribution is questionable when analyzing biological replicates (i.e., samples from different individuals within a given biological group, such as, patients with the same type of cancer). The use of negative binomial or empirical Bayes methods, as described in the SAGE literature [21,22], may be sensible in such settings of increased variability.

Our analyses also confirm the previously noted small technical differences between flow-cells [6], though there is evidence of slightly more variation between flow-cells than between replicate lanes ([Additional file 1: Supplemental Figure S6c]). Regardless of their statistical significance, estimated flow-cell effects are small in magnitude and thus have a minor impact only in detecting extremely small biological effects; almost none for genes with more than 3 reads/lane.

To the best of our knowledge, there has been no published examination of the technical variation introduced during library preparation; replication of the library preparation is both expensive and time-consuming. There are clear library preparation effects on the total number of reads ([Additional file 1: Supplemental Figure S1]). After adjusting for differences in total lane counts, there is evidence for increased variation across replicate library preparations as compared to flow-cells and lanes ([Additional file 1: Supplemental Figure S6d]); however, this increased variability is mainly due to high-count genes for which there is high power to detect small differences. A direct comparison of library preparation effects to flow-cell and biological effects is not possible due to the experimental design, but comparison of the magnitude of the estimated differences suggests that library preparation effects are much smaller than the biological effects between Brain and UHR ( Figure 4) and slightly larger for some genes than flow-cell effects The biological differences between Brain and UHR samples may be much larger than those typically observed; therefore, technical sources of variation need not always be irrelevant. Finally, we note that the MAQC data are somewhat "ideal", in the sense that: (1) commercial-grade RNA was sequenced and (2) the sequencing was performed in-house by Illumina. A typical mRNA-Seq experiment begins with the extraction of RNA from biological specimens and variability induced during extraction may be much larger than the technical variability seen here.

Normalization of mRNA-Seq data

Because the total number of reads varies between lanes, read counts must be normalized to allow comparison of expression measures across lanes or samples. While this subject has received relatively little attention in the mRNA-Seq literature, the common practice is to scale the gene counts by lane totals [6,7]. We find, however, that more general quantile-based procedures yield much better concordance with qRT-PCR and are hopefully more robust than normalization by a single housekeeping gene.

Here, we evaluate a variety of normalization procedures and focus on two main questions: (1) Does the normalization improve DE detection (sensitivity)? (2) Does the normalization result in low technical variability across replicates (specificity)? To assess DE detection, we rely on the qRT-PCR data of [13] as a gold-standard for determining true and false positives. Because there are a limited number of non-DE genes in the qRT-PCR data, we also assess goodness-of-fit to the Poisson model for replicate lanes (GLM 1 in [Additional file 2: Supplemental Table S4]).

The simplest form of normalization is achieved by scaling gene counts, in lane i, by a single lane-specific factor d i . In essence, these global scaling factors define the null hypothesis of no differential expression: if a gene has the same proportions of counts across lanes as the proportions determined by the vector of d i 's, then it is deemed non-differentially expressed.

The standard total-count normalization results in low variation across lanes, flow-cells, and library preparations, as discussed above. What has not been understood previously, is that this normalization technique reflects the behavior of a relatively small number of high-count genes: 5% of the genes account for approximately 50% of the total counts in both Brain and UHR. These genes are not guaranteed to have similar levels of expression across different biological conditions and, in the case of the MAQC-2 dataset, they are noticeably over-expressed in Brain, as compared to the majority of the genes ( Figure 5).

Accordingly, the performance of total-count normalization is not particularly impressive for detecting DE ( Figure 6): sensitivity is only slightly higher as compared to the microarray data, even for genes with relatively large differences in expression (> 2 absolute log-ratio). When including genes with lower levels of differential expression (> 0.5 absolute log-ratio), performance is no better (and perhaps slightly worse) than that of microarrays. This contradicts general expectation given that the mRNA-Seq data are less noisy and thus better at detecting small expression differences. For small levels of DE, the bias in estimated log-ratios using total-count normalization makes the sequencing estimates less accurate.

We evaluate two alternatives for normalization of mRNA-Seq data. One approach relies on a single housekeeping gene like POLR2A, a standard technique for normalizing qRT-PCR expression measures. However, this is not a feasible solution in general, since it is not known a priori which genes have stable expression levels (in [13], POLR2A was chosen only after examining many replicates for UHR and Brain across a number of plates).

In analogy with standard techniques for normalizing microarray data, we propose to match the between-lane distributions of gene counts in terms of parameters such as quantiles. For instance, one could simply scale counts within lanes by their median. In our case, due to the preponderance of zero and low-count genes, the median is uninformative for the different levels of sequencing effort. Instead, we use the per-lane upper-quartile (75th percentile), after excluding genes with zero reads across all lanes (see Methods).

Compared to total-count normalization, both POLR2A and upper-quartile normalization significantly reduce the bias of DE relative to qRT-PCR (Figure 7 and [Additional file 1: Supplemental Figure S7]), with upper-quartile having bias near zero. ROC curves illustrate that both upper-quartile and POLR2A normalization are unequivocally better than total-count normalization at detecting DE ( Figure 6 and [Additional file 1: Supplemental Figure S8a]) and result in improved sensitivity of sequencing relative to microarray data ( Figure 6 and [Additional file 1: Supplemental Figure S9]).

A closer look at technical variation for the different normalization procedures shows that upper-quartile normalization does not noticeably increase the level of variability as compared to total-count normalization; POLR2A normalization is slightly more variable but still comparable ( Figure 8).

Finally, it is also feasible to perform quantile normalization across lanes, as is often done in microarray experiments [23]. However, there does not seem to be added benefit to this more complicated normalization strategy. Quantile normalization performs similarly in the ROC analyses (Figure [Additional file 1: Supplemental Figure S8a]) and induces comparable, or even slightly more, variability than upper-quartile normalization ( Figure 8). We again recall the somewhat artificial nature of the MAQC data, which were obtained at essentially the same time, by one lab, using ideal RNA samples. As more data become available, there may be larger variations in gene count distributions necessitating more aggressive normalization.

Conclusions

Our main novel finding is the extent to which normalization affects differential expression results: sensitivity varies more between normalization procedures, than between test statistics. Although the standard total-count normalization results in Poisson variation across replicate lanes, it has poor detection sensitivity when benchmarked against qRT-PCR. Instead, we propose scaling gene counts by a quantile of the gene-count distribution (the upper-quartile) and show that such normalization improves sensitivity without loss of specificity.

An important aspect of the MAQC datasets, which could have an impact on the interpretation of the Figure 6 Comparison of mRNA-Seq and microarray differential expression calls to qRT-PCR: ROC curves. Genes common to all three platforms and present for both qRT-PCR and sequencing (see [Additional file 2: Supplemental Section S6]). evaluated and declared DE if their qRT-PCR absolute log-ratio was (a) greater than 2 or (b) greater than 0.5; genes were declared non-DE if their absolute log-ratio was less than 0.2. The GLM-based likelihood ratio test was used for the sequencing data. Two normalization procedures are presented for mRNA-Seq: totalcount (black) and upper-quartile (blue) normalization. Microarray data were normalized using RMA (gray). Note that we require a true positive to be differentially expressed in the same direction according to both mRNA-Seq and qRT-PCR (see Table 1 and Methods).

analyses presented, is the large difference in gene expression between Brain and UHR. Often, gene expression analyses consider much more closely related sets of samples, with only relatively few genes expected to be differentially expressed. In the comparison of Brain and UHR, by contrast, only 5-30% of genes examined by qRT-PCR were deemed as non-differentially expressed (depending on the choice of the multiple testing procedure used to correct the pvalues). Indeed, there may be no truly non-DE genes queried by the qRT-PCR experiment, but rather, very small differences in expression for every gene. This creates a possibility for errors when specifying a set of true negatives; we have tried to control for this by a careful and stringent definition of true negatives and by evaluating the effect of changes in this definition (see [Additional file 2: Supplemental Section S6]).

Furthermore, the extreme difference in transcriptional profiles between the Brain and UHR samples means that the p-values from the sequencing experiment are smaller than would be expected if all the genes were truly non-DE. In particular, the p-values for non-DE genes (according to qRT-PCR) do not follow the expected uniform distribution, but are noticeably shifted toward zero ([Additional file 1: Supplemental Figure S10]). The microarray data demonstrate the same behavior ([Additional file 1: Supplemental Figure S10]), suggesting it is caused by the samples under consideration and not by inherent problems of the statistical methods. In contrast to the qRT-PCR tests for differential expression, the tests applied to sequencing data take into account the total number of reads mapping to each gene and, as a result, tend to have greater power for longer genes.

Another possible critique is that the improvement of UQ over total-count normalization is due to this normalizations more closely matching the normalization procedure used with the qRT-PCR data rather than proper reflection of actual biological differences; in other words, UQ normalization might be closely matching the effect of dividing by POLR2A, as is done with the qRT-PCR data but not the underlying biology. Indeed, additional scaling of the microarray data by POLR2A slightly improves the ROC compared to the standard microarray quantile normalization ([Additional file 1: Supplemental Figure S8b]). It is more likely, however, that total-count normalization, with its reliance on high-count genes, poorly reflects biological differences. This can be seen by taking a closer look at the POLR2A gene, which was chosen as a reference for qRT-PCR data because of its very similar expression in UHR and Brain across many qRT-PCR replicates [13]: the UHR to Brain fold-change of POLR2A is estimated as 1.3 for total-count normalization in contrast to 0.97 for upperquartile normalization and 0.90 for microarray data.

In regards to DE test statistics, the GLM-based likelihood ratio statistics and Fisher's exact statistics perform equally well in terms of sensitivity and handling of lowcount genes. We find likelihood ratio tests appealing because of their generality. Indeed, using the GLM Figure 7 Comparison of mRNA-Seq and microarray differential expression measures to qRT-PCR. Difference scatterplots comparing the estimates of UHR/Brain expression log-ratio from qRT-PCR to those from (a) mRNA-Seq, using the standard total-count normalization, and (b) microarrays, using the standard RMA normalization. Shown are the genes shared between all three platforms, present in both Brain and UHR according to both mRNA-Seq and qRT-PCR (see [Additional file 2: Supplemental Section S1]), and having absolute qRT-PCR expression log-ratio less than 4. Horizontal lines in (a) represent the median UHR/Brain log-ratio for the sequencing data after the standard total-count normalization (black), POLR2A normalization (red), quantile normalization (yellow), upper-quartile normalization (blue); horizontal lines in (b) show the median UHR/Brain log-ratio for the microarray data after the standard RMA normalization (black) and POLR2A normalization (red). framework, one can adjust for potential confounding variables, including quantitative covariates, e.g., age of sample, as well as accommodate different count distributions (negative binomial in cases of over-dispersion).

A serious concern with all the DE methods considered here is the inherent dependence of power on read count, which in turn is related to both gene expression level and length. As most DE studies produce gene-lists, which are often then related to functional annotation (e. g., GO), it is undesirable for significance values to be driven by features such as length. A weighted analysis based on gene length might lead to a reasonable lengthindependent ranking of genes, that would allow short genes with large effects to gain in significance compared to long genes with small effects.

We find that technical variation is quite low across lanes and flow-cells and slightly larger across library preparations. In all cases, however, the effect on differential expression results is minimal. As noted above, the MAQC datasets are unusual, in that we expect extremely large differences in expression between Brain and UHR and only small library preparation effects because of the high quality of the RNA. In practice, library preparation effects may be closer in magnitude to biological effects.

We have demonstrated that while there are some differences between phi X and auto-calibration in the early stages of the analysis pipeline, the differences in terms of differential expression are small. Overall, auto-calibration seems advantageous, as it yields more  The analysis conducted in this work, as well as others, is predicated on a "whole-gene" view of expression profiling. We evaluated technical effects, phi X calibration, and normalization methods using a very constrained UI gene definition. We limited ourselves to such a strict definition in order to ensure that the evaluation was not biased by alternative splicing or overlapping genes. Our UI gene definition is a gross over-simplification, as a large amount of biologically relevant information is lost; we exclude more than 50% of the reads which fall within Ensembl genes.

As high-throughput sequencing becomes more prevalent, our ability to precisely characterize the transcriptome of a sample will dramatically increase. More refined analyses, such as isoform-level expression, allele-specific expression, and genome annotation (segmentation), involve comparing distinct regions within a sample as opposed to the same region across samples. Such analyses will require an understanding of the effect of sequence composition on base coverage to account for the heterogeneity of base-level count distributions

The true positive rate (TPR) is estimated as TP/P and the false positive rate (FPR) as FP/N.

Figure 1
1Comparison of differential expression statistics: ROC curves. (a) All DE statistics, no gene filtering. (b)

Figure 2
2Differential expression statistics, by length. Boxplots of the ranks of DE statistics vs. gene lengths for UI genes at least 250 bp-long and with non-zero counts in both Brain and UHR. (a) Delta method t-statistics. (b) Delta method t-statistics weighted by the inverse of the square root of gene length.

Figure 3
3Impact of base-calling calibration method on read-mapping. Barplots of average read counts per lane with and without phi X calibration, for each of the four biological sample (Brain, UHR) and flow-cell (F2, F3) combinations. Reads are classified into three nested categories: purity-filtered perfectly matching reads (FPM); purity-filtered reads with either 0, 1, or 2 mismatches (FMM); unfiltered reads with either 0, 1, or 2 mismatches (MM).

Figure 4
4Comparison of biological, library preparation, and flow-cell effects. Boxplots of estimated log-fold-changes for UHR vs. Brain biological effects (GLM 2 in [Additional file 2: SupplementalTable S4]), flow-cell effects adjusting for biology (GLM 4), library preparation effects within flow-cell (GLM 7). Estimates are presented for total-count (black) and upper-quartile (blue) normalization.

Figure 5
5Impact of highly-expressed genes. (a) Cumulative percentage of total read count for Brain (green) and UHR (purple) samples, starting with the gene with the highest read count (across the seven Brain or UHR lanes). Cumulative read counts are marked for the 5, 10, 20, and 30 percent most highly expressed genes. (b) Running value of the UHR/Brain expression fold-change for unnormalized counts, starting with the gene with the lowest total count across all 14 lanes. Horizontal lines correspond to: the ratio of the counts for all genes (black), the ratio of the counts for the POLR2A gene (red), and the ratio of the per-lane upper-quartile of counts for genes with reads in at least one lane (blue).

Figure 8
8Comparison of normalization procedures: Goodness-of-fit of Poisson model. The multiplicative Poisson model (GLM 1 in [Additional file 2: Supplemental Table S4]) is fit to the seven Brain lanes in the MAQC-2 experiment after (a) total-count, (b) POLR2A, (c) upperquartile, and (d) quantile normalization. Goodness-of-fit statistics are computed and displayed in c 2 quantile-quantile plots. Genes with goodness-of-fit statistics in the top quantiles of the c 2 -distribution are displayed using colored plotting symbols: red (1, 5]%, purple (.1, 1]%, gold [0, .1]%. Similar plots for UHR show the same patterns.

balanced designs, frees up one lane per flow-cell, and produces a larger number of higher quality reads per lane.

Additional file 1: Supplementary Figures File. Additional figures referred to in the main article. Click here for file [ http://www.biomedcentral.com/content/supplementary/1471-2105-11-94-S1.PDF ] Additional file 2: Supplementary Text File. Additional text to describe further details and results of the analysis. Click here for file [ http://www.biomedcentral.com/content/supplementary/1471-2105-11-94-S2.PDF ]
AcknowledgementsWe wish to thank Steffen Durinck and Gary Schroth (Illumina, Inc.) for stimulating discussions on high-throughput sequencing assays and for providing us with the MAQC datasets. We are also grateful to Terry Speed and MargaretTaubSubmit your next manuscript to BioMed Central and take full advantage of:
High-resolution mapping of copynumber alterations with massively parallel sequencing. D Y Chiang, G Getz, D B Jaffe, O' Kelly, Mjt Zhao, X Carter, S L Russ, C Nusbaum, C Meyerson, M Lander, E S , Nature Methods. 6Chiang DY, Getz G, Jaffe DB, O'Kelly MJT, Zhao X, Carter SL, Russ C, Nusbaum C, Meyerson M, Lander ES: High-resolution mapping of copy- number alterations with massively parallel sequencing. Nature Methods 2009, 6:99-103.

Substantial biases in ultra-short read data sets from high-throughput DNA sequencing. J C Dohm, C Lottaz, T Borodina, H Himmelbauer, Nucleic Acids Research. 3616105Dohm JC, Lottaz C, Borodina T, Himmelbauer H: Substantial biases in ultra-short read data sets from high-throughput DNA sequencing. Nucleic Acids Research 2008, 36(16):e105.

Deep sequencingbased expression analysis shows major advances in robustness, resolution and inter-lab portability over five microarray platforms. Pac Hoen, Y Ariyurek, H H Thygesen, E Vreugdenhil, Rham Vossen, R X De Menezes, J M Boer, Gjb Van Ommen, J T Den Dunnen, Nucleic Acids Research. 3621141Hoen PAC, Ariyurek Y, Thygesen HH, Vreugdenhil E, Vossen RHAM, de Menezes RX, Boer JM, van Ommen GJB, den Dunnen JT: Deep sequencing- based expression analysis shows major advances in robustness, resolution and inter-lab portability over five microarray platforms. Nucleic Acids Research 2008, 36(21):e141.

Novel low abundance and transient RNAs in yeast revealed by tiling microarrays and ultra high-throughput sequencing are not conserved across closely related yeast species. A Lee, K D Hansen, J Bullard, S Dudoit, G Sherlock, PLoS Genetics. 4121000299Lee A, Hansen KD, Bullard J, Dudoit S, Sherlock G: Novel low abundance and transient RNAs in yeast revealed by tiling microarrays and ultra high-throughput sequencing are not conserved across closely related yeast species. PLoS Genetics 2008, 4(12):e1000299.

Determination of tag density required for digital transcriptome analysis: Application to an androgen-sensitive prostate cancer model. H Li, M T Lovci, Y S Kwon, M G Rosenfeld, X D Fu, G W Yeo, PNAS105Li H, Lovci MT, Kwon YS, Rosenfeld MG, Fu XD, Yeo GW: Determination of tag density required for digital transcriptome analysis: Application to an androgen-sensitive prostate cancer model. PNAS 2008, 105(51):20179-20184.

RNA-seq: An assessment of technical reproducibility and comparison with gene expression arrays. J C Marioni, C E Mason, S M Mane, M Stephens, Y Gilad, Genome Research. 189Marioni JC, Mason CE, Mane SM, Stephens M, Gilad Y: RNA-seq: An assessment of technical reproducibility and comparison with gene expression arrays. Genome Research 2008, 18(9):1509-1517.

Mapping and quantifying mammalian transcriptomes by RNA-Seq. A Mortazavi, B A Williams, K Mccue, L Schaeffer, B Wold, Nature Methods. 57Mortazavi A, Williams BA, McCue K, Schaeffer L, Wold B: Mapping and quantifying mammalian transcriptomes by RNA-Seq. Nature Methods 2008, 5(7):621-628.

The transcriptional landscape of the yeast genome defined by RNA sequencing. U Nagalakshmi, Z Wang, K Waern, C Shou, D Raha, M Gerstein, M Snyder, Science. 3205881Nagalakshmi U, Wang Z, Waern K, Shou C, Raha D, Gerstein M, Snyder M: The transcriptional landscape of the yeast genome defined by RNA sequencing. Science 2008, 320(5881):1344-1349.

Alternative isoform regulation in human tissue transcriptomes. E T Wang, R Sandberg, S Luo, I Khrebtukova, L Zhang, C Mayr, S F Kingsmore, G P Schroth, C B Burge, Nature. 4567221Wang ET, Sandberg R, Luo S, Khrebtukova I, Zhang L, Mayr C, Kingsmore SF, Schroth GP, Burge CB: Alternative isoform regulation in human tissue transcriptomes. Nature 2008, 456(7221):470-476.

The MicroArray Quality Control (MAQC) project shows inter-andintraplatform reproducibility of gene expression measurements. Maqc Consortium, Nature Biotechnology. 249MAQC Consortium: The MicroArray Quality Control (MAQC) project shows inter-andintraplatform reproducibility of gene expression measurements. Nature Biotechnology 2006, 24(9):1151-1161.

Transcript length bias in RNA-seq data confounds systems biology. A Oshlack, M J Wakeffeld, Biology Direct. 144Oshlack A, Wakeffeld MJ: Transcript length bias in RNA-seq data confounds systems biology. Biology Direct 2009, 4(14).

Sequencing Analysis Software User Guide For Pipeline Version 1.3 and CASAVA Version 1.0 T Illumina. Illumina, Part # 1005359 Rev. AIllumina: Sequencing Analysis Software User Guide For Pipeline Version 1.3 and CASAVA Version 1.0 T Illumina, Inc. 2008http://icom.illumina.com/icom/ software.ilmn?id=277, [Part # 1005359 Rev. A].

Evaluation of DNA microarray results with quantitative gene expression platforms. R D Canales, Y Luo, J C Willey, B Austermiller, C C Barbacioru, C Boysen, K Hunkapiller, R V Jensen, C R Knight, K Y Lee, Y Ma, B Maqsodi, A Papallo, E H Peters, K Poulter, P L Ruppel, R R Samaha, L Shi, W Yang, L Zhang, F M Goodsaid, Nature Biotechnology. 249Canales RD, Luo Y, Willey JC, Austermiller B, Barbacioru CC, Boysen C, Hunkapiller K, Jensen RV, Knight CR, Lee KY, Ma Y, Maqsodi B, Papallo A, Peters EH, Poulter K, Ruppel PL, Samaha RR, Shi L, Yang W, Zhang L, Goodsaid FM: Evaluation of DNA microarray results with quantitative gene expression platforms. Nature Biotechnology 2006, 24(9):1115-1122.

Preparing Samples for Sequencing mRNA Ilumina. Illumina, Part # 1004898 Rev. AIllumina: Preparing Samples for Sequencing mRNA Ilumina, Inc. 2009http:// icom.illumina.com/icom/software.ilmn?id=277, [Part # 1004898 Rev. A].

Accurate whole human genome sequencing using reversible terminator chemistry. D R Bentley, S Balasubramanian, H P Swerdlow, Nature. 4567218Bentley DR, Balasubramanian S, Swerdlow HP, et al: Accurate whole human genome sequencing using reversible terminator chemistry. Nature 2008, 456(7218):53-59.

Ultrafast and memoryefficient alignment of short DNA sequences to the human genome. B Langmead, C Trapnell, M Pop, S L Salzberg, Langmead B, Trapnell C, Pop M, Salzberg SL: Ultrafast and memory- efficient alignment of short DNA sequences to the human genome.

. Genome Biology. 10325Genome Biology 2009, 10(3):R25.

Base-calling of automated sequencer traces using phred. II. Error probabilities. B Ewing, P Green, Genome Research. 83Ewing B, Green P: Base-calling of automated sequencer traces using phred. II. Error probabilities. Genome Research 1998, 8(3):186-194.

Exploration, Normalization, and Summaries of High Density Oligonucleotide Array Probe Level Data. R A Irizarry, B Hobbs, Collin F Beazer-Barclay, Y D Antonellis, K J Scherf, U Speed, T P , Biostatistics. 42Irizarry RA, Hobbs B, Collin F, Beazer-Barclay YD, Antonellis KJ, Scherf U, Speed TP: Exploration, Normalization, and Summaries of High Density Oligonucleotide Array Probe Level Data. Biostatistics 2003, 4(2):249-264.

Analysis of high-throughput biological data: some statistical problems in RNA-seq and mouse genotyping. M A Taub, Department of Statistics, UC BerkeleyPhD thesisTaub MA: Analysis of high-throughput biological data: some statistical problems in RNA-seq and mouse genotyping. PhD thesis Department of Statistics, UC Berkeley 2009.

GenomeGraphs: integrated genomic data visualization with R. S Durinck, J Bullard, P T Spellman, S Dudoit, BMC Bioinformatics. 102Durinck S, Bullard J, Spellman PT, Dudoit S: GenomeGraphs: integrated genomic data visualization with R. BMC Bioinformatics 2009, 10:Article 2.

Identifying differential expression in multiple SAGE libraries: an overdispersed log-linear model approach. J Lu, J K Tomfohr, T B Kepler, BMC Bioinformatics. 6165Lu J, Tomfohr JK, Kepler TB: Identifying differential expression in multiple SAGE libraries: an overdispersed log-linear model approach. BMC Bioinformatics 2005, 6:165.

. Bullard, BMC Bioinformatics. 11Bullard et al. BMC Bioinformatics 2010, 11:94 http://www.biomedcentral.com/1471-2105/11/94

Moderated statistical tests for assessing differences in tag abundance. M D Robinson, G K Smyth, Bioinformatics. 2321Robinson MD, Smyth GK: Moderated statistical tests for assessing differences in tag abundance. Bioinformatics 2007, 23(21):2881-2887.

Exploration, normalization, and summaries of high density oligonucleotide array probe level data. R A Irizarry, B Hobbs, Collin F Beazer-Barclay, Y D Antonellis, K J Scherf, U Speed, T P , 10.1186/1471-2105-11-94Biostatistics. 4PrintIrizarry RA, Hobbs B, Collin F, Beazer-Barclay YD, Antonellis KJ, Scherf U, Speed TP: Exploration, normalization, and summaries of high density oligonucleotide array probe level data. Biostatistics 2003, 4(1465-4644 (Print)):249-64. doi:10.1186/1471-2105-11-94

Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments. Bullard, BMC Bioinformatics. 1194Cite this article asCite this article as: Bullard et al.: Evaluation of statistical methods for normalization and differential expression in mRNA-Seq experiments. BMC Bioinformatics 2010 11:94.