Strategies and challenges associated with recruiting retirement village communities and residents into a group exercise intervention

Rachel L Duckham
Jamie L Tait
Caryl A Nowson
Kerrie M Sanders
Dennis R Taaffe
Keith D Hill
Robin M Daly
Strategies and challenges associated with recruiting retirement village communities and residents into a group exercise intervention
10.1186/s12874-018-0633-4R E S E A R C H A R T I C L E Open Access
Background: Randomized controlled trials (RCTs) provide the highest level of scientific evidence, but successful participant recruitment is critical to ensure the external and internal validity of results. This study describes the strategies associated with recruiting older adults at increased falls risk residing in retirement villages into an 18month cluster RCT designed to evaluate the effects of a dual-task exercise program on falls and physical and cognitive function.Methods: Recruitment of adults aged ≥65 at increased falls risk residing within retirement villages (size 60-350 residents) was initially designed to occur over 12 months using two distinct cohorts (C). Recruitment occurred via a three-stage approach that included liaising with: 1) village operators, 2) independent village managers, and 3) residents. To recruit residents, a variety of different approaches were used, including distribution of information pack, on-site presentations, free muscle and functional testing, and posters displayed in common areas. Results: Due to challenges with recruitment, three cohorts were established between February 2014 and April 2015 (14 months). Sixty retirement villages were initially invited, of which 32 declined or did not respond, leaving 28 villages that expressed interest. A total of 3947 individual letters of invitation were subsequently distributed to residents of these villages, from which 517 (13.1%) expressions of interest (EOI) were received. Across three cohorts with different recruitment strategies adopted there were only modest differences in the number of EOI received (10.5 to 15.3%), which suggests that no particular recruitment approach was most effective. Following the initial screening of these residents, 398 (77.0%) participants were deemed eligible to participate, but a final sample of 300 (58.0% of the 517 EOI) consented and was randomized; 7.6% of the 3947 residents invited. Principal reasons for not participating, despite being eligible, were poor health, lack of time and no GP approval. Conclusion: This study highlights that there are significant challenges associated with recruiting sufficient numbers of older adults from independent living retirement villages into an exercise intervention designed to improve health and well-being.

(Continued from previous page)

Background

Falls among older adults are a serious public health problem, with at least 30% of community-dwelling older adults over the age of 65 years, and 50% over the age of 80 years, falling each year [1,2]. The consequence of falling among older adults includes injury and fractures, which can result in a loss of independence, reduced quality of life and an upward trajectory in injury-related health care utilization [3,4]. With the incidence of falls increasing with our aging population, there is currently a concerted effort to implement safe, effective and cost-effective falls prevention programs to a broad range of older adults residing in the community [5]. A healthier, independent older population will reduce the substantial direct and indirect health care costs that are attributable to falls and fractures [6].

In Australia, there has been an increased trajectory in the number of older adults relocating to 'retirement villages' or 'retirement communities': modes of housing for older adults located in a gated community or complex that often consists of group housing or independent living villas or apartments for adults aged 55 years and over. Currently, Australia has > 2300 retirement villages, of which 184,000 (5.7%) men and women over the age of 55 years reside [7]. By 2025, the number of residents is projected to increase to over 380,000 older adults [7]. These retirement communities are designed to offer older adults a range of health, leisure and support services as well as providing opportunities for increased social interaction and a sense of belonging in a safe and secure environment [8]. In this context, retirement villages represent a captive audience in which to recruit older adults into intervention trials.

Previous research has reported challenges with regard to the recruitment of older adults into fall prevention trials, particularly exercise interventions, as many older adults are often in denial that they are at risk of falling [9,10]. Randomized controlled trials (RCT) provide the highest level of evidence for health and clinical outcomes; however, the successful recruitment of participants is critical to ensure the external and internal validity of results [11]. One of the major challenges when conducting any RCT in older adults is finding an adequate balance between the recruitment requirements, study timeline and allocated budget [11,12]. Failing to recruit the required number of eligible participants within these constraints can significantly impact the statistical power of the trial and ultimately the overall findings [11]. Currently, there is limited data on successful recruitment strategies of older adults into fall prevention programs, particularly those residing in independent living retirement villages.

The aim of this report is to describe the strategies, efficacy, pitfalls and successes associated with recruiting 300 older adults residing in retirement communities at increased risk for falling, into an 18-month RCT referred to as the Physical and Cognitive Exercise Intervention Trial (PACE-IT). This is a community-based, cluster RCT, with a 6-month supervised and structured intervention, a 6-month 'step-down' maintenance phase and a 6-month follow-up, in which older adults residing in retirement villages, who were at a high risk of falling, were randomly allocated to either an exercise program involving dual-task functional power training (DT-FPT), or a usual care control group. The primary aim of the RCT was to determine whether DT-FPT could reduce the rate of falls in older adults residing in independent living retirement communities [13]. The initial goal was to recruit 280 older adults from a total of 14-16 retirement villages within Melbourne, Victoria over a 12-month period. Here we provide a detailed report of the recruitment approaches and subsequent successes and failures associated with this RCT, in order to inform strategies for future trials aimed at recruiting this population group.

Methods

Study overview

A detailed study protocol of the PACE-IT intervention has previously been published [13]. Briefly, this study was an 18-month, community-based, cluster RCT in which older adults residing in retirement villages, who were at a high risk of falling, were randomly allocated to 1) an exercise program involving DT-FPT, or 2) a usual care control group. The intervention was divided into three distinct phases: 6 months of supervised and structured DT-FPT, a 6-month step down maintenance phase and a 6-month follow-up. The primary outcome of this study was the rate of falls over the 6-, 12-, and 18-month study period. Secondary outcome measures assessed at these time points included changes in lower limb functional muscle strength and power, isometric knee extensor, dorsiflexion and grip strength, dynamic balance and reaction time, gait, quality of life, cognitive function and falls related self-efficacy.

Following initial screening and baseline testing, participants were randomised by cluster (retirement village), stratified by village size (< 75 or ≥ 75 residents), to either the DT-FPT program or the usual care group. Participants residing in the villages assigned to the DT-FPT program were asked to attend two supervised exercise sessions per week for 26 weeks followed by a 26-week step-down phase of one supervised session per week. Detailed information about the training program has been reported previously [13]. Briefly, all training was conducted onsite at each retirement village in small groups (8 to 10 per group) and supervised by an accredited exercise physiologist who has completed a University degree or certificate IV fitness trainers who have typical completed an 18-month course that allows them to practice as personal trainers. The 26-week training program was divided into a 2-week familiarisation (orientation) period, followed by three distinct but interrelated 8-week mesocycles, which were designed to be progressively more challenging. Each training session (45-60 min in duration) was divided into four components: 1) a warm-up consisting of rhythmic and range of motion exercises, 2) challenging balance and mobility activities, 3) high-velocity progressive resistance training (HV-PRT), and 4) a cool-down. Dual-task cognitive and motor activities were incorporated simultaneously into the challenging balance, mobility and HV-PRT exercises. All exercise programs were individualized to the participant's functional ability.

This study was managed by the Institute for Physical Activity and Nutrition (IPAN) at Deakin University, Burwood, Melbourne, Australia, and was funded by a National Health and Medical Research Council (NHMRC) project grant (ID1046267). The study was approved by the Deakin University Human Research Ethics Committee (HREC 2013-051) and was registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12613001161718). Written informed consent was obtained from all participants prior to commencement in the trial.

Recruitment strategies

This study was conducted in retirement villages within the Melbourne metropolitan region and surrounding areas of regional Victoria, Australia. For this trial, a three-stage recruitment strategy was adopted which included the recruitment of 1) retirement village operators 2) village managers, and 3) residents residing in the retirement villages. These participants were recruited via advertisements placed on community notice boards, and word of mouth from village residents and managers.

Recruitment of retirement village operators

The first level of recruitment required identifying all independently living retirement villages within a 125 km radius of Deakin University, Burwood, Victoria, Australia through internet searches, yellow pages, and senior citizen expos. In total, 39 retirement village operational managers were identified, equating to a total of 447 individual villages. The retirement village operational managers oversee all individual village managers and were the first point of contact prior to obtaining permission to contact individual village managers. Retirement village operators with the largest number of independent villages and/or resident numbers and in closest proximity to Deakin University were listed and subsequently contacted in order from the largest to smallest number of village occupants until recruitment saturation. Contact with, and recruitment of, the retirement village operators occurred from November 2013 to December 2014. Initial contact was made first by telephone to determine if the management would be interested in the trial being run within their communities, which was followed up with a face-to-face meeting once interest was confirmed. For each of the three cohorts, the location or distance of each village in relation to metropolitan Melbourne was scattered across the north, south, east and western regions.

Recruitment of retirement village managers

The second level of recruitment required the recruitment of the individual retirement villages and their managers. Based on a previous four-month trial from our group [14], we estimated that approximately 14-16 villages would need to be recruited to accommodate the 280 participants needed for this trial (~20 participants per village). Managers of the independent villages were initially contacted via email by the operations management to show their support for the trial to be run in the independent villages. Research staff followed up this contact with a village invitation letter sent directly to the manager's personal work email. A follow-up telephone call was made to all managers following the email to determine the interest and support for the study. Once the managers agreed to participate in the trial, research staff arranged for a study information pack (information letter, EOI form, and a reply-paid envelope) to be delivered to each resident aged 65 years and over residing within the village. In addition, an information session was arranged with the management to be held a week following the mail drop. Posters were also provided to the managers to advertise the trial, and an information session conducted by the research staff was held within the retirement village community centre to promote the study.

Recruitment of village residents

The third level of recruitment required the recruitment of the village residents from within the retirement villages. Recruitment of participants was conducted from February 2014 to April 2015. A range of different strategies was used to increase the recruitment of the residents living within each of the villages. First, since it was assumed that many residents within the communities may not have transportation to travel to Deakin University, all aspects of the assessment and intervention were conducted within the retirement community, requiring exercise trainers and researchers to travel to the villages to reduce participant burden. Secondly, to maximize recruitment, the timeline for recruitment for this trial was initially designed to recruit participants over two distinct cohorts in year 1 and 2. For each cohort, all residents residing in the communities received a study information packet delivered to their home inviting them to participate in the trial. Posters were also displayed in all common areas at each community and an on-site information session was provided to educate the residents about 'healthy ageing' and the potential health benefits of participating in the trial. The study was not promoted as a falls prevention trial but rather as an intervention designed to improve overall health, mobility and well-being. For each of the two cohorts, residents were asked to indicate their interest in the trial by completing and returning (via a reply-paid envelope) an EOI form to the research staff. Once received, research staff contacted all interested participants from the same village by telephone to determine their eligibility. For a village to be deemed eligible at least six participants had to meet the eligibility criteria for inclusion into the study.

Eligibility criteria

As reported previously [13], all interested participants were initially assessed for eligibility using a telephonescreening questionnaire. In brief, participants were eligible for the study based on the following criteria: scoring > 3 points on a falls risk algorithm adapted from identified risk factors for falls [15] indicating increased falls risk, able to walk unaided or with minimal assistance for at least 50 m, cognitively intact (score < 2 errors on the Short Portable Mental State questionnaire) [16], and able to speak English. All eligible participants were further screened using the Exercise and Sports Science Australia (ESSA) exercise screening tool to evaluate any contraindicated medical conditions to exercise. Participants answering 'yes' to any of these screening questions were required to obtain medical clearance from their local doctor prior to participating in the intervention.

Participants were ineligible for the study based on the following criteria: 1) current or prior participation in a structured progressive resistance training (PRT) program and/or a balance training program more than once per week in the past 3 months, or accumulation of > 150 min of moderate to vigorous physical activity a week; 2) acute or terminal illness likely to compromise exercise participation; 3) unstable or ongoing cardiovascular/respiratory disorders; 4) musculoskeletal or neurological diseases disrupting voluntary movement or that might limit training; 5) upper or lower extremity fracture in the past 6 months, or 6) visual impairment not corrected with glasses.

Data collection and analysis

Data collection and analyses for this study were conducted at three levels: the village operators, the village managers and the residents. Data were collected regarding the number of operators approached and recruited, village managers recruited, resident EOIs received, and residents screened. Reasons for ineligibility and non-participation in the trial were also recorded at each level of recruitment. Response rates were calculated and reported as the number of operators/villages/residents recruited into the trial divided by the number initially invited. Recruitment yield was calculated as the total number of residents recruited divided by the number of EOIs received.

Results

Recruitment timeline

Recruitment of participants for this trial was initially intended to be conducted across two cohorts (12 months apart) each comprising 140 residents (280 in total) from a total of 7-8 retirement villages each year (14-16 in total). However, recruitment was extended to a period of 14 months (February 2014 to April 2015) over three cohorts separated by 6 months due to the lower than expected recruitment rate for the first cohort. Furthermore, the participant recruitment goal was amended from 280 to 310 upon completion of recruitment for cohort 1 to allow for a 10% dropout immediately following baseline testing and prior to the start of the intervention. Overall, 13 participants withdrew from the study following baseline testing and prior to randomization, due to the testing being too difficult or challenging (n = 6), health-related reasons (n = 6), or a lack of General Practitioner (GP) approval (n = 1).

Response rates, eligibility and reasons for ineligibility or non-participation

A total of 11 retirement village operators were contacted to participate in the trial, of which three declined and one did not have adequate facilities to implement the intervention. The remaining seven (63.6%) village operators encompassed 60 individual retirement communities. After initial contact with all retirement village managers at each of these communities, a total of 28 (46.7%) were recruited; of the 32 (53.3%) who were not recruited, nine declined and 23 did not respond to the invitation. Overall, 3947 individual letters of invitation were sent to residents across 28 retirement villages, however, only 22 villages were deemed eligible following participant screening as six (21.4%) of the interested villages had an insufficient number of interested and/or eligible residents (Fig. 1).

Of the 3947 letters of invitation which were sent to residents, 517 (13.1%) EOIs were received. Following the screening of these residents, 398 (77.0%) participants were deemed eligible to participate, however, only 300 (58.0%) consented, attended baseline testing and were subsequently randomized. Of the 119 (23.0%) participants deemed ineligible (Table 1), the main reasons were no longer interested or could not be contacted (n = 50, 42.0%), being classified as too physically active to meet the eligibility criteria (n = 34, 28.6%), and not being classified as at increased risk for falling (n = 27, 22.7%). The main reasons for non-participation in the 98 participants that were initially deemed eligible included: exclusion of the retirement village due to low numbers (n = 18, 18.4%), poor health (n = 11, 11.2%), a lack of time to commit to the study (n = 6, 6.1%), and failure to gain GP approval (n = 5, 5.1%). Fifty of the 98 participants (51%) provided no reason for non-participation in the trial despite being deemed eligible. Overall 7.6% of the 3947 residents invited to participate were deemed eligible and randomized into the trial.

Recruitment return by recruitment strategies

Over the entire 14-month recruitment period a range of different recruitment strategies were utilized. Cohort 1 was provided with study information packs, which were given to the village managers for distribution and for use at on-site information sessions. However, due to the lower than expected recruitment numbers for this cohort, a number of new recruitment approaches were trialled for cohort 2. This included on-site information sessions, an offer of a free 30-min muscle health and functional assessment, and individually addressed study information packs mailed to the residents via the village managers. Cohort 3 included on-site information sessions and hand delivering of study information packets to residents. Village managers within all three cohorts did not provide permission for the recruitment of participants to be extended to non-village residents that fulfilled the inclusion criteria. As shown in Table 2, the response rate in terms of the EOI received differed between the three cohorts, with both cohort 1 (n = 176, 15.3%) and cohort 2 (n = 180, 14.2%) yielding a better response compared to cohort 3 (n = 161, 10.5%) (chi-squared, p < 0.001 and p < 0.01 respectively). However, after accounting for the variability between villages for the three cohorts these differences were no longer significant. In contrast, based on the number of EOIs received for each cohort, the recruitment approach adopted during cohort 3 resulted in the greatest number of eligible and subsequently randomized participants (chi-squared, p < 0.001, cohort 1: n = 82 (46.7%), cohort 2: n = 104 (57.8%), cohort 3: n = 114 (70.8%). This did not appear to be due to differences in the physical characteristics of participants across cohorts as there were no consistent differences in the physical performance measures (data not shown).

Recruitment return by gender and age

Of the 398 participants initially deemed eligible to participate in this trial, the vast majority were female (n = 288, 72%). Figure 2 presents the number of males and females recruited at each retirement village over the 14-month period. The average age of the 398 individuals was 77.6 ± 7.2 years; 32% were aged 65-74 years, 46% were aged 75-84 years and 22% were aged > 85 years. When analysing the recruitment return by sex and age of the 300 participants who were randomised, a greater percentage were female (73%) than male, and the mean age for females was on average 1 year younger than males (mean ± SD; female: 77.1 ± 6.1 male: 78.6 ± 6.6 years).

Discussion

This study highlights that there were significant challenges associated with recruiting operators, managers, and residents of independent living retirement villages into an exercise intervention trial promoted to improve overall health, mobility and well-being. Overall, 64% of village operators, 47% of managers and 13% of residents that were approached expressed an interest in participating in this trial, with 7.6% of residents deemed eligible and randomised to participate in the trial. Because of this low response rate, the recruitment period for this intervention trial had to be extended and a range of different recruitment strategies utilized to reach the desired number of participants for the study. However, adapting the recruitment strategies throughout the study did not significantly increase the response rates in terms of expressions of interest from residents after accounting for variability between the villages for each cohort. In contrast, the recruitment approach adopted during cohort 3 did yield the greatest number of eligible and subsequently randomized participants, which was not explained by differences in physical characteristics.

Well-designed RCTs provide the highest level of evidence for health and clinical outcomes, but finding an adequate balance between the recruitment requirements, study timeline and allocated budget can result in a failure to recruit the required number of eligible participants, thus impacting the statistical power and overall findings of the trial. In Australia, approximately 5.7% of

Total Number Enrolled in Study 300

Abbreviations: SPMSQ Short Portable Memory State Questionnaire, GP general practitioner a Participants were deemed ineligible if they were meeting or exceeding the Australian National Physical Activity guidelines of > 150 min per week of moderate to vigorous activity, and/or participated in more than one structured resistance (strength) training class or balance class per week men and women (n = 184,000) over the age of 65 years reside in independent living retirement communities [7], and this is projected to increase as our population ages. Thus recruiting people residing in these communities and conducting intervention trials may seem like an ideal opportunity to reach a captive cohort. In our trial, seven of 11 village operational managers agreed to support the study and provide approval for village managers of their 60 retirement villages to be contacted and invited to participate. Of those managers, only approximately half (n = 28, 47%) agreed to participate, despite the program being: 1) supported by higher level management; 2) free; 3) managed directly by professional research staff, and 4) with limited time commitment required by the managers. However, consistent with these findings, comparable response rates from a Retirement villages and participants deemed ineligible following screening due to inadequate participant numbers with a given village (< 6 individuals). Percentage response rates were calculated as a proportion to the number of expressions of interest received b Eligibility and ineligibility rates were calculated as a proportion of the response rate c Randomized rates were calculated as the proportion of the response rate d Randomized rates calculated as the proportion of the number of expression of interest distributed managers have been reported in at least two other exercise intervention studies conducted in retirement villages (48-50%) [17,18]. Collectively, these findings indicate that despite the approval and support from higher-level operational village managers, recruiting residents into such studies requires the support from independent village managers who are ultimately responsible for overseeing the research on-site. In future research, it would be useful to understand barriers and enablers to gain the support of village managers to enhance participation in programs that are beneficial for their residents' overall health and well-being. In order to reach our recruitment goal, we utilized a number of different recruitment strategies across each of the three cohorts, some of which were similar to those reported in previous studies conducted in retirement villages [17,19] and community-dwelling populations [15,18,[20][21][22]. These included on-site presentations, advertising posters of the study at each village and delivery of individual information packs to all residents. Based on the similar number of expressions of interest received for each cohort (10.5 to 15.3%), it appears that no one recruitment strategy utilized in our trial was substantially more effective than another. These observations are in contrast to those of previous studies, whereby on-site presentations (66.4%) [18] and individualized mail-outs (70.3%) [15] provided the greatest yield of interest from potential participants. Overall our response rate of 13.0% in terms of the expression of interest and the final recruitment rate of 7.6% is similar to that reported by at least two large-scale clinical trials (5.9 and 11.0%) that aimed to reduce the risk of falls and fracture in older people residing within the community [15,20].

However, several other similar exercise and falls prevention intervention trials have reported much higher rates of success (12 to 53%) [17,19,23]. These marked differences may be predominantly due to our strict inclusion/ exclusion criteria as we excluded individuals who were physically active and at low risk for falls. In contrast, other exercise-related intervention trials conducted in similar communities excluded individuals who were unable to walk unaided or had medical complications that precluded their involvement in exercise [17,19]. However, comparing the recruitment success rates of different trials can be problematic due to differences in inclusion/exclusion criteria, even if the community population (i.e. retirement communities) and study design are comparable.

Understanding the reasons for non-participation in clinical RCTs provides valuable information to guide future recruitment strategies. In our trial, 42% of individuals who initially expressed interest in our study reported that it required 'too much time commitment' , and thus declined at the screening and informed consent phase. Other studies conducted in retirement villages have reported comparable results in terms of non-participants (39 to 44%) [19,23]. However, in our trial, we also report a further 19% loss of eligible participants prior to the trial commencement, which was due to some participants not receiving GP approval, poor health and lack of time. While these rates of participant loss prior to trial commencement are similar to that reported in other falls and fracture trials (17-18%) [15,20], another study involving retirement villages reported a much lower rate of loss (3%) [19]. While this study provided no explanation for the non-participation, it is recommended that all future studies provide detailed information about the reason(s) for non-participation in clinical trials to help guide future recruitment strategies.

Limitations

There are a number of limitations associated with this study. First, our trial specifically targeted older adults residing in retirement villages who were at an increased risk of falling to participate in a dual-task exercise-training program to reduce falls risk. Therefore, the same experiences may not be evident in other groups residing outside of a purpose built retirement community or clinical population. Second, during recruitment for cohort 1 and 2, we relied on the individual managers rather than research staff to deliver the recruitment information packets to the mailboxes of all residents. Thus, we are unable to comment on the exact method of distribution of the information, nor the reach of that information, and subsequently why some communities had a lower response rate than others. Third, we did not quantify the cost associated with the recruitment of eligible participants into our trial. This would provide valuable information when costing future clinical trials, as failing to recruit the required number of eligible participants within the set budget can significantly impact the overall study findings. Specifically, it would be beneficial to quantify the cost of each individual participant by determining the cost of printing and mailing information, and the staffing costs to deliver recruitment material. Finally, we did not monitor the barriers associated with the recruitment of retirement villages and residents into the trial. However, it is possible that some of the barriers may include a lack of time or even attitudes to ageing and environmental restraints i.e. for managers this may be a lack of understanding of the importance of exercise and for residents there may be a concern about their physical ability to participate and ability to move around the new environment such as the exercise room. Future research should consider addressing barriers associated with recruitment of retirement village managers and residents through qualitative research to investigate possible successful recruitment strategies in this community.

Conclusions

In conclusion, we have identified that there were significant challenges associated with recruiting operators, managers and residents of retirement villages into a cluster clinical intervention trial designed to evaluate the effectiveness of a dual-task exercise program for preventing falls and improving physical and cognitive function in older people. Although it is common for researchers conducting clinical RCTs to face recruitment difficulties [24], there are currently limited data describing the strategies, efficacy, pitfalls and successes associated with recruiting older adults. In this study, adapting recruitment strategies throughout the study did not significantly increase the response rates in terms of expressions of interest from residents suggesting that no particular recruitment approach was most effective.

Retirement village communities present as a captive, and potentially viable recruitment population for large clinical trials, but we have identified that there are a number of challenges associated in working with this population. Based on the findings from this trial, we have developed a set of five recommendations to optimize recruitment among residents of retirement villages:

1. Given the complexity of recruiting residents within retirement villages, researchers should ensure that adequate time is available to establish rapport with operators, managers and residents, and not underestimate the time frame needed to gain access to the communities. 2. It is important to gain approval and support from the higher level operational managers when planning to conduct research studies in retirement communities, but from a practical perspective, it is the independent village managers who are responsible for supporting and promoting the research on-site and so it is essential to gain their support. 3. Researchers must consider the time commitment of managers and insist on the delivery of advertisements and information packets to all residents by the research staff to ensure all residents receive trial material. 4. Recruitment strategies need to be continually reviewed, developed and refined to maximize the involvement of the retirement village residents into research trials while still protecting their right to refuse. 5. When working in retirement villages researchers must be prepared for an approximate 40% nonparticipation rate (from those who initially express an interest in the study), and a further~20% loss of eligible participants prior to the study commencement.

Trial status

Recruitment and data collection for this trial is complete and data analysis has commenced.

Fig. 1
1Flow diagram illustrating the recruitment at the operations, management, and resident level within the retirement villages (EOI, expression of interest)

Fig. 2
2The number of female and male participants (n = 398) that were eligible for this trial presented according to the individual retirement villages

C: Cohort; DT-FPT: Dual-Task Functional Power Training; EOI: Expression of Interest; ESSA: Exercise and Sports Science Australia; IPAN: Institute for Physical Activity and Nutrition; NHMRC: National Health and Medical Research Council; PACE-IT: Physical and Cognitive Exercise Intervention Trial; PRT: Progressive Resistance Training; RCT: Randomized Controlled Trial

Table 1
1Number and proportion of participants deemed eligible and ineligible for the study, including the reasons for ineligibility and non-participationTotal, n (%)

Table 2
2Numbers of: expressions of interest, the response rate and proportion of residents deemed eligible and ineligible, and those randomized: data presented according to the study cohorts and retirement villagesExpression of Interest

Village
Distributed
Response Rate n (%)
Eligibility Rate b n (%)
Ineligibility Rate b n (%)
Randomized c n (%)

Cohort 1 (n = 13)
1
70
21 (30.0)
16 (76.2)
5 (23.8)
12 (57.1)

2
155
35 (22.6)
28 (80.0)
7 (20.0)
11 (39.3)

3
60
15 (25.0)
14 (93.3)
1 (6.7)
12 (80.0)

4
65
14 (21.5)
12 (85.7)
2 (14.3)
11 (78.6)

5
80
12 (15.0)
9 (75.0)
3 (25.0)
7 (58.3)

6
170
16 (9.4)
12 (75.0)
4 (25.0)
8 (50.0)

7
126
24 (19.0)
21 (87.5)
3 (12.5)
16 (66.7)

8
156
14 (9.0)
10 (71.4)
4 (28.6)
5 (35.7)

9 a
65
5 (7.7)
5 (100)
0 (0)

a

10 a
22
1 (4.5)
0 (0)
1 (100)

a

11 a
56
5 (8.9)
2 (40)
3 (60)

a

12 a
45
9 (20.0)
5 (55.6)
4 (44.4)

a

13 a
80
5 (6.3)
3 (60.0)
2 (40)

a

1150
176 (15.3)
137 (77.8)
39 (22.2)
82 [46.7 (7.1 d )]

Cohort 2 (n = 6)
14
262
25 (9.5)
17 (68)
8 (32)
17 (68)

15
79
12 (15.2)
10 (83.3)
2 (16.7)
10 (83.3)

16
139
25 (18.0)
18 (72.0)
7 (28)
14 (56.0)

17
198
35 (17.7)
27 (77.1)
8 (22.9)
23 (65.7)

18
310
33 (10.6)
24 (72.7)
9 (27.3)
19 (57.6)

19
277
50 (18.1)
27 (54.0)
23 (46.0)
21 (42.0)

1265
180 (14.2)
123 (68.3)
57 (31.7)
104 [57.8 (8.2 d )]

Cohort 3 (n = 9)
20
120
22 (18.3)
21 (95.5)
1 (4.5)
15 (68.2)

21
169
13 (7.7)
13 (100)
0 (0)
12 (92.3)

22
144
19 (13.2)
19 (100)
0 (0)
16 (84.2)

23
134
14 (10.4)
12 (85.7)
2 (14.3)
8 (57.1)

24
196
19 (9.7)
17 (89.3)
2 (10.5)
15 (78.9)

25
154
27 (17.5)
24 (88.9)
3 (11.1)
21 (77.8)

26
120
16 (13.3)
13 (81.3)
3 (18.7)
12 (75.0)

27
350
27 (7.7)
16 (59.3)
11 (40.7)
15 (55.6)

28 a
145
4 (2.8)
3 (75.0)
1 (25.0)

a

1532
161 (10.5)
138 (85.7)
23 (14.2)
114 [70.8 (7.4 d )]

Total
3947
517 (13.1)
398 (77.0)
119 (23.0)
300 [58.0 (7.6 d )]

Duckham et al. BMC Medical Research Methodology (2018) 18:173
AcknowledgementsWe acknowledge and thank all the research assistants involved in recruitment and functional assessments and the exercise physiologists who conducted the exercise training for this trial. We would also like to thank the retirement operators, managers and residents who volunteered to participate in this trial.FundingThis project is funded by a grant from the National Health and Research Council (NHMRC) project (APP1046267). The funding body had no influence on the study design.4Australian Institute for Musculoskeletal Sciences (AIMSS), University of Melbourne and Western Health, Melbourne, St. Albans, Australia.5Availability of data and materialsThe datasets generated and/or analysed during the current study are not publicly available as there is currently no ethical approval to share data. Data however may become available from the corresponding author on reasonable request and with additional ethics approval,Ethics approval and consent to participateThe study was approved by the Deakin University Human Research Ethics Committee (HREC 2013-051) and was registered with the Australian and New Zealand Clinical Trials Registry (ACTRN12613001161718). Written informed consent was obtained from all participants prior to commencement in the trial.Consent for publicationNot applicable.Competing interestsThe authors declare that they have no competing interests.Publisher's NoteSpringer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Prevention of falls in older people. Jct Close, Disabil Rehabil. 27Close JCT. Prevention of falls in older people. Disabil Rehabil. 2005;27(18- 19):1061-71.

Preventing falls in elderly persons. M E Tinetti, N Engl J Med. 3481Tinetti ME. Preventing falls in elderly persons. N Engl J Med. 2003;348(1):42-9.

Falls in older people. Risk factors and strategies for prevention. S Lord, C Sherrington, H Menz, Cambridge University pressCambridgeLord S, Sherrington C, Menz H. Falls in older people. Risk factors and strategies for prevention. Cambridge: Cambridge University press; 2001.

Interventions for preventing falls in older people living in the community. L Gillespie, M Robertson, W Gillespie, C Sherrington, S Gates, L Clemson, S Lamb, Cochrane Database Syst Rev. 2101002Gillespie L, Robertson M, Gillespie W, Sherrington C, Gates S, Clemson L, Lamb S. Interventions for preventing falls in older people living in the community. Cochrane Database Syst Rev. 2009;2(10):1002.

Physical activity interventions to prevent falls among older people: update of the evidence. C Sherrington, S Lord, C Finch, J Sci Med Sport. 71Sherrington C, Lord S, Finch C. Physical activity interventions to prevent falls among older people: update of the evidence. J Sci Med Sport. 2004;7(1):43-51.

Osteoporosis costing all Australians a new burden of disease analysis. J J Watts, J Abimanyi-Ochom, K M Sanders, 978-0-9923698-1-1Glebe, NSW, Australiawww. osteoporosis.org.auWatts JJ, Abimanyi-Ochom J, Sanders KM. Osteoporosis costing all Australians a new burden of disease analysis -2012 to 2022, www. osteoporosis.org.au, ISBN 978-0-9923698-1-1, Glebe, NSW, Australia.

National overview of the retirement village sector. Grant Thorton Australia Limited. Property Council, Australia, Property Council of Australia. National overview of the retirement village sector. Grant Thorton Australia Limited; 2014. www.grantthornton.com.au. Accesed 25 Mar 2018.

Review of community facilities in Australian retirement villages: a content analysis. B Xia, M Skitmore, Aust J Ageing. 343Xia B, Skitmore M. Review of community facilities in Australian retirement villages: a content analysis. Aust J Ageing. 2015;34(3):144-8.

Older peoples views of falls prevention interventions in six European countries. L Yardley, F Bishop, N Beyer, K Hauer, G Kempen, C Piot-Ziegler, C Todd, T Cuttelod, M Horne, K Lanta, Gerontologist. 465Yardley L, Bishop F, Beyer N, Hauer K, Kempen G, Piot-Ziegler C, Todd C, Cuttelod T, Horne M, Lanta K, et al. Older peoples views of falls prevention interventions in six European countries. Gerontologist. 2006;46(5):650-60.

Older peoples views of advice about falls prevention: a qualitative study. L Yardley, M Donovan-Hall, K Francis, C Todd, Health Educ Res. 214Yardley L, Donovan-Hall M, Francis K, Todd C. Older peoples views of advice about falls prevention: a qualitative study. Health Educ Res. 2006;21(4):508-17.

Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. S Treweek, P Lockhart, M Pitkethly, J Cook, M Kjeldstom, M Johansen, T Taskola, F Sullivan, S Wilson, C Jackson, BMJ Open. 32360Treweek S, Lockhart P, Pitkethly M, Cook J, Kjeldstom M, Johansen M, Taskola T, Sullivan F, Wilson S, Jackson C, et al. Methods to improve recruitment to randomised controlled trials: Cochrane systematic review and meta-analysis. BMJ Open. 2013;3:e002360.

Randomisation to protect against selection bias in healthcare trials. J Odgaard-Jensen, G Vist, A Timmer, R Kunz, E Aki, H Schunemann, M Briel, A Nordmann, S Pregno, A Oxman, 10.1002/14651858.MR000012.pub3Cochrane Database Syst Rev. 4Odgaard-Jensen J, Vist G, Timmer A, Kunz R, Aki E, Schunemann H, Briel M, Nordmann A, Pregno S, Oxman A. Randomisation to protect against selection bias in healthcare trials. Cochrane Database Syst Rev. 2011;4: MR000012. https://doi.org/10.1002/14651858.MR000012.pub3

Effectiveness of dual-task functional power training for preventing falls in older people: protocol for a cluster randomised controlled trial. R M Daly, R L Duckham, J Tait, T Rantalainen, C Nowson, D Taaffe, K Sanders, K Hill, D Kidgell, L Busija, Trials. 16120Daly RM, Duckham RL, Tait J, Rantalainen T, Nowson C, Taaffe D, Sanders K, Hill K, Kidgell D, Busija L. Effectiveness of dual-task functional power training for preventing falls in older people: protocol for a cluster randomised controlled trial. Trials. 2015;16:120.

Proteinrich diet, with the use of lean red meat, combined with progressive resistance training enhances lean tissue mass and muscle strength and reduces circulating IL-6 concentrations in elderly women: a cluster randomised controlled trial. R M Daly, S O'connell, N Mundell, C Grimes, D Dunstan, C Nowson, Am J Clin Nutr. 994Daly RM, O'Connell S, Mundell N, Grimes C, Dunstan D, Nowson C. Protein- rich diet, with the use of lean red meat, combined with progressive resistance training enhances lean tissue mass and muscle strength and reduces circulating IL-6 concentrations in elderly women: a cluster randomised controlled trial. Am J Clin Nutr. 2014;99(4):899-910.

Trials and tribulations of recruiting 2000 older women onto a clinical trial investigating falls and fractures: vital D study. K Sanders, A Stuart, E Merriman, M Read, M Kotowicz, D Young, T Roderick, I Blair-Holt, A Mander, G Nicholson, BMC Med Res Meth. 978Sanders K, Stuart A, Merriman E, Read M, Kotowicz M, Young D, Roderick T, Blair-Holt I, Mander A, Nicholson G. Trials and tribulations of recruiting 2000 older women onto a clinical trial investigating falls and fractures: vital D study. BMC Med Res Meth. 2009;9(78):1-10.

A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. E Pfeiffer, J Am Geriatr Soc. 2310Pfeiffer E. A short portable mental status questionnaire for the assessment of organic brain deficit in elderly patients. J Am Geriatr Soc. 1975;23(10):433-41.

Social dancing and incidence of falls in older adults: a cluster randomised controlled trial. D Merom, E Mathieu, E Cerin, R Morton, J Simpson, C Rissel, K Anstey, C Sherrington, S Lord, R Cumming, PLoS Med. 1381002112Merom D, Mathieu E, Cerin E, Morton R, Simpson J, Rissel C, Anstey K, Sherrington C, Lord S, Cumming R. Social dancing and incidence of falls in older adults: a cluster randomised controlled trial. PLoS Med. 2016;13(8):e1002112.

Effects of a physical activity and nutrition program in retirement villages: a cluster randomised controlled trial. J Jancey, A Holt, A Lee, D Kerr, S Robinson, L Tang, A Anderson, A Hills, P Howat, Int J Behav Nutr Phys Act. 1492Jancey J, Holt A, Lee A, Kerr D, Robinson S, Tang L, Anderson A, Hills A, Howat P. Effects of a physical activity and nutrition program in retirement villages: a cluster randomised controlled trial. Int J Behav Nutr Phys Act. 2017;14:92.

The effect of group exercise on physical functioning and falls in frail people living in retirement villages: a randomized controlled trial. S Lord, S Castell, J Corcoran, J Dayhew, B Matters, A Shan, P Williams, J Am Geriatr Soc. 51Lord S, Castell S, Corcoran J, Dayhew J, Matters B, Shan A, Williams P. The effect of group exercise on physical functioning and falls in frail people living in retirement villages: a randomized controlled trial. J Am Geriatr Soc. 2003;51:1685-92.

Lessons learnt during a complex, multicentre cluster randomised controlled trial: the ProAct65+ trial. Z Stevens, H Carpenter, S Gawler, C Belcher, D Haworth, D Kendrick, R Morris, T Masud, D Skelton, S Iliffe, Trials. 14192Stevens Z, Carpenter H, Gawler S, Belcher C, Haworth D, Kendrick D, Morris R, Masud T, Skelton D, Iliffe S. Lessons learnt during a complex, multicentre cluster randomised controlled trial: the ProAct65+ trial. Trials. 2013;14(192):1-10.

A physical activity program to mobilize older people: a practical and sustainable approach. J Jancey, A Clarke, P Howart, A Lee, T Shilton, J Fisher, Gerontologist. 482Jancey J, Clarke A, Howart P, Lee A, Shilton T, Fisher J. A physical activity program to mobilize older people: a practical and sustainable approach. Gerontologist. 2008;48(2):251-7.

Physical activity and nutrition program for seniors (PANS): protocol of a randomised controlled trial. L Burke, J Jancey, P Howart, A Lee, D Kerr, T Shilton, BMC Public Health. 107Burke L, Jancey J, Howart P, Lee A, Kerr D, Shilton T. Physical activity and nutrition program for seniors (PANS): protocol of a randomised controlled trial. BMC Public Health. 2010;10(7):1-10.

The effect of group-based exercise on cognitive performance in seniors residing in intermediate care and self care retirement facilities: a randomised controlled trial. A Brown, T Lui-Ambrose, R Tate, S Lord, Br J Sports Med. 43Brown A, Lui-Ambrose T, Tate R, Lord S. The effect of group-based exercise on cognitive performance in seniors residing in intermediate care and self care retirement facilities: a randomised controlled trial. Br J Sports Med. 2009;43:608-14.

Recruitment to randomised trials: strategies for trial enrollment and participation study. The STEPS study. M Campbell, C Snowdon, D Francis, Health Technol Access. 11105Campbell M, Snowdon C, Francis D. Recruitment to randomised trials: strategies for trial enrollment and participation study. The STEPS study. Health Technol Access. 2007;11:105.


BMC Genomics Relative power and sample size analysis on gene expression profiling data

M Van Iterson
Center for Human and Clinical Genetics
Leiden University Medical Center
Leidenthe Netherlands

Netherlands Bioinformatics Centre
the NetherlandsNijmegen

Pac 't Hoen
Center for Human and Clinical Genetics
Leiden University Medical Center
Leidenthe Netherlands

P Pedotti
Center for Human and Clinical Genetics
Leiden University Medical Center
Leidenthe Netherlands

Gjej Hooiveld
Nutrigenomics Consortium
TI Food and Nutrition
The NetherlandsWageningen

Division of Human Nutrition
Nutrition, Metabolism and Genomics Group
Wageningen University
Wageningen The Netherlands

J T Den Dunnen
Center for Human and Clinical Genetics
Leiden University Medical Center
Leidenthe Netherlands

Leiden Genome Technology Center
Leiden University Medical Center
the NetherlandsLeiden

Gjb Van Ommen
Center for Human and Clinical Genetics
Leiden University Medical Center
Leidenthe Netherlands

J M Boer
Center for Human and Clinical Genetics
Leiden University Medical Center
Leidenthe Netherlands

Netherlands Bioinformatics Centre
the NetherlandsNijmegen

R X Menezes
Center for Human and Clinical Genetics
Leiden University Medical Center
Leidenthe Netherlands

Netherlands Bioinformatics Centre
the NetherlandsNijmegen

Laboratory of Pediatrics Erasmus Medical Center
Sophia Children's Hospital
RotterdamThe Netherlands

Department Epidemiology and Biostatistics
VU Medical Center
Amsterdamthe Netherlands

BMC Genomics Relative power and sample size analysis on gene expression profiling data
10.1186/1471-2164-10-439BioMed Central Page 1 of 10 (page number not for citation purposes) Methodology article * Corresponding author
Background:With the increasing number of expression profiling technologies, researchers today are confronted with choosing the technology that has sufficient power with minimal sample size, in order to reduce cost and time. These depend on data variability, partly determined by sample type, preparation and processing. Objective measures that help experimental design, given own pilot data, are thus fundamental.Results: Relative power and sample size analysis were performed on two distinct data sets. The first set consisted of Affymetrix array data derived from a nutrigenomics experiment in which weak, intermediate and strong PPARα agonists were administered to wild-type and PPARα-null mice. Our analysis confirms the hierarchy of PPARαactivating compounds previously reported and the general idea that larger effect sizes positively contribute to the average power of the experiment. A simulation experiment was performed that mimicked the effect sizes seen in the first data set. The relative power was predicted but the estimates were slightly conservative. The second, more challenging, data set describes a microarray platform comparison study using hippocampal δC-doublecortinlike kinase transgenic mice that were compared to wild-type mice, which was combined with results from Solexa/ Illumina deep sequencing runs. As expected, the choice of technology greatly influences the performance of the experiment. Solexa/Illumina deep sequencing has the highest overall power followed by the microarray platforms Agilent and Affymetrix. Interestingly, Solexa/Illumina deep sequencing displays comparable power across all intensity ranges, in contrast with microarray platforms that have decreased power in the low intensity range due to background noise. This means that deep sequencing technology is especially more powerful in detecting differences in the low intensity range, compared to microarray platforms.Conclusion: Power and sample size analysis based on pilot data give valuable information on the performance of the experiment and can thereby guide further decisions on experimental design. Solexa/Illumina deep sequencing is the technology of choice if interest lies in genes expressed in the low-intensity range. Researchers can get guidance on experimental design using our approach on their own pilot data implemented as a BioConductor package, SSPA

Background

Genome-wide technologies such as microarray and sequencing are intensively used to study differential expression in e.g. disease and/or treatment, often compared with controls. Power and sample size analysis give valuable information about the performance of the experiment: what is the optimal number of replicates? Is the power sufficient to detect a biological effect?

Absolute power and sample size estimation must be done using pilot experimental data in each problem separately, as they are influenced by variability that is both technical as well as biological [1]. For this a technology must be chosen. So it is important to understand beforehand how relative power and sample size behave depending on the technology used. In particular, different technologies may display different power depending on the gene expression range.

Here we focus on estimating relative change in power and sample size, given either different effect sizes or different expression profiling technologies. In each case results are derived from pilot experiments, so conclusions relate directly to practice. The different expression profiling technologies include commercial and home-spotted geneexpression microarray platforms, as well as a deep sequencing technology. For power and sample size calculations, we adapted the method proposed by Ferreira et al. [2].

Methods

Power and sample size estimation

Consider the case where samples are studied under two conditions, and interest lies in finding genes differentially expressed between these conditions. We use the power and minimal sample size calculation method of Ferreira et al. [2]. This method assumes that for a set of test statistics measuring the differential expression, their distributions are given as each having a normal distribution (μ, σ 2 ).

For each gene, under the null hypothesis H 0 of non-differential expression we have the mean μ = 0, and under the alternative H 1 of differential expression μ ≠ 0. If we represent by K, L the cumulative distribution functions (CDF) of the test statistics under H 0 and under H 1 , respectively, then the observed test statistics have mixture CDF M given by where λ represents the density of effect sizes θ and π 0 the proportion of non-differentially expressed genes.

Effect sizes can be seen as the difference between a gene's mean expression levels at two conditions, divided by its pooled standard deviation. Note that M is observed and K and L are given, so π 0 and λ need to be estimated. After estimating π 0 using the approach suggested by Langaas et al. [3], λ is estimated by a deconvolution estimator. The average power can be estimated by solving the following equation for u:

where Γ represents the power for a single gene as function of the p-value u and effect size θ and δ is the user-defined false discovery rate. In fact, Ferreira et al. [2] showed that the average power, given by equation 2, is controlled for multiple testing through the adaptive Benjamini-Hochberg method [4]. This is essentially the same as the originally proposed false discovery rate method of Benjamini and Hochberg [5], corrected by the proportion of differentially expressed genes to avoid over-estimation.

The effect size density is estimated unconstrained, so after it is obtained it must be constrained to being non-negative, whilst integrating to 1. To avoid discontinuities where the constraint is applied, we re-adjust the π 0 estimate (see Additional file 1).

Note that to estimate the average power in this way involves also the power to detect effect sizes around zero, which are technically very difficult to measure accurately. A small region around zero can be defined that will be excluded from the density of effect sizes and thereby increases the estimated average power.

Data Description

Simulation experiment A simulation study was performed based on the simulation perviously described by Langaas et al. [3]. While keeping the proportion of non-differentially expressed genes fixed at 0.8, we varied the effect size distribution. Specifically, three different effect size distributions were constructed based on a symmetric bitriangular distribution [3]. This means that each differentially expressed gene is either over-or under-expressed with equal probability, and that the mean effect size per over-expressed gene is chosen at random from values in a window between a = log 2 (1.2) and b >a, with the mode at m = log 2 (2), and for under-expressed genes a window between -a and -b is used, with the mode at -m. We used three values for b, namely log 2 (2), log 2 (4), log 2 (5), generating situations with weak, intermediate and strong effect sizes respectively. This means for example that the weak-effect situation has genes with effect sizes between log 2 (1.2) and log 2 (2).
 M t K t L t d ( ) ( ) ( ) ( , ) ( ) = + − −∞ +∞ ∫ p p qlq q 0 0 1 (1) Γ( , ) ( ) ( ) ( ) , u d u q l q q p d d p = − − −∞ +∞ ∫ 0 1 1 0 (2)
A total of N = 250 simulations were performed with 20000 independently generated normalized expression values and J = 5 samples in each of two groups. Test-statistics were calculated as described by Langaas et al. [3]. The observed power was calculated as the number of differentially expressed genes with Benjamini-Hochberg FDR ≤ 10% (true positives), divided by m 1 = 4000. In addition, the estimated power was evaluated using the method proposed by Ferreira et al. [6].

Example 1 -different biological effect sizes

PPARα is a transcription factor that is activated upon binding by a variety of agonists, both of synthetic and natural origin [7]. In this experiment the outcome of specific PPARα activation on murine small intestinal gene expression was examined using Affymetrix GeneChip Mouse 430 2.0 arrays. PPARα was activated by several PPARαagonists that differed in activating potency. In this paper the data of three agonists were used, namely Wy14,643, fenofibrate and trilinolenin (C18:3). The first two compounds belong to the fibrate class of drugs that are widely prescribed to treat dyslipidemia, whereas trilinolenin is an agonist frequently found in the human diet. For intestinal PPARα, Wy14,643 is the most potent agonist followed by C18:3 and fenofibrate, as is detailed elsewhere (Hooiveld et al. manuscript in preparation). Since time of exposure also affects the effect size, intestines were collected 6 hrs (all three agonists) or 5 days (Wy14,643 and fenofibrate only) after exposure. Details of the rationale behind these designs have been previously published [8,9]. An overview of the dataset is given in the upper part of table 1. Probesets were redefined according to Dai et al. [10]. Expression estimates of probesets were obtained by GCrobust multi-array (GCRMA) analysis, employing the empirical Bayes approach for background adjustment, followed by quantile normalization and summarization [11]. For each compound and exposure time, lists of moderated t-test statistics were computed, using the empirical Bayes linear regression model as implemented in limma [12], for each contrast representing the effect of compound in PPARα-null mice compared to wild-type mice. Based upon these lists, which include the full range of effect sizes, the power and sample size analysis was performed.

Example 2 -different expression profiling platforms

Gene expression profiles in the hippocampi of transgenic δC-doublecortin-like kinase mice were compared to wildtype. The sample size and power analysis was applied to expression data relating to the same samples, measured either by one of several microarray platforms, or by Solexa/Illumina deep sequencing [13].

In this analysis we included expression data for the same samples obtained with four microarray platforms, previously analyzed in the platform comparison study of Pedotti et al. [14], namely Affymetrix, Agilent, Illumina and home-spotted oligonucleotide arrays, the last containing the 22K Sigma-compugen collection. Ten microarrays were used for each platform. For the one-color platforms (Affymetrix and Illumina), each individual RNA was hybridized to one microarray, yielding five hybridizations in each of the groups wild-type and transgenic mice. A direct design was used for hybridization of the two-color arrays (Agilent and home-spotted), i.e. each microarray was hybridized with two RNA samples from different groups. Dye-swapped hybridizations were done with non-identical pairs. This design yielded ten hybridizations in each of the groups, five of which being biologically independent. The data has been pre-processed as described by Pedotti et al. [14]. mRNA expression levels from four samples of each group were also measured using the Solexa/Illumina deep sequencing; for details about sample preparation and pre-processing, see [13]. Moderated t-test statistics were calculated, for each technology, using the empirical Bayes linear regression model as implemented in limma [12].

To enable comparison between technologies, for each technology subsets were generated containing only genes mapped to the same Ensembl tags, leaving 9504 genes per technology [14]. The datasets were stratified into three subsets of genes of approximately equal size (3168) as follows. First the median-expression values across wild-type samples for the Solexa/Illumina data were calculated, per Ensembl tag. Then the 33rd and 67th percentiles of these values were obtained. These two values were used to determine three strata: tags with Solexa/Illumina medianexpression below 33rd percentile were put into the lowintensity range, followed by tags with median-expression between 33th and 67th percentiles being put into the intermediate-intensity range, and finally remaining tags were put into the high-intensity range. In the power and sample size analysis a region near zero ([-0.5, 0.5]) was excluded from the density of effect sizes.  [11], limma [12] and multtest [17], qvalue [18] and for graphs we used the lattice [19] package. We adapted the original S-Plus scripts kindly made available to us by Ferreira [6] and developed an R package called SSPA, available from BioConductor [16]http://biocon ductor.org/packages/release/bioc/html/SSPA.html.

Results

Simulation experiment

The main purpose of this small simulation experiment was to show that the proposed method of Ferreira et al. [2] is able to perform relative average power estimations in a scenario of varying effect size distributions. Although the estimated average power is slightly conservative, the relative power is in agreement with the true positive rate, the fraction of truly differentially expressed genes amongst the significant genes (figure 1).

Example 1 -different biological effect sizes

In this first real-life example we provide a proof-of-principle that our method is able to correctly estimate relative power in a setting where different effect sizes are induced as a consequence of the treatment with compounds with different potency's and exposure times. This experiment was originally setup to examine the effects of PPARα-activation by synthetic as well as nutritional agonists on gene expression in various tissues in mice, including intestine. Comparative analysis in PPARα-null mice enabled the specific study of the regulatory role of this transcription factor. Previous, biology-driven analysis of the array data sets identified a hierarchy in PPARα-activating potency [8] (Hooiveld et al., manuscript in preparation). Test statistics were obtained by measuring the differential response on gene expression between wild-type and PPARα-null mice. We estimated power and minimal sample size for each combination of compound and exposure time separately, fixing the false discovery rate at 10%. Figure 2 shows the estimated densities of effect sizes, per compound and exposure time. These densities describe the standardized effect sizes among the differentially expressed genes. For each exposure time, the densities show increasingly heavier tails with the expected PPARα-activating potency of the compounds, from the weakest (fenofibrate) to the strongest (Wy14,643). Moreover, for all compounds there is relatively more up-regulation than down-regulation. When

Simulation Experiment: Compares estimated powerwith the true positive rate From the left-panel of table 2 we can see that, by increasing exposure time from six hours to five days, the estimated power was doubled for the more potent compound Wy14,643, and increased almost by a factor of four for the weaker compound fenofibrate. Indeed, the Wy14,643 reached around 60% power, where the fenofibrate had just 40% power. In addition, as the compound got less potent, the proportion of differentially expressed genes decreased, as expected. So it is clear that less potent compounds, or the ones exposed for a shorter time, yield little power to detect differences.

A more complete picture of the variation in power as the sample size increases is given in figure 3, for each combination of compound and exposure time. As expected, the average power increased with PPARα-activating potency and with exposure time. For example, if a power of 50% was desired to study the strong Wy14,643 compound, 10 arrays per group would be necessary considering a six hours exposure, whilst the same power would be already achieved using a five days exposure with four arrays per group. The other two compounds needed more than 15 samples per group for a six hours exposure to yield the same power, which after a five days exposure would be achieved with the fenofibrate compound with six samples per group.

Example 2 -different expression profiling platforms

This study was designed to explore the capabilities of expression profiling technology to identify subtle differences in gene expression. To estimate power, it is essential to estimate the proportion of non-differentially expressed genes (π 0 ). In table 2 (right-panel) we report π 0 for the different expression profiling platforms, as estimated by the method of Langaas et al. [3]. We used alternative methods for estimation of π 0 with essentially similar results (see Additional file 2). The estimated proportion of non-differentially expressed genes differ significantly between the different technologies, with Illumina displaying the highest proportion (right-panel of table 2). However, perhaps the most marked point is that the power is generally low for all technologies used. This is a reflection of the fact that the effects on expression are mostly subtle.

As before, a better overview is given by the estimated power curves in figure 4. The power continued to be generally low for the sample sizes considered, but increased with sample size. Here Solexa/Illumina, Agilent and Affymetrix are shown to be the best performing technologies for all sample sizes. The higher performance of Agilent compared to Affymetrix may be a consequence of the pilot study size, upon which the estimation of the power curve depends. As the number of samples differs per technology (one-color: 5 biological replicates per group; twocolor: 5 biological replicates, each with 2 technical replicates; and Solexa/Illumina deep sequencing: 4 biological replicates), there is considerably more certainty about estimates for Agilent and home-spotted, since their technical variability is better estimated, than for the other cases, especially so for Solexa/Illumina with the smallest sample size. Yet, for all sample sizes Solexa/Illumina was estimated as having the highest power, which is an indication of how precise and powerful this technology is.

Considering the two one-color platforms, Illumina yields less power than Affymetrix in this experiment (see rightpanel of table 2). This is also reflected by the estimated densities of effect sizes (not shown).

The largest effects in this experiment were observed for genes expressed in the low-intensity range, so technologies that are more sensitive to detect differences in this range are expected to perform better. In order to gain more insight into the sensitivity of the various technologies, we split genes into subsets according to them being expressed in the low-, medium-and high-intensity ranges, as given by wild-type median-expression ranking of Solexa/Illumina. For each technology and subset the power was estimated again for a range of sample sizes. Results are displayed in figures 5 and 6 for Affymetrix, Agilent and Solexa/Illumina, where figure 5 is ordered by intensity range and figure 6 is ordered by platform. In the lowintensity range, Solexa/Illumina displays considerably more power than the two microarray platforms, whilst in medium-and high-intensity ranges Affymetrix and Solexa/Illumina were comparable, with Agilent having a higher power ( figure 5). More interestingly perhaps, Solexa/Illumina displays comparable power across the three intensity ranges, whilst each one of the microarray platforms displays markedly less power in the low-intensity range compared to the intermediate-and high-intensity ranges ( figure 6). This is due to the presence of background, which is absent in Solexa/Illumina, but affects measurements in the low-intensity range of microarray platforms. So the largest effects found in the low-intensity range in this study are duly detected by Solexa/Illumina, but missed by the microarray platforms. This is further exemplified in Additional file 3, where the same comparison is made without filtering for small effect sizes.

Discussion

We aim at helping researchers to understand: a) how much change in sample size different expression profiling technologies require to yield the same power; and b) by how much the sample size required for a fixed power would change, if smaller effects were to be detected. In order to do so, we use experiments carefully designed to answer these questions. Our experiments use tissue samples from animal models, commonly used in practice. Not only our conclusions are easily applicable to the design of microarray experiments, but also our approach is available for researchers to use on their own data via our BioConductor package SSPA.

For our study, we use the method proposed by Ferreira et al. [2]. It makes use of a pilot study to estimate distribution of effect sizes and proportion of non-differentially expressed genes, based upon which power is estimated.

Various other methods to estimate sample size and power in microarray studies have been proposed. The first few assumed that multiple testing correction is done via control of the familywise error rate [20,21], which is unlikely to be the case in current practice. More recently, methods were proposed to handle the more commonly used control of false discovery rate [22][23][24][25]2]. The methods differ in how they treat the distribution of effect sizes; simpler methods assumed a fixed value for all differentially expressed genes, or took a subset of the largest effect sizes [22,25]. These are unlikely to correctly describe effects from experimental studies. Recently proposed methods [2,24,26] estimated the distribution of effect sizes from a pilot data set. Ferreira [2] assumes that the test statistics follow a normal distribution, which is unlikely to be the case. However, the extension of Ferreira's method to consider statistics with a more suitable Student-t distribution is not trivial. Indeed, to solve equation 2 is a much harder analytical problem under the Student-t distribution. Jørstad et al. [24] proposed to solve this problem by discretizing the effect sizes and then estimating the components of the resulting mixture. Ruppert et al. [26] proposed to estimate the density of effect size by a linear combination of splines optimized via penalized least squares. The number of parameters that needs to be estimated by both methods is considerably larger than by the method of Ferreira et al. [2], making them computationally much more intensive. For this reason, we chose to use Ferreira's method.

The simulation experiment shows that the estimated power is in agreement with the observed power. Impor-tant for the power estimation is the estimation of the proportion of non-differentially expressed genes. The π 0 estimates of the simulation experiment were all conservative and a little less than π 0 = 0.8, which may have led to overestimation of the power. However, the opposite was observed: the estimated power was found to be conservative. Generally speaking it is good to be on the conservative side in power calculations. However, this may not be the case in other applications. Indeed, we have observed that the relationship between observed and estimated power depends on both π 0 and the effect size distribution (data not shown). This issue deserves further investigation but is beyond the scope of this paper.

Both our simulation study as well as example 1 demonstrate that our method correctly ranks the power of different experimental scenarios and is thus suited to evaluate the relative capacity to identify differentially expressed genes. Some of the results obtained with this method are as expected: in example 1, potent compounds yield higher power than weaker compounds, as does longer exposure time compared to a shorter one (example 1). The hierarchy of PPARα-activating compounds found with this method confirms previously biologically-driven analysis [8] (Hooiveld et al. manuscript in preparation). However, some results are unexpected, such as that the power seems to increase little after a certain sample size (12, for 5-days exposure). Since the power for 12 samples per group is already acceptably high (80% for the stronger compound, over 70% for the weaker one), this result suggests that it is useless to analyse more than 12 samples to find differentially expressed genes.

The higher power for the longer-exposed compounds is associated with a markedly bimodal density of effect sizes ( figure 2). This density represents additional and important information for the researcher. In this case, for example, it can be seen that there are more genes up-regulated than down-regulated after short-term exposure, and that after long-term exposure this is more balanced, suggesting that up-regulation is kicked off earlier on. So, we have shown how the density of effect sizes reflects varying distributions of differential expression. This is intuitive and seems trivial, but no other method has produced this result before [6].

Example 2 shows great difference in performance for different expression profiling technologies to detect a subtle biological effect. Commercial microarray platforms perform better than the home-spotted mainly due to their higher reproducibility. The poor performance of Illumina may be attributed to the fact that less probes are used than Affymetrix. By estimating power separately for genes expressed in intensity ranges varying from low to high, we can clearly see the added value of the Solexa/Illumina deep sequencing technology compared with microarrays. Due to background, microarray platforms often cannot reliably measure expression in the low-intensity range. Indeed, Solexa/Illumina displays with 4 replicates per group the same power as Agilent and Affymetrix with 7 replicates per group (figure 4). The reduced power of microarray technologies is shown to be mainly due to lack of power to detect differential expression for lowly expressed genes, possibly due to presence of background intensities, a problem that does not affect Solexa/Illumina.

The estimated proportion of non-differentially expressed genes p 0 shows great difference between the different expression profiling technologies. This is likely due to differences in genome mapping [14] and hybridization efficiency. Since it is known that the same samples were hybridized to the different platforms, one might wonder if by using a common, fixed π 0 , value for all platforms more consistent results would have followed. That is not the case: power estimates are robust to variations of roughly 10% around the estimated π 0 value (data not shown). Moreover, we believe this should not be done, as effects of technical differences between the platforms would then have been ignored, which is undesirable. So, we believe that for each experiment π 0 should be esti-mated from the data, and that a sensitivity analysis may help reassure the researcher that power estimates are reliable.

Our example 2 is not trivial and no previous article, to our knowledge, has produced power and sample size calculations in such datasets. Indeed, the MAQC study [27,28] only involves technical replicates. So we do believe our work, by involving not only 4 different microarray platforms, but more importantly also a deep sequencing technology, does yield new knowledge. In particular, no previous work has shown that deep sequencing technology displays more power than microarrays in the presence of biological, in addition to technical, variability.

The objective of the study plays an important role in choosing the FDR-control level. Indeed, if results are meant to be further explored via high-dimensional tools such as pathway analysis, then there is interest in having a longer list of possibly interesting features, albeit with a larger FDR. This effectively expands the space on which the subsequent analysis tool will look for associations, improving the chance of finding more subtle, and for that less obvious, ones. If, on the other hand, the list of selected features must be validated by time-consuming and labour intensive techniques, then a shorter list obtained with as low an FDR as possible is the best. For our objectives, we needed only to have at least a few features selected at each instance in order to be able to draw comparisons, but preferably not too many, and we find this is achieved by controlling the FDR at 10%. We made a fast and easy-to-use implementation of a power and minimal sample size calculation method adapted from Ferreira et al. [2]. The only input needed is a set of test statistics obtained from the pilot data and the number of samples of the two groups. More details about this R package will appear elsewhere.

Conclusion

In conclusion relative power and sample size analysis can help researchers make important decisions about technology used for gene expression profiling. We showed that if interest lies in genes expressed in the low-intensity range Solexa/Illumina deep sequencing is the superieur technology compared to microarray technology. Furthermore, we have implemented our method via a BioConductor package so that other researchers can use it on their own pilot data.

Authors' contributions

MI performed the data analysis. PACH provided the platform comparison data and interpretations of the results, and contributed to the manuscript. PP performed the differential gene expression analysis of the platform comparison. GJEJH provide the nutrigenomics data and contributed to the manuscript. JTD was involved in many useful discussions. GJBO actively supported the deep sequencing experiments. JMB was involved in study design and interpretation of results. MI and RXM wrote the manuscript. All authors read and approved of the final manuscript.

Additional material

Figure 1 SimulationExample 1 :
11Experiment: Compares estimated powerwith the true positive rate. Box-and-whisker plot of the estimated average power and true positive rate for all the simulations, for the three different effect size distributions. The Power is displayed on the y-axis and the different simulation scenario's are displayed on the x-axis fromleft to right: weak-, intermediate-and strong-effect sizes respectively. Estimated densities of effect sizes for combina-tions of compounds and exposure times Figure 2 Example 1: Estimated densities of effect sizes for combinations of compounds and exposure times. The upper panel shows densities for 6 hours exposure time, the lower panel for 5 days exposure time, with on the x-axis the standardized effect size and on the y-axis the estimated densities. The different compounds Wy14,643 (strong-), C18:3 (intermediate-) and fenofibrate (weak-activating potency) correspond respectively to the solid(blue)-, dashed(pink)and dotted(green)-lines. times (upper and lower panels), it is obvious that longer exposure yields more pronounced effects, represented by the bimodal densities, compared with shorter exposure, where densities representing the mixture of up-and down-regulation on genes display a single overlapping mode, with a subtle separate effect seen on the lower tail for the stronger compounds.

Example 1 :
1Power curves for the different combinations of compounds and effect sizes Figure 3 Example 1: Power curves for the different combinations of compounds and effect sizes. The upper panel shows the power for 6 hours exposure time, the lower panel the 5 days exposure time, with on the x-axis the sample size and on the y-axis the estimated power using a 10% FDR. The different compounds Wy14,643 (strong-), C18:3 (intermediate-) and fenofibrate (weak-activating potency) correspond respectively to the solid(blue)-, dashed(pink)-and dotted(green)-lines. curves for the different expression profil-ing platforms Figure 4 Example 2: Power curves for the different expression profiling platforms. The five curves solid(blue), shortdashed(pink), dotted(green), dot-dashed(red) and twodashed(orange) correspond to Solexa/Illumina, Agilent, Affymetrix, Illumina, and home-spotted oligonucleotide arrays respectively.On the x-axis the sample size is displayed and on the y-axis is the estimated power using a 10% FDR.

Example 2 :Example 2 :
22Power curves for three technologies and three intensity ranges (ordered by intensity range)Figure 5Example 2: Power curves for three technologies and three intensity ranges (ordered by intensity range). The three panels show power curves for Affymetrix, Agilent and Solexa/Illumina, with on the x-axis the sample size and on the y-axis the estimated power using a 10% FDR. Power curves for three technologies and three intensity ranges (ordered by platform)Figure 6Example 2: Power curves for three technologies and three intensity ranges (ordered by platform). The three panels show power curves for Affymetrix, Agilent and Solexa/Illumina, with on the x-axis the sample size and on the y-axis the estimated power using a 10% FDR.

Table 1 :
1Overview of data sets in Example 1 and 2Experiment
Genes
group A 1
group B

strong, 6 hours
16539
4 (wild-type)
4 (null)
intermediate, 6 hours
16539
4 (wild-type)
5 (null)
weak, 6 hours
16539
5 (wild-type)
5 (null)
strong, 5 days
16539
4 (wild-type)
4 (null)
weak, 5 days
16539
4 (wild-type)
4 (null)

Affymetrix
45101
5 (wild-type)
5 (transgenic)
Agilent
41232
5 (wild-type)
5 (transgenic)
Illumina
46120
5 (wild-type)
5 (transgenic)
home-spotted
21771
5 (wild-type)
5 (transgenic)
Solexa/Illumina
34477
4 (wild-type)
4 (transgenic)

Table 2 :
2Estimated power and adjusted proportions of non-differentially expressed genes for the pilot studies.Example 1

(page number not for citation purposes)
The sample size given refer to the number of biologically independent samples.
AcknowledgementsThis work was part of the BioRange programme of the Netherlands Bioinformatics Centre (NBIC), which is supported by a BSIK grant through the Netherlands Genomics Initiative (NGI). The work was conducted within the Centre for Medical Systems Biology (CMSB), established by the Netherlands Genomics initiative/Netherlands Organisation for Scientific Research (NGI/NWO).Additional file 1Adjusting the proportion of non-differentially expressed genes. A description of how the method for adjusting the the proportion of non-differentially expressed genes works. Click here for file [http://www.biomedcentral.com/content/supplementary/1471-2164-10-439-S1.PDF]Additional file 2Estimates of proportion of non-differentially expressed genes (π 0 ) using three different methods(
Statistical design and the analysis of gene expression microarray data. M Kerr, G Churchill, Genetics Research. 772Kerr M, Churchill G: Statistical design and the analysis of gene expression microarray data. Genetics Research 2001, 77(2):123-128.

Approximate Power and Sample Size Calculations with the Benjamini-Hochberg Method. J Ferreira, A Zwinderman, International Journal of Biostatistics. 218Ferreira J, Zwinderman A: Approximate Power and Sample Size Calculations with the Benjamini-Hochberg Method. International Journal of Biostatistics 2006, 21:8.

Estimating the proportion of true null hypotheses, with application to DNA microarray data. M Langaas, B Lindqvist, E Ferkingstad, Journal of the Royal Statistical Society, Series B. 674Langaas M, Lindqvist B, Ferkingstad E: Estimating the proportion of true null hypotheses, with application to DNA microarray data. Journal of the Royal Statistical Society, Series B 2005, 67(4):555-572.

Adaptive linear step-up procedures that control the false discovery rate. Y Benjamini, A Krieger, D Yekutieli, Biometrics. 933Benjamini Y, Krieger A, Yekutieli D: Adaptive linear step-up pro- cedures that control the false discovery rate. Biometrics 2006, 93(3):491-507.

Controlling the false discovery rate: a practical and powerful approach to multiple testing. Y Benjamini, Y Hochberg, Journal of the Royal Statistical Society, Series B. 57Benjamini Y, Hochberg Y: Controlling the false discovery rate: a practical and powerful approach to multiple testing. Journal of the Royal Statistical Society, Series B 1995, 57:289-300.

Approximate Sample Size Calculations with Microarray Data: An Illustration. J Ferreira, A Zwinderman, Statistical Applications in Genetics and Molecular Biology. 525Ferreira J, Zwinderman A: Approximate Sample Size Calcula- tions with Microarray Data: An Illustration. Statistical Applica- tions in Genetics and Molecular Biology 2006, 5:25.

Peroxisome proliferator-activated receptors: nuclear control of metabolism. B Desvergne, W Wahli, Endocrine Reviews. 205Desvergne B, Wahli W: Peroxisome proliferator-activated receptors: nuclear control of metabolism. Endocrine Reviews 2006, 20(5):649-688.

Effect of synthetic dietary triglycerides: a novel research paradigm for nutrigenomics. L Sanderson, P De Groot, G Hooiveld, A Koppen, E Kalkhoven, M Müller, S Kersten, PLoS ONE. 321681Sanderson L, de Groot P, Hooiveld G, Koppen A, Kalkhoven E, Müller M, Kersten S: Effect of synthetic dietary triglycerides: a novel research paradigm for nutrigenomics. PLoS ONE 2008, 3(2):e1681.

Genome-wide analysis of PPARalpha activation in murine small intestine. M Bünger, Bosch H Van Den, Meijde J Van Der, S Kersten, G Hooiveld, M Müller, Physiological Genomics. 302Bünger M, Bosch H van den, Meijde J van der, Kersten S, Hooiveld G, Müller M: Genome-wide analysis of PPARalpha activation in murine small intestine. Physiological Genomics 2007, 30(2):192-204.

Evolving gene/transcript definitions significantly alter the interpretation of GeneChip data. M Dai, P Wang, A Boyd, G Kostov, B Athey, E Jones, W Bunney, R Myers, T Speed, H Akil, S Watson, F Meng, Nucleic Acids Research. 3320175Dai M, Wang P, Boyd A, Kostov G, Athey B, Jones E, Bunney W, Myers R, Speed T, Akil H, Watson S, Meng F: Evolving gene/tran- script definitions significantly alter the interpretation of GeneChip data. Nucleic Acids Research 2005, 33(20):e175.

A model-based background adjustment for oligonucleotide expression arrays. Z Wu, R Irizarry, R Gentleman, F Martinez-Murillo, F Spencer, Journal of the American Statistical Association. 99Wu Z, Irizarry R, Gentleman R, Martinez-Murillo F, Spencer F: A model-based background adjustment for oligonucleotide expression arrays. Journal of the American Statistical Association 2004, 99:909-917.

Deep Sequencing-based Expression analysis shows Major Advances in Robustness, Resolution and Inter-lab Portability over Five Microarray Platforms. P Smyth G ; T Hoen, Y Ariyurek, H Thygesen, E Vreugdenhil, R Vossen, J Boer, G Van Ommen, J Den Dunnen, Statistical Applications in Genetics and Molecular Biology. 321Nucleic Acid ResearchSmyth G: Linear models and empirical Bayes methods for assessing differential expression in microarray experiments. Statistical Applications in Genetics and Molecular Biology 2004, 3:3. 13. 't Hoen P, Ariyurek Y, Thygesen H, Vreugdenhil E, Vossen R, Boer J, van Ommen G, den Dunnen J: Deep Sequencing-based Expres- sion analysis shows Major Advances in Robustness, Resolu- tion and Inter-lab Portability over Five Microarray Platforms. Nucleic Acid Research 2008, 36(21):.

Can subtle changes in gene expression be consistently detected with different microarray platforms?. P Pedotti, P Hoen, E Vreugdenhil, G Schenk, R Vossen, Y Ariyurek, G Van Ommen, J Den Dunnen, J Boer, R De Menezes, BMC Genomics. 9124Pedotti P, 't Hoen P, Vreugdenhil E, Schenk G, Vossen R, Ariyurek Y, van Ommen G, den Dunnen J, Boer J, de Menezes R: Can subtle changes in gene expression be consistently detected with dif- ferent microarray platforms? BMC Genomics 2008, 9(124):.

Team: R: A Language and Environment for Statistical Computing. Development Core, R Development Core Team: R: A Language and Environment for Statis- tical Computing 2007 [http://www.R-project.org].

Bioconductor: Open software development for computational biology and bioinformatics. R Gentleman, V Carey, D Bates, B Bolstad, M Dettling, S Dudoit, B Ellis, L Gautier, Y Ge, J Gentry, K Hornik, T Hothorn, W Huber, S Iacus, R Irizarry, F Leisch, C Li, M Maechler, A Rossini, G Sawitzki, C Smith, G Smyth, L Tierney, J Yang, J Zhang, R Foundation for Statistical Computing. Vienna, Austria580R Foundation for Statistical Computing, Vienna, Austria 16. Gentleman R, Carey V, Bates D, Bolstad B, Dettling M, Dudoit S, Ellis B, Gautier L, Ge Y, Gentry J, Hornik K, Hothorn T, Huber W, Iacus S, Irizarry R, Leisch F, Li C, Maechler M, Rossini A, Sawitzki G, Smith C, Smyth G, Tierney L, Yang J, Zhang J: Bioconductor: Open soft- ware development for computational biology and bioinfor- matics. Genome Biology 2004, 5:R80.

Laan van der: Multiple Testing Procedures and Applications to. K Pollard, M S Dudoit, Genomics Springer Series in Statistics. SpringerPollard K, Dudoit MS, Laan van der: Multiple Testing Procedures and Applications to Genomics Springer Series in Statistics, Springer, New York; 2008.

A direct approach to false discovery rates. J Storey, Journal of the Royal Statistical Society, Series B. 64Storey J: A direct approach to false discovery rates. Journal of the Royal Statistical Society, Series B 2002, 64:479-498.

Multivariate Data Visualization with R. S: Deepayan, Lattice, Deepayan S: Lattice: Multivariate Data Visualization with R. 2008 [http://lmdvr.r-forge.r-project.org/].

Power and sample size for DNA microarray studies. M Lee, G Whitmore, Statistics in Medicine. 21Lee M, Whitmore G: Power and sample size for DNA microar- ray studies. Statistics in Medicine 2002, 21:3543-3570.

Sample Size Estimation for Microarray Experiments. G Warnes, P Liu, Warnes G, Liu P: Sample Size Estimation for Microarray Experiments. 2006 [http://www.bioconductor.org/packages/2.3/ bioc/vignettes/ssize/inst/doc/ssize.pdf].

Sample size for FDR-control in microarray data analysis. S H Jung, Bioinformatics. 2114Jung SH: Sample size for FDR-control in microarray data anal- ysis. Bioinformatics 2005, 21(14):3079-3104.

Quick calculation for sample size while controlling false discovery rate with application to microarrays. Bioinformatics. P Liu, J Hwang, 26Liu P, Hwang J: Quick calculation for sample size while control- ling false discovery rate with application to microarrays. Bio- informatics 2007, 26(6):739-746.

A mixture model approach to sample size estimation in two-sample comparative microarray experiments. T Jørstad, H Midelfart, A Bones, BMC Bioinformatics. 9117Jørstad T, Midelfart H, Bones A: A mixture model approach to sample size estimation in two-sample comparative microar- ray experiments. BMC Bioinformatics 2008, 9(117):.

Practical guidelines for assessing power and false discovery rate for fixed sample size in microarray experiments. T Tong, H Zhao, Statistics in Medicine. 27Tong T, Zhao H: Practical guidelines for assessing power and false discovery rate for fixed sample size in microarray experiments. Statistics in Medicine 2008, 27:1960-1972.

Exploring the information in p-values for the analysis and planning of multiple-test experiments. D Ruppert, D Nettleton, J Hwang, Biometrics. 632Ruppert D, Nettleton D, Hwang J: Exploring the information in p-values for the analysis and planning of multiple-test exper- iments. Biometrics 2007, 63(2):483-95.

L Shi, L H Reid, W D Jones, R Shippy, J A Warrington, S C Baker, P J Collins, F De Longueville, E S Kawasaki, K Y Lee, Y Luo, Y A Sun, J C Willey, R A Setterquist, G M Fischer, W Tong, Y P Dragan, D J Dix, F W Frueh, F M Goodsaid, D Herman, R V Jensen, C D Johnson, E K Lobenhofer, R K Puri, U Schrf, J Thierry-Mieg, C Wang, M Wilson, P K Wolber, L Zhang, S Amur, W Bao, C C Barbacioru, A B Lucas, V Bertholet, C Boysen, B Bromley, D Brown, A Brunner, R Canales, X M Cao, T A Cebula, J J Chen, J Cheng, T M Chu, E Chudin, J Corson, J C Corton, L J Croner, C Davies, T S Davison, G Delenstarr, X Deng, D Dorris, A C Eklund, X Hui Fan, H Fang, S Fulmer-Smentek, J C Fuscoe, K Gallagher, W Ge, L Guo, X Guo, J Hager, P K Haje, J Han, T Han, H C Harbottle, S C Harris, E Hatchwell, C A Hauser, S Hester, H Hong, P Hurban, S A Jackson, H Ji, C R Knight, W P Kuo, J E Leclerc, S Levy, Q Z Li, C Liu, Y Liu, M J Lombardi, Y Ma, S R Magnuson, B Maqsodi, T Mcdaniel, N Mei, O Myklebost, B Ning, N Novoradovskaya, M S Orr, T W Osborn, A Papallo, T A Patterson, R G Perkins, E H Peters, R Peterson, K L Philips, P S Pine, L Pusztai, F Qian, H Ren, M Rosen, B A Rosenzweig, R R Samaha, M Schena, G P Schroth, S Shchegrova, D D Smith, F Staedtler, Z Su, H Sun, Z Szallasi, Z Tezak, D Thierry-Mieg, K L Thompson, I Tikhonova, Y Turpaz, B Vallanat, C Van, S J Walker, S J Wang, Y Wang, R Wolfinger, A Wong, J Wu, C Xiao, Q Xie, J Xu, W Yang, L Zhang, S Zhong, Y Zong, W J Slikker, The MicroArray Quality Control (MAQC) project shows inter-and intraplatform reproducibility of gene expression measurements. 24Shi L, Reid LH, Jones WD, Shippy R, Warrington JA, Baker SC, Collins PJ, de Longueville F, Kawasaki ES, Lee KY, Luo Y, Sun YA, Willey JC, Setterquist RA, Fischer GM, Tong W, Dragan YP, Dix DJ, Frueh FW, Goodsaid FM, Herman D, Jensen RV, Johnson CD, Lobenhofer EK, Puri RK, Schrf U, Thierry-Mieg J, Wang C, Wilson M, Wolber PK, Zhang L, Amur S, Bao W, Barbacioru CC, Lucas AB, Bertholet V, Boysen C, Bromley B, Brown D, Brunner A, Canales R, Cao XM, Cebula TA, Chen JJ, Cheng J, Chu TM, Chudin E, Corson J, Corton JC, Croner LJ, Davies C, Davison TS, Delenstarr G, Deng X, Dorris D, Eklund AC, hui Fan X, Fang H, Fulmer-Smentek S, Fuscoe JC, Gal- lagher K, Ge W, Guo L, Guo X, Hager J, Haje PK, Han J, Han T, Har- bottle HC, Harris SC, Hatchwell E, Hauser CA, Hester S, Hong H, Hurban P, Jackson SA, Ji H, Knight CR, Kuo WP, LeClerc JE, Levy S, Li QZ, Liu C, Liu Y, Lombardi MJ, Ma Y, Magnuson SR, Maqsodi B, McDaniel T, Mei N, Myklebost O, Ning B, Novoradovskaya N, Orr MS, Osborn TW, Papallo A, Patterson TA, Perkins RG, Peters EH, Peterson R, Philips KL, Pine PS, Pusztai L, Qian F, Ren H, Rosen M, Rosenzweig BA, Samaha RR, Schena M, Schroth GP, Shchegrova S, Smith DD, Staedtler F, Su Z, Sun H, Szallasi Z, Tezak Z, Thierry-Mieg D, Thompson KL, Tikhonova I, Turpaz Y, Vallanat B, Van C, Walker SJ, Wang SJ, Wang Y, Wolfinger R, Wong A, Wu J, Xiao C, Xie Q, Xu J, Yang W, Zhang L, Zhong S, Zong Y, Slikker WJ: The MicroArray Quality Control (MAQC) project shows inter-and intraplat- form reproducibility of gene expression measurements. Nature Biotechnology 2006, 24(9):1151-61.

Performance comparison of one-color and twocolor platforms within the MicroArray Quality Control (MAQC) project. T A Patterson, E K Lobenhofer, S B Fulmer-Smentek, P J Collins, T M Chu, W Bao, H Fang, E S Kawasaki, J Hager, I R Tikhonova, S J Walker, L Zhang, P Hurban, F De Longueville, J C Fuscoe, W Tong, L Shi, R D Wolfinger, Nature Biotechnology. 249Patterson TA, Lobenhofer EK, Fulmer-Smentek SB, Collins PJ, Chu TM, Bao W, Fang H, Kawasaki ES, Hager J, Tikhonova IR, Walker SJ, Zhang L, Hurban P, de Longueville F, Fuscoe JC, Tong W, Shi L, Wolf- inger RD: Performance comparison of one-color and two- color platforms within the MicroArray Quality Control (MAQC) project. Nature Biotechnology 2006, 24(9):1140-50.