Evaluating the effect of innovative motivation and supervision approaches on community health worker performance and retention in Uganda and Mozambique: study protocol for a randomised controlled trial Evaluating the effect of innovative motivation and supervision approaches on community health worker performance and retention in Uganda and Mozambique: study protocol for a randomised controlled trial
2015

Karin Källander
Daniel Strachan
Seyi Soremekun
Zelee Hill
Raghu Lingam
James Tibenderana
Frida Kasteng
Anna Vassall
Sylvia Meek
Betty Kirkwood
Evaluating the effect of innovative motivation and supervision approaches on community health worker performance and retention in Uganda and Mozambique: study protocol for a randomised controlled trial Evaluating the effect of innovative motivation and supervision approaches on community health worker performance and retention in Uganda and Mozambique: study protocol for a randomised controlled trial

Trials
16157201510.1186/s13063-015-0657-6Källander et al. S T U D Y P R O T O C O L Open Access
Background: If trained, equipped and utilised, community health workers (CHWs) delivering integrated community case management for sick children can potentially reduce child deaths by 60%. However, it is essential to maintain CHW motivation and performance. The inSCALE project aims to evaluate, using a cluster randomised controlled trial, the effect of interventions to increase CHW supervision and performance on the coverage of appropriate treatment for children with diarrhoea, pneumonia and malaria.Methods/Design: Participatory methods were used to identify best practices and innovative solutions. Quantitative community based baseline surveys were conducted to allow restricted randomisation of clusters into intervention and control arms. Individual informed consent was obtained from all respondents. Following formative research and stakeholder consultations, two intervention packages were developed in Uganda and one in Mozambique. In Uganda, approximately 3,500 CHWs in 39 clusters were randomised into a mobile health (mHealth) arm, a participatory community engagement arm and a control arm. In Mozambique, 275 CHWs in 12 clusters were randomised into a mHealth arm and a control arm. The mHealth interventions encompass three components: 1) free phone communication between users; 2) data submission using phones with automated feedback, messages to supervisors for targeted supervision, and online data access for district statisticians; and 3) motivational messages. The community engagement arm in Uganda established village health clubs seeking to 1) improve the status and standing of CHWs, 2) increase demand for health services and 3) communicate that CHWs' work is important. Process evaluation was conducted after 10 months and end-line surveys will establish impact after 12 months in Uganda and 18 months in Mozambique. Main outcomes include proportion of sick children appropriately treated, CHW performance and motivation, and cost effectiveness of interventions.Discussion: Study strengths include a user-centred design to the innovations, while weaknesses include the lack of a robust measurement of coverage of appropriate treatment. Evidence of cost-effective innovations that increase motivation and performance of CHWs can potentially increase sustainable coverage of iCCM at scale.

Methods/Design

Study aim and objectives

The aim of this study is to develop and evaluate innovative approaches for increasing CHW supervision, motivation, performance and retention, and assess the impact of these interventions on coverage of appropriate treatment for children with diarrhoea, pneumonia and malaria in Uganda and Mozambique.

Primary objectives Formative phase

The primary objectives during the formative phase of this study are as follows:

1. To identify innovative solutions with the potential to increase coverage of iCCM and improve its quality through better CHW performance, motivation and retention. 2. To assess the feasibility of these innovative solutions and assess their acceptability among community members, CHWs, facility-based health workers, sub-national and national health authorities.

Evaluation phase

The primary objectives during the evaluation phase of this study are as follows:

1. To evaluate the impact of the selected interventions on CHW performance, motivation, retention, and coverage of appropriate treatment for children with diarrhoea, pneumonia and malaria. 2. To assess the costs of the interventions and investigate their potential cost-effectiveness.

Dissemination phase

The primary objective during the dissemination phase of this study is as follows:

1. To promote the implementation and spread of iCCM by sharing with the Ministry of Health (MoH), sub-national health authorities and stakeholders the experiences and findings that improve coverage and quality of iCCM.

Study context

The pre-packed chloroquine-sulphadoxine/pyrimethamine for presumptive treatment of fever by volunteer CHWs. In 2004, after the rebel war, UNICEF supported the MoH to pilot the implementation of home-based care of children with diarrhoea, pneumonia and malaria in Northern Uganda using oral rehydration solution (ORS), cotrimoxazole and artemether-lumefantrine (Coartem; Novartis). Beginning in 2006, the MoH instituted the Village Health Team (VHT) strategy to strengthen community capacity for health promotion and health service delivery. A VHT has 4 to 5 volunteers (referred to as 'VHTs') selected by the community and equipped with basic materials (storage boxes, badges, T-shirts and gumboots). In 2010, Uganda launched iCCM, whereby two community-elected VHTs per village were trained to manage children 2 to 59 months with malaria, pneumonia, and diarrhoea using diagnostics (malaria rapid diagnostic tests (mRDTs) and respiratory timers for detection of fast breathing) and treatment (dispersible artemether-lumefantrine, dispersible amoxicillin, ORS, zinc and rectal artesunate). These VHTs also carry out home visits for active case detection and referral of sick newborns. Children with severe malaria and pneumonia are given pre-referral treatment with rectal artesunate and amoxicillin, respectively. The VHTs are supervised by health workers from the nearest health facility who conduct quarterly supervision group meetings at the health facility, at which time the VHTs also replenish their medicine stocks. A transport refund of about US$ 4 is given to VHTs at these meetings. Health workers supplement group meetings at health facilities with monthly community visits to VHTs during the first 3 months after training [10] and subsequently on at least an annual basis.

In Mozambique, the MoH introduced the concept of the lay CHW through the ' Agente Polivalente Elementar' (APE) in the 1970s to support public health activities at the district level [11]. The programme, which was initially very strong despite implementation challenges, weakened in the 1980s due to a shift in focus in the health sector: from provision of primary health care at the community level to provision at the facility level, partly influenced by system breakdowns during the civil war. Some of the original APEs still continued to work, but a lack of supervision, re-training and effective integration into the health system resulted in low programme effectiveness. In 2007, the MoH held a national level meeting to refocus the roles and responsibilities of the APEs, and in 2010 the Minister endorsed the APE operational plan and revised curriculum [12]. Recommendations highlighted that APEs should be respected and trusted individuals selected by their community and trained by the MoH to promote better health within their community and provide preventive and curative care. They receive Kit C -a basic medicine kit including ORS and zinc for diarrhoea, amoxicillin for pneumonia, artemether-lumefantrine for uncomplicated malaria, and pre-referral rectal artesunate for children with severe malaria. In 2013, mRDTs and adult artemether-lumefantrine were also provided in a separate kit from the National Malaria Control Programme. The APEs diagnose, refer, and treat children between 2 and 59 months of age for diarrhoea and pneumonia, as well as all age groups with confirmed malaria. Guidelines recommended that the majority of their time (80%) should be spent in the communities doing house-to-house visits, holding health education talks ('palestras'), and conducting active case detection and referral of sick newborns and pregnant women. As per the APE programme guideline, the APEs receive quarterly supervision from the nearest health facility (when medicine kits are replenished) in addition to six-monthly supervision by outreach teams from the district. APEs receive a monthly allowance of 60% of the minimum salary level (US$ 40).

Study sites inSCALE Uganda

inSCALE operates in eight districts in Midwestern Uganda (Buliisa, Masindi, Kibaale, Kyegegwa, Kyankwazi, Kiryandongo, Kiboga and Hoima), where iCCM implementation has been supported by the Malaria Consortium through a grant from the Canadian International Development Agency (CIDA) since August 2010 ( Figure 1). Under the CIDA grant, Malaria Consortium provides technical support to the programme design; trains Master and district trainers; facilitates the training of VHTs; supplies health facilities and VHTs with medicines; and supports health facilities and districts to do monitoring and evaluation as well as supportive supervision. The districts where the projects have been implemented have an estimated 1.8 million people living in approximately 4,000 villages, with 20% being children under 5 years of age. The population is multiethnic and has a variety of cultural practices, with the majority of the population being able to read and write. It includes nomadic cattle herders, fishing communities, and peasant farmers. In the area, approximately 10,000 VHTs are operating, with 5,700 VHTs (approximately two per village or 1 per 250 people on average) trained in iCCM for 6 days between July 2010-June 2011. Each VHT sees, on average, 14 children every month [13].

inSCALE Mozambique inSCALE operates in all 12 districts in Inhambane Province (Funhalouro, Govuro, Homoíne, Inharrime, Inhassoro, Jangamo, Mabote, Massinga, Morrumbene, Panda, Vilankulo, and Zavala). Since 2011, iCCM implementation in seven of these districts was supported by Malaria Consortium, while UNICEF supported the other five ( Figure 2). Under the same CIDA grant as for Uganda, the Malaria Consortium supported the revisions of the training manuals and job aids for the APEs; the implementation of the 4 months training; the provision of APE medicines to the provincial warehouse; and the implementation of supervision and monitoring and evaluation activities. The province and districts where the programmes were implemented have an estimated 1.3 million people living in approximately 145 villages (bairros), with 18% being children under 5 years of age. There are three ethnic groups whose constituents communicate mostly in Portuguese (the official language in Mozambique), Chope (the southern part of Inhambane) and Matsua (the northern part of  Inhambane). The majority of the population are subsistence farmers with little or no literacy skills. In the province, 275 APEs were trained between August and December 2011 in the new curriculum, which includes a module of iCCM. Each APE serves approximately 500 to 2,000 people and sees, on average, 50 patients every month.

The inSCALE interventions

inSCALE developed two integrated intervention packages based on extensive formative research [14][15][16]. At the beginning of the project, several reviews and consultations took place to ensure that the interventions designed drew on experience from previous work and appropriate theory. There was an additional focus on using these sources to identify areas of legitimate need with genuine potential for innovation. An initial team meeting determined what to review with three strands of enquiry identified and described below, and tasks were allocated to team members with relevant expertise, written up in reports [17] and presented and discussed in subsequent meetings:

1 Literature reviews of the areas of supervision, motivation theory, incentives, data use in quality improvement, mobile health (mHealth), business management and human resources, agriculture and community development. 2. Reviews of the history of Uganda and Mozambique as it relates to CHW programmes. 3. Consultations with stakeholders, academic and programme implementers working with healthfocused CHWs in a variety of contexts to identify strategies that could increase CHW motivation and retention.

These reviews were used to inform the development of a conceptual framework on CHW motivation to perform and remain in role and to develop a list of potential alternative solutions to these problems -called 'innovations'. Using the reviews and the theoretical findings, the inSCALE team selected a 'long list' of potential innovations to test based on ratings for a) impact potential, b) ability to fulfil required needs (in relation to CHW motivation and performance), c) acceptability, d) feasibility and e) sustainability. The long list of innovations fell into two clusters: a mHealth arm and a participatory community engagement arm. Both of these approaches aimed to positively influence CHW motivation, retention and performance by promoting their sense of collective identity.

The long list of potential innovations was presented to key personnel at the MoH in each country who gave their feedback on feasibility, acceptability and impact using pile sorting methodology [18], the results of which were used to eliminate several innovations felt to be unfeasible or unacceptable. Consequently, a list was created of 15 potential activities identified across the two intervention packages in the two countries. Formative research, gauging views of CHWs, their supervisors, district officials and key programme implementers, as well as caregivers, heads of households and traditional community leaders, was then conducted to ensure that innovations were acceptable and feasible at the community level, filled a need, and to determine the details of the approach (Strachan DL, Källander K, Nakirunda M, Ndima S, Muiambo A, Hill Z for the inSCALE study group, unpublished paper). Data were also collected on the current work of the CHWs and support that they received in order to understand motivation and retention and to identify priority issues. The results were used to finalise the two intervention packages. These consisted of a number of innovations ( Figure 3) with both intervention packages implemented in Uganda and one in Mozambique [15]. Materials developed to support these packages were extensively reviewed and pre-tested.

Intervention package 1 -The participatory community engagement approach In Uganda, the inSCALE project is supporting the implementation of an innovative participatory community engagement approach called the Village Health Clubs (VHCs). The VHC aims to improve child health through a community-led forum with the CHWs as the main focus point. VHC meetings are intended to provide a forum where CHWs and community members who are part of the club can work together to identify child health and CHWs challenges. They use village networks, knowledge, creativity and other community assets to help solve child health problems. Village health clubs are implemented through a four-step learning, planning and action cycle facilitated by the CHW. The time between each step is 1, 2 or 4 weeks depending on the decision taken by the club members. Health club members rank child health challenges faced by their community using picture cards. They discuss solutions, which include supporting the functioning of CHW services, and take actions to meet these challenges. Throughout the cycle the health clubs monitor and report on their progress.

Village health clubs are open to all members of the village and designed to be fun while focusing on the CHW as the main village health asset. A participatory empowerment approach is adopted where the CHW (as facilitator) encourages members to plan and carry out the club's activities. They also promote group decisionmaking and ownership. Solutions to village health challenges developed by village members are a key focus of the village health club approach. Table 1 describes the contents of the participatory community engagement intervention in Uganda.

Intervention package 2 -the mHealth supported approach

In Mozambique and Uganda, inSCALE is giving CHWs phones with which they can send their weekly reports, receive immediate automated feedback on performance and access a closed user group with their supervisors in order to increase communication and support. Every month, a motivational performance-related SMS is sent out, and supervisors receive weekly automated actionable messages for CHWs who are performing at high or low standards. Table 2 describes the components of the mHealth intervention arm for Mozambique and Uganda.

Supportive activities

Several supportive actions took place to promote the interventions before their introduction in the communities. In both countries, consensus dialogue meetings on the planned interventions, the randomisation process and the evaluation of the RCT were organised with local leaders, using printed sensitisation briefs and presentations. In Uganda, meetings were held with district and sub-county leaders, and in Mozambique, two sensitisation meetings were held with provincial and district leaders. After these meetings, the districts and sub county leaderships (Uganda) and provincial and district stakeholders (Mozambique) pledged their support towards the interventions and approved the forthcoming randomisation process. Figure 4 outlines the conceptual framework for the evaluation of the inSCALE interventions and how they link to the main outcomes. Both interventions assume that motivation is the key to performance and retention, and that when these improve, the result will be improved access to services and, ultimately, improvements in appropriate treatment of sick children. The mHealth supported approach, using mobile phones with unlimited talk time and custom-made applications, is intended to improve the CHWs' sense of connectedness to the health system and to their CHW peers. As a result of the increased support and performance-related feedback, it is hoped that the CHWs will feel an improved sense of status. The free talk time is expected to enable supervisors to provide targeted supervision, while the reminder messages and respiratory timers are meant to improve performance in diagnosis, treatment and referral. In Mozambique, the CHW performance was explored during the formative phase and was found to be lower than in Uganda, presumably because of the lack of effective job aids. Therefore, additional multimedia phone software was developed for Mozambique, which provides the CHW with a step-by-step guide to the consultation steps as well as treatment guidance when all steps are complete. • open to all

Effect pathways of inSCALE's interventions

• village owned

• intended to support VHT work

• strength based (using village assets), and

• fun and focused

Training and club facilitation Two VHT club facilitators from each village trained for four days to encourage club members to plan and carry out the club's activities using an action and planning cycle.

Accessories and materials • Picture cards for ranking common child health problems

Instructional VHC flip books

• Stamps and other stationary to help with the establishment and operation of the clubs in the communities Supervisor support and patrons Village leaders were appointed as patrons and sensitised to support the mobilisation of the communities to join the clubs.

VHT supervisors, health assistants and sub-county development officers were trained by District Health Educators and Malaria Consortium master trainers in effective supervision skills using a core competency assessment tool, and as trainers of VHTs in the VHC intervention.

Numbers of users A total of 880 VHTs across the eight districts, to facilitate the set-up of 440 VHCs. Software 'InSCALE APE CommCare app' providing audio and images for each step in the sick child assessment process. Individual patient data is synchronised with an online database, and aggregated patient data and drug stock reports are submitted on a weekly basis.

'inSCALE Mobile VHT system' to send aggregated weekly reports on patients seen (sex, mRDT results, symptoms and classification of signs, treatment given and outcome of treatment) and current drug stock levels.

Respiratory rate application where user presses the centre button for each breath observed during one minute added to the phone tool box.

Respiratory rate application where user taps the screen for each breath observed during one minute built into the electronic algorithm.

Feedback messages

Off-line decision support for diagnosis, treatment and referral provided at the end of the consultation process.

Relevant and personalised feedback messages based on submitted data sent instantly after reports are received.

Supervisor support Automated weekly and monthly reports are emailed to health facility and district supervisors on APE activities, suggesting targeted follow-up actions.

Automated SMS sent to supervisors flagging problems and strengths identified in the data submitted, and alerting supervisors about VHTs requiring targeted supervision.

Trained as trainers and in effective supervision skills using paper based core competency assessment tools Supervisors' phones programmed with an electronic checklist related to core APE competencies linked to an APE Performance Checklist.

The community engagement approach aims to enhance the perceived value of the CHW, both for themselves and for the communities they serve, through inclusive and participatory local activities. The actions of health club members will help communicate to the CHW and village members that their work is important, of value and appreciated. In doing so, village health clubs (VHCs) will improve the status and standing of CHWs as key village health assets, increasing their motivation and the quality of service provision. VHCs will also increase the community's understanding of what CHWs can and cannot do and their potential to improve child health in the village. This increased understanding could mean that a greater demand for CHW services and, correspondingly, an increase in the number of children accessing them. The training of CHWs to facilitate the health clubs will reinforce and add to skills acquired during iCCM training and may also improve quality of care. VHCs will hopefully become valued forums, increasing the capacity of both the CHW and the village inhabitants to address health issues in their communities and ensure children are receiving timely, effective and appropriate treatment.

There is also a possibility that both interventions could lead to substantial engagement of community members with child health issues, over and beyond what could be reasonably attributed to their relationship with CHWs. In this scenario, improved care-seeking will not only be due to increased CHW motivation and retention, but also to enhanced community knowledge and action for child health. This process will be captured through the alternative mechanism outlined in Figure 4.

Overview of trial design

The inSCALE interventions are being evaluated through cluster randomised controlled trials (cRCTs) in Uganda and Mozambique. The clusters are sub-counties in Uganda and districts in Mozambique, and correspond to the lowest administrative units where CHW services are coordinated, with an average of 60 VHTs and 18 to 25 APEs per cluster, respectively. In Uganda, there are 95 clusters in the eight study districts. In order to be eligible for randomisation, a cluster had to have CHWs trained in iCCM by 31 January 2011 (3 months before starting the baseline survey). Clusters that had less than ten villages or that were participating in another Malaria Consortium mobile phone pilot study were excluded. Overall 41/95 clusters were eligible for randomisation. In Uganda, the trial consists of a three-arm cRCT with 13 clusters in each arm; two clusters were not  . Detailed process, cost and cost-effectiveness evaluations are being carried out in both countries. All data collection is expected to be completed in May 2015, and results will be available by the later part of 2015.

Randomisation

Restricted randomisation was performed to minimise the difference between the intervention and control arms for key indicators of the average proportion of children appropriately treated for fever, diarrhoea, and pneumonia (FDP); VHT motivation (mean); and the mean (log10) cost of treatment for children with FDP [19]. The randomisation of clusters to the three arms was restricted to those randomisation schemes where there were minimal between-arm differences for these five key parameters but still allowed adequate numbers of randomisation schemes, as too few schemes could elicit selection bias [20].

In Mozambique, appropriate treatment for childhood FDP could not be measured reliably at baseline due to a delay in the roll-out of the APE kits containing iCCM medicines. Attempts to seek care from an APE or a public facility were chosen as proxy markers for overall appropriate treatment, as these showed the strongest correlation with appropriate treatment. This was based on an assessment of the relationship between appropriate treatment with these and other candidate variables using parallel data from the Uganda baseline survey (where appropriate treatment was measured), and by assessment of the cluster averages and variances for several care seeking variables from the Mozambique dataset itself.

In Uganda, out of 500,000 random allocations of 39 clusters to three arms, 13,683 of the allocations fit all criteria. After applying a further filter to ensure that all districts had at least one sub-county in the mHealth intervention arm, and that at least the larger districts had a control arm, a final scheme was picked from the resulting sub-sample of 1,791 allocation schemes. In this scheme, all districts had at least one sub-county in the control arm and one in the mHealth intervention arm, and five out of eight districts also had at least one sub-county in the participatory community engagement arm ( Figure 5).

Within Mozambique, there were 924 possible unique ways to sort 12 districts into two groups. In total, 84 schemes from these potential 924 met the all restriction criteria. A scheme was picked at random from the final 84 schemes, and was thus the final chosen scheme for the Mozambique study ( Figure 6). In both sites, the selected schemes had no more than a 5% (proportions) or a 0.5 (scores/counts) difference between arms for any parameter. Sorting of clusters and random selection of schemes were carried out in Stata v12.1 (StataCorp, Texas USA).

Intervention areas

The inSCALE interventions as described above were implemented in the 26 sub-counties in Uganda (13 in the 'mHealth intervention arm' and 13 in the 'community engagement intervention arm') and in six districts in Mozambique. All CHWs and children living in these clusters were potential recipients of the CHW intervention, in addition to having access to routine care currently available from private and public health services.

Control areas

Children living in the control areas continued to benefit from the routine Ministry of Health iCCM package provided by the CHWs who were supported by the national and sub-national health services with funding from Malaria Consortium's CIDA-iCCM project.

Sample size

The sample size calculations, which are based on a cluster RCT with 13 and 6 clusters per arm in Uganda and Mozambique, respectively, are based on the main evaluation outcome: the percentage of children receiving appropriate treatment during illness episodes for each of pneumonia, diarrhoea and fever. This is also the outcome that requires the largest samples, as not all children surveyed will have had a recent illness episode. Inter-cluster coefficients of variation, baseline appropriate treatment rates, and prevalences of illness were calculated from data collected during the baseline surveys in each site and used in the sample size calculations (assuming a 5% significance level).

The calculation was based on the formula for comparison of proportions, adjusted for cluster effects stated in equation 7.7 of Hayes and Moulton [21], that is, where k is the between cluster coefficient of variation:
C ¼ 1 þ zα=2 þ zβ ð Þ 2ð ðπ0ð1 -π0Þ=m þ π1 1 -π1 ð Þ=m þ k2 π02 þ π12 ð Þ Þ = π0 -π1 ð Þ 2 Þ
The number of sick children required/cluster was determined by varying this number in the formula for the number of clusters until the number of clusters/arm equals 13 and 6, respectively [21]. Thus, sampling 155 children per cluster in Uganda would yield enough sick children to have 90% power to detect a 15% difference in the appropriate treatment of pneumonia and fever and a 20% difference in diarrhoea between arms. Sampling 390 children per cluster in Mozambique would yield enough sick children to have 90% power to detect 20% and 25% differences in appropriate treatment of fever and pneumonia respectively, and 80% power to detect a 25% difference in appropriate treatment of diarrhoea between arms.

The sample size formula and sampling scheme were also used to inform the design of the baseline surveys (which involved surveillance of households, CHWs and health facilities). At baseline, site-specific data was not available for k; therefore, a conservative estimate of 0.15 was used to inform the baseline sample size calculations in both sites. Prevalence values for FDP in the sites were based on estimates from recent Demographic and Health Surveys [22,23], Malaria Indicator Surveys [24,25], and in the case of Mozambique, an Integrated Health Systems and Child Friendly District Survey by UNICEF [26]. In Uganda, 3,900 households with children under the age of 5 years were sampled at baseline; of the 6,501 children in these households, 47% were reported as having had symptoms of fever, 11% with diarrhoea and 24% with pneumonia in the two weeks prior to the survey and were thus eligible for the full household survey questionnaire on treatment, care seeking, and health practices. In addition, 360 VHTs and the 79 health facilities supporting these VHTs were also sampled as part of the baseline. In Mozambique, 2,970 households with children under the age of 5 years were sampled; of the 4,422 children in these households, 29.2% were reported as having had symptoms of fever, 5.2% with diarrhoea and 10.7% with pneumonia in the two weeks prior to the survey. Data were also collected from 256 iCCM-trained APEs and the 80 health facilities to which they reported.

Impact evaluation

The primary outcome is the change in carer reported appropriate treatment for fever, diarrhoea and pneumonia in children in households receiving the intervention(s) compared to those in areas with routine iCCM ('control' households). A proportion of children will have had two or more conditions simultaneously. We intend to group all conditions and assess the 'use of appropriate drugs for this episode of illness for all illness episodes by intervention arm. Additionally, the effect of the interventions on appropriate treatment for each of the three conditions individually will be established as a way of understanding the pathways through which the interventions worked.

The main comparison for effect will be obtained from the measured difference between interventions and control arms at end-line. Baseline characteristics were used to perform restricted randomisation to minimise the difference between the intervention and control arms on key indicators.

Secondary outcomes include a number of performance, retention, motivation, access and cost-effectiveness outcomes:

1. Mean motivation score (motivation outcome) 2. % CHWs staying in post after 1 year of implementation (retention outcome) Figure 6 Intervention and control arm clusters in Mozambique.

3. % monthly reports complete and submitted on time (performance outcome) 4. % CHWs with medicine stock-out <1 week each quarter (performance outcome) 5. % caregivers with sick child in the last 2 weeks who accessed iCCM treatment (coverage outcome) 6. % caregivers willing to use CHWs (access outcome) 7. % of start-up costs compared to recurring costs of the intervention packages (cost outcome) 8. Cost per CHW trained (cost outcome) 9. % of start-up costs compared to recurring costs of the intervention packages (cost outcome) 10.Cost per child who receives appropriate treatment for each of diarrhoea, pneumonia and fever (costeffectiveness outcome) 11.Cost per episode of illness treated correctly (costeffectiveness outcome) 12.Cost-per life year gained (LYG) and DALY averted (cost-effectiveness outcome)

Household, CHWs and health facility surveys

The primary outcome in both sites will be collected using an end-line household survey administered to the child's primary carer. Data will be collected on illness prevalence, proportion of children who have received appropriate treatment and on household socio-economic status. Secondary outcomes, such as CHW performance, will be assessed in a sample of CHWs using a structured questionnaire with pre-tested case scenarios and knowledge questions administered by a trained research assistant. A motivation tool has been developed and validated to establish a composite index of CHW motivation. A health facility questionnaire will be used to collect data from health facility staff/supervisors on CHW supervision, support and health system costs. The quantitative survey questionnaires will be adopted from the tools used for the baseline survey. These were developed in an iterative process: i) with the team of researchers in London and Kampala/Mozambique, ii) with potential field supervisors, and finally, iii) with trainees of the data collector training. Both the field supervisors and data collectors will visit local homes under supervision of a researcher to practice and test the questions and make revisions. All questionnaires will be written in English and translated into the local languages (Runyoro, Luo and Luganda for Uganda and Portuguese for Mozambique), and the accuracy of the translation will be checked by back translation into English. Key illness terminologies and other sensitive expressions will be translated into the native languages during the training and provided to the research assistants on a hard copy which guided them during the interview. Field editing will be done for consistency and completeness and after correction, data will be double-entered by trained data entry clerks into a SQL data management system capable of consistency, range, inter-database checks and audit trails.

Intention-to-treat analyses

All analyses will be intention-to-treat, and will account for the cluster-randomised design. Initial crude analyses will be based on summary measures for each outcome in each cluster and t-tests will be used to compare each intervention arm with the control arm. Multivariate random-effects models, which allow for cluster-specific constant amounts and work well for normally distributed outcomes and Poisson outcomes, will be used to assess the effect of the intervention on retention, motivation and coverage of appropriate treatment; adjustments will be made to account for clustering and any imbalances found between groups at baseline.. Random effects logistic regression models can be computationally demanding, and can fail to converge; if this is the case, generalized estimating equations (GEE) will be used instead if the random effects are found to be unreliable. The GEE approach modifies both parameter estimates and standard errors to allow for clustering and are appropriate where the clusters themselves are of no intrinsic interest. According to the analysis approach proposed [27], random effects models, whatever the outcome, will be used, only reverting to GEE where these do not converge.

Record review

During the project cycle, CHW record books will be reviewed to estimate the proportion of children classified with diarrhoea, malaria and pneumonia; the mRDT positivity rate, and frequency and outcome of referral. Drug stock cards will be reviewed to estimate drug stock-outs. Supervisor log books will be reviewed for frequency and content of supervision visits, and reports from district and health facility staff will be reviewed for CHW retention. Health facility records will be reviewed for patient load for health system costs. To estimate costs of CHW activities, the results of the process evaluation interviews described below will be combined with data collection from financial and activity reporting.

Process evaluation and intervention monitoring

Process evaluation (PE), which started approximately 10 months after implementation of the interventions (January to February 2014 in Uganda and April to May 2014 in Mozambique), aims to identify lessons learned, successes and challenges and to crystallise best practices using key informant interviews with CHWs and their health facility supervisors. In Uganda, 24 VHTs and eight supervisors in the mHealth intervention, and 24 VHTs and eight supervisors in the participatory community engagement intervention in the districts of Buliisa, Hoima, Kyegegwa and Masindi were interviewed using predesigned topic guides. In Mozambique, 33 semistructured key informant interviews with 24 APEs and nine supervisors were conducted in the districts of Inharrime, Morrumbene and Inhassoro of Inhambane Province. The qualitative most significant change methodology [28] was applied during the process evaluation as a way of monitoring changes observed as a result of the iCCM programme, and the interventions implemented, to document success stories from community members, CHWs and health workers. Specifically the process evaluation explored:

1. whether the interventions were delivered as designed and taken up and utilised as intended. 2. which the context-based factors had an impact on aspects of work motivation and satisfaction and the existing approaches to measuring motivation, instruments that could be adapted to the specific case of CHWs, best practices and new methods. 3. whether the interventions influenced motivation, retention, performance and/or other factors. 4. whether increased motivation impacted on performance and retention, and whether this lead to children receiving more appropriate treatment. 5. how the programme was implemented, how the programme operated, the services it delivered, and the functions it carried out.

Cost and cost-effectiveness evaluation

The cost and cost-effectiveness analyses will take a societal perspective, assessing the economic impact for all parties affected by the interventions.

Intervention costs

The costs of the intervention packages will be estimated both from a financial and economic perspective. Costs will include direct project expenditure incurred by the implementing partner, the costs borne by the MoH at the national and local level during the implementation phase, the value of the work time contributed by the volunteering CHWs in Uganda, and any non-reimbursed out-of-pocket costs incurred by the CHWs in relation to their work. In order to allow generalisability of the findings to other contexts, cost will be categorised by principal inputs and activities, with quantities and values collected separately as far as possible.

Household and community costs

Direct (out-of-pocket) costs as well as indirect costs (value of lost production) for households in relation to care and treatment-seeking will be measured at the end of the intervention trials across trial arms. Direct costs will include transport costs, out of pocket payments to service providers, and any money spent on items complementary to treatment. Indirect costs will focus on the time spent caring for the sick child and seeking care. Costs will be broken down by socioeconomic status.

Health facility costs

It is likely that resource use in local health facilities will be affected by the introduction of a new level of healthcare worker; due to changes in the numbers of children seeking care at the health centres, of severe cases seen due to early treatment in the community, and of cases referred. The unit costs of treating the illnesses included in the iCCM package at different facility levels will be estimated, including staff time, drugs, diagnostics and other consumables and the use of equipment and facilities. The costs per episode of care in different facilities will be brought together with health seeking behaviour information from the household survey to assess the net difference in resources use.

Cost-effectiveness analysis

The cost-effectiveness analysis will assess the incremental cost and outcomes of the two intervention and control arms in Uganda, and between the intervention arm and control arm in Mozambique. Incremental costs will be derived from the cost analyses above, combining unit costs with data on service provision/ utilisation from the trial surveys in each arm. Outcome measures will be based in the principal outcome measure of the intervention trial, the incremental difference in the number of sick children appropriately treated during the trial period. This outcome will then be converted into lifeyears gained (LYGs) and disability-adjusted life years (DALYs) averted using standard methods.

Informed consent

Individual informed consent is being obtained from all respondents throughout the projects' data collection phases. Information sheets containing information about the study are provided (or in the case of phone interviews, e-mailed in advance) in local languages (Luganda, Luo, Runyoro/Rutoro or Portuguese (in Mozambique)). Respondents are given time to read the information sheet, and key points are summarised verbally and questions answered. Agreement to participate is indicated by signature or for phone interviews through verbal consent. The individual's right to refuse consent or to stop the interview at any time after consent has been given is preserved; individuals are not required to provide explanation for such decisions. Stakeholders are asked whether they consent to the inclusion of verbatim quotes in reports and for their name to be included in the list of interviewed persons. Confidentiality of all data collected is maintained at all times, except where it relates to routine monitoring of performance of CHWs.

All CHWs are identified by a unique ID number. No hazardous substances or medical procedures are used in this study.

Trial monitoring

The project has a nine-person Technical Advisory Group (TAG) to facilitate dissemination and uptake of any findings within Uganda and Mozambique, as well as to provide technical support. Members include influential experts and national stakeholder representatives, WHO and UNICEF representatives and advisers with expertise in public health and community-based research, iCCM, health policy analysis, and international health policy development. It is also attended by the principle investigators and members of the inSCALE study group. The Data Monitoring and Ethics Committee (DMEC) has five external members with expertise in health economy, child health, epidemiology, iCCM and randomised controlled trials, chosen to provide guidance to the trial in respect of study design, data analysis and study safety. The DMEC is responsible for reviewing and evaluating data with respect to the effectiveness of the supervision/ motivation interventions. Both committees met at the start of the project to examine trial conduct and progress and to advise the inSCALE technical team, and will meet again in the final year of the project. The DMEC is not carrying out any interim analyses, as the inSCALE intervention does not involve any drugs or medical procedures and as the evaluation is based on outcomes occurring over a period of just one year.

Ethical approval

The trial protocol was approved by Makerere University Institutional Review Board in Uganda

Discussion

For child health interventions like iCCM to have impact when implemented at scale, it is crucial that cost-effective strategies that improve performance and motivation of CHWs are identified and evaluated in a 'real-life' health systems context. The current, multi-country, cluster randomised controlled trial aims to meet this need by assessing two innovative approaches using mHealth and participatory community engagement to increase CHW support, supervision and motivation in Uganda and Mozambique. We hypothesise that both interventions can lead to improvements in coverage of appropriate treatment for children with diarrhoea, pneumonia and malaria (primary outcome) and CHW retention rate, motivation and performance (secondary outcomes).

This study's strength is that, as well as drawing on theoretical perspectives and evidence of best practices, it applies a user-centred design to the innovations based on extensive formative research. In so doing the potential for innovations to be acceptable, feasible and scalable is increased by prioritising evidence generated from the operating context through exploration of needs as stated by intended users (CHWs and national stakeholders). The multidisciplinary research team is an additional strength that enables the application of robust methods for both qualitative and quantitative evaluation of the innovations, as well as high quality implementation. A further strength is the development of an economic model, parameterised for each country using data from iCCM implementation, the inSCALE intervention trial, and secondary data sources, to support policy decisions in the study countries.

The lack of an accurate measurement to estimate coverage of appropriate treatment of sick children is one weakness of the study. Coverage of maternal, neonatal and child health (MNCH) interventions are typically monitored through household surveys, which estimate treatment rates based on 2-week recall of illness symptoms by caregivers. However, these survey tools identify children with reported symptoms, and for illnesses like pneumonia where the true prevalence in community settings is low, this methodology does not provide an accurate denominator of cases for monitoring treatment rates [29]. Issues with recall of diagnostic results and treatments provided have also been documented [30,31]. While we are aware of these limitations, we attempted to increase the validity of our estimates by increasing the recall period to 4 weeks to increase the number of recent cases detected in the survey, and including an expanded list of symptoms indicative of pneumonia in order to increase the specificity of the pneumonia case definition. We also used drug charts to increase the validity of reported treatments. Another possible weakness of the study is the lack of control over contextual factors and other changes in the health system in which the innovations are implemented. While this was anticipated and attempts are made to access district and provincial operational plans and internal project reports, lack of data on factors that may mediate the effect of the interventions [32], such as drug stock-outs, health staff rotation, and concurrent programme implementation by other partners, leaves us unable to plot these influences with absolute confidence in their accuracy.

If evidence yields support for cost-effective innovations that can generate a highly motivated and well-performing workforce of CHWs, there is potential for sustainable coverage of high quality iCCM at scale and significant reductions in child morbidity and mortality.

Public engagement and dissemination

The inSCALE study is designed to directly influence policy and practice, especially for government-led scale-up of iCCM in the two countries. Implementation has been carried out in close consultation with policy makers. In addition, study materials and early results will be made accessible on the project website (www.malariaconsortium. org/inscale) well before the final evaluation is made. There has been close documentation of all implementation steps, challenges, innovations and experiences. In addition, the study employs a rigorous economic modelling for iCCM implementation in the two countries, in order to provide directly support to the iCCM policy agenda.

Trial findings will be shared promptly with the local district health teams and local dissemination meetings with the study populations will be held. Policy briefs will be prepared and circulated nationally and internationally to relevant policy and donor organisations, and national workshop held to discuss the findings, lessons learnt concerning implementation and policy implications. Trial findings will also be disseminated in national and international scientific meetings, and peer-reviewed publications will be produced on the impact of the intervention on appropriate treatment of sick children, cost and cost-effectiveness of the interventions, impact on CHW motivation and performance, and process outcomes and lessons learned concerning the interventions evaluated.

Selected stories generated for the most significant change methodology will be disseminated to policy makers, implementers and the general public using audio-visual and written messages.

Trial status

The implementation of the interventions in Uganda is completed and end-line survey data has been collected. Mozambique is continuing implementation until April 2015. Final evaluation results will be available April 2015 for Uganda and September 2015 for Mozambique.

Figure 1
1inSCALE implementation area in Midwestern Uganda.

Figure 2
2inSCALE implementation area in Mozambique. Källander et al. Trials (2015) 16:157

Figure 3
3Details of inSCALE's two integrated intervention packages.

Figure 4
4inSCALE conceptual framework.

randomised and were designated as spare/back-up clusters in case of the need to drop one of the selected 39. In Mozambique, there are 12 clusters, allowing a twoarm cRCT.The trial planning started in April 2011, and the inSCALE interventions were developed and fully implemented in the intervention areas by the end of 2012 in Uganda and by the end of June 2013 in Mozambique. Impact data on the main outcomes was collected in an endline household survey after 12 months of implementation in Uganda (March-May 2014) and will be collected after 18 months of implementation inMozambique (February  2015)

Figure 5
5Interventions and control arm clusters in Uganda.

Table 1
1Content of the participatory community engagement intervention in UgandaKey principlesVillage health clubs (VHCs) aiming to improve child health through a community led-forum with the village health team (VHT) as the main focus point.Intervention components
Uganda

Based on five main pillars:

Table 2
2Content of the mHealth intervention in Mozambique and UgandaIntervention
components
('innovations')

Mozambique
Uganda

Mobile phone
Samsung Galaxy Y (Android smart phone)
Nokia C2-00 (Java enabled dual SIM card feature phone)

Accessories
Solar lamp (Sun King Pro) with multiple phone charging pins
Solar lamp (Sun King Pro) with multiple phone charging
pins

, the Uganda National Council of Science and Technology (ref. HS 958), the Comité Nacional de Bioética para a Saúde in Mozambique (ref. 331/CNBS/12) and London School of Hygiene & Tropical Medicine Ethics Committee in the UK (ref. 5762). In addition, approval was obtained from the district authorities and local leaders in the communities where the study is being conducted. The study has been registered as a randomised controlled trial with http://www.clinicaltrials.gov (identifier NCT01972321).
Competing interestsThe authors declare that they have no competing interests.KK, DS, ZH, JT, BK and SM conceived the study; KK, DS, ZH,Authors' contributions
Global, regional, and national causes of child mortality in 2000-13, with projections to inform post-2015 priorities: an updated systematic analysis. L Liu, S Oza, D Hogan, J Perin, I Rudan, J E Lawn, Lancet. 385Liu L, Oza S, Hogan D, Perin J, Rudan I, Lawn JE, et al. Global, regional, and national causes of child mortality in 2000-13, with projections to inform post-2015 priorities: an updated systematic analysis. Lancet. 2015;385:430-40.

How many child deaths can we prevent this year?. G Jones, R W Steketee, R E Black, Z A Bhutta, S S Morris, Lancet. 362Jones G, Steketee RW, Black RE, Bhutta ZA, Morris SS. How many child deaths can we prevent this year? Lancet. 2003;362:65-71.

World Health Organization/ United Nations Children's Fund joint statement on integrated community case management: an equity-focused strategy to improve access to essential treatment services for children. M Young, C Wolfheim, D R Marsh, D Hammamy, Am J Trop Med Hyg. 875SupplYoung M, Wolfheim C, Marsh DR, Hammamy D. World Health Organization/ United Nations Children's Fund joint statement on integrated community case management: an equity-focused strategy to improve access to essential treatment services for children. Am J Trop Med Hyg. 2012;87(5 Suppl):6-10.

Access, acceptability and utilization of community health workers using diagnostics for case management of fever in Ugandan children: a crosssectional study. D Mukanga, J K Tibenderana, S Peterson, G W Pariyo, J Kiguli, P Waiswa, Malar J. 11121Mukanga D, Tibenderana JK, Peterson S, Pariyo GW, Kiguli J, Waiswa P, et al. Access, acceptability and utilization of community health workers using diagnostics for case management of fever in Ugandan children: a cross- sectional study. Malar J. 2012;11:121.

World Development Report: Investing in Health. W Bank, Oxford University PressOxfordBank W. World Development Report: Investing in Health. Oxford: Oxford University Press; 1993.

Community health workers for universal health-care coverage: from fragmentation to synergy. K Tulenko, S Møgedal, M M Afzal, D Frymus, A Oshin, M Pate, Bull World Health Organ. 91Tulenko K, Møgedal S, Afzal MM, Frymus D, Oshin A, Pate M, et al. Community health workers for universal health-care coverage: from fragmentation to synergy. Bull World Health Organ. 2013;91:847-52.

Community-based health workers: head start or false start towards health for all?. P A Berman, D R Gwatkin, S E Burger, Soc Sci Med. 25Berman PA, Gwatkin DR, Burger SE. Community-based health workers: head start or false start towards health for all? Soc Sci Med. 1987;25:443-59.

Review of the utilisation and effectiveness of community-based health workers in Africa, A joint learning initiative: Human resources for health and development. Cape Town: University of the Western Cape. U Lehmann, I Friedman, D Sanders, Lehmann U, Friedman I, Sanders D. Review of the utilisation and effectiveness of community-based health workers in Africa, A joint learning initiative: Human resources for health and development. Cape Town: University of the Western Cape; 2004.

Basic Support for Institutionalizing Child Survival Project (BASICS II) for the United States Agency for International Development. K Bhattacharyya, P Winch, K Leban, M Tien, Arlington, VirginiaCommunity Health Worker Incentives and Disincentives: How They Affect Motivation, Retention, and SustainabilityBhattacharyya K, Winch P, LeBan K, Tien M. Community Health Worker Incentives and Disincentives: How They Affect Motivation, Retention, and Sustainability. Arlington, Virginia: Basic Support for Institutionalizing Child Survival Project (BASICS II) for the United States Agency for International Development; 2001.

Integrated Community Case Management of Childhood Malaria, Pneumonia and Diarrhoea: Implementation Guidelines. Kampala: MOH/WHO/UNICEF. Moh Uganda, Uganda MOH. Integrated Community Case Management of Childhood Malaria, Pneumonia and Diarrhoea: Implementation Guidelines. Kampala: MOH/WHO/UNICEF; 2010.

An integrated approach of community health worker support for HIV/AIDS and TB care in Angonia district. S Simon, K Chu, M Frieden, B Candrinho, N Ford, H Schneider, Mozambique. BMC Int Health Hum Rights. 913Simon S, Chu K, Frieden M, Candrinho B, Ford N, Schneider H, et al. An integrated approach of community health worker support for HIV/AIDS and TB care in Angonia district, Mozambique. BMC Int Health Hum Rights. 2009;9:13.

Programa de Revitalização dos Agentes Polivalentes Elementares. Maputo: MISAU. Moh Mozambique, Mozambique MOH. Programa de Revitalização dos Agentes Polivalentes Elementares. Maputo: MISAU; 2010.

Managing acute febrile illness in the community. K Kallander, E Kertho, C Sinyangwe, B Ntebeka, C Counihan, Malaria ConsortiumWashington DC, USKallander K, Kertho E, Sinyangwe C, Ntebeka B, Counihan C. Managing acute febrile illness in the community. American Society of Tropical Medicine and Hygiene. Washington DC, US: Malaria Consortium; 2013.

Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low-and middle-income countries: a review. K Kallander, J K Tibenderana, O J Akpogheneta, D L Strachan, Z Hill, A H Ten Asbroek, J Med Internet Res. 1517Kallander K, Tibenderana JK, Akpogheneta OJ, Strachan DL, Hill Z, ten Asbroek AH, et al. Mobile health (mHealth) approaches and lessons for increased performance and retention of community health workers in low-and middle-income countries: a review. J Med Internet Res. 2013;15:e17.

Developing intervention strategies to improve community health worker motivation and performance. Kampala: Malaria Consortium. T Frank, K Källander, Frank T, Källander K. Developing intervention strategies to improve community health worker motivation and performance. Kampala: Malaria Consortium; 2012. http://www.malariaconsortium.org/inscale/downloads/ 1139-lp-inscale-approaches_ug-mo_oct-2012-lr.pdf. Accessed 20 Feb 2015.

Interventions to Improve Motivation and Retention of Community Health Workers Delivering Integrated Community Case Management (iCCM): Stakeholder Perceptions and Priorities. D L Strachan, K Kallander, Ten Asbroek, A H Kirkwood, B Meek, S R Benton, L , Am J Trop Med Hyg. 875SupplStrachan DL, Kallander K, Ten Asbroek AH, Kirkwood B, Meek SR, Benton L, et al. Interventions to Improve Motivation and Retention of Community Health Workers Delivering Integrated Community Case Management (iCCM): Stakeholder Perceptions and Priorities. Am J Trop Med Hyg. 2012;87(5 Suppl):111-9.

Malaria Consoritum. Reports and publications. Malaria Consoritum. Reports and publications. 2014. http://www.webcitation. org/6T0F4wpjn. Accessed 20 Feb 2015.

Qualitative Methodology for International Public Health. L Dahlgren, M Emmelin, A Winkvist, Epidemiology and Public Health SciencesUmeå; Umeå UniversityDahlgren L, Emmelin M, Winkvist A. Qualitative Methodology for International Public Health. Umeå: Epidemiology and Public Health Sciences, Umeå University; 2007.

Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. N M Ivers, I J Halperin, J Barnsley, J M Grimshaw, B R Shah, K Tu, Trials. 13120Ivers NM, Halperin IJ, Barnsley J, Grimshaw JM, Shah BR, Tu K, et al. Allocation techniques for balance at baseline in cluster randomized trials: a methodological review. Trials. 2012;13:120.

Covariate-based constrained randomization of group-randomized trials. L H Moulton, Clin Trials. 1Moulton LH. Covariate-based constrained randomization of group-randomized trials. Clin Trials. 2004;1:297-305.

Cluster Randomised Trials (Chapman & Hall/CRC Interdisciplinary Statistics). USA: Chapman and Hall/CRC. R J Hayes, L H Moulton, Hayes RJ, Moulton LH. Cluster Randomised Trials (Chapman & Hall/CRC Interdisciplinary Statistics). USA: Chapman and Hall/CRC; 2008.

Instituto Nacional de Estatística (INE), ICF International (ICFI). Saude Ministerio Da, Calverton, Maryland, USA: MISAU, INE e ICFIMoçambique Inquérito Demográfico e de SaúdeMinisterio da Saude (MISAU), Instituto Nacional de Estatística (INE), ICF International (ICFI). Moçambique Inquérito Demográfico e de Saúde. Calverton, Maryland, USA: MISAU, INE e ICFI; 2011.

ICF International Inc. Uganda Demographic and Health Survey. ICF International IncKampala, Uganda; Calverton, MarylandUganda Bureau of Statistics (UBOSUganda Bureau of Statistics (UBOS), ICF International Inc. Uganda Demographic and Health Survey 2011. Kampala, Uganda: UBOS and Calverton, Maryland: ICF International Inc; 2012.

World Health Organisation, US Agency for International Development, Centres of Disease Control, Satellife Project, Department for International Development et al. National Malaria Indicator Survey, Mozambique (MIS-2007). Saude Ministerio Da, Maputo, Mozambique: National Malaria Control ProgrammeMinisterio da Saude (MISAU), World Health Organisation, US Agency for International Development, Centres of Disease Control, Satellife Project, Department for International Development et al. National Malaria Indicator Survey, Mozambique (MIS-2007). Maputo, Mozambique: National Malaria Control Programme; 2007.

Uganda Malaria Surveillance Project, National Malaria Control Programme, ICF Macro. Uganda Malaria Indicator Survey. Kampala, UgandaUganda Bureau of Statistics (UBOSMinistry of HealthUganda Bureau of Statistics (UBOS). Uganda Malaria Surveillance Project, National Malaria Control Programme, ICF Macro. Uganda Malaria Indicator Survey 2009. Kampala, Uganda: Ministry of Health; 2010.

Survey: Integrated Health Systems and Child-Friendly Districts Maputo. UNICEF: Mozambique. Unicef Mozambique, UNICEF Mozambique. Survey: Integrated Health Systems and Child-Friendly Districts Maputo. UNICEF: Mozambique; 2008.

. B R Kirkwood, Jac Sterne, Essential Medical Statistics. Massachusetts: Blackwell Science Inc. Kirkwood BR, Sterne JAC. Essential Medical Statistics. Massachusetts: Blackwell Science Inc.; 2003.

Story-Based Evaluation Tool: The Most Significant Change Technique. J Dart, R Davies, Dialogical, Am J Eval. 24Dart J, Davies R. A Dialogical, Story-Based Evaluation Tool: The Most Significant Change Technique. Am J Eval. 2003;24:137-55.

Measuring coverage in MNCH: challenges in monitoring the proportion of young children with pneumonia who receive antibiotic treatment. H Campbell, S El Arifeen, T Hazir, J O'kelly, Bryce J Rudan, I , 10.1371/journal.pmed.1001421PLoS Med. 101001421Campbell H, El Arifeen S, Hazir T, O'Kelly J, Bryce J, Rudan I, et al. Measuring coverage in MNCH: challenges in monitoring the proportion of young children with pneumonia who receive antibiotic treatment. PLoS Med. 2013;10:e1001421. doi: 10.1371/journal.pmed.1001421.

Measuring coverage in MNCH: accuracy of measuring diagnosis and treatment of childhood malaria from household surveys in Zambia. T P Eisele, K Silumbe, J Yukich, B Hamainza, J Keating, A Bennett, PLoS Med. 101001417Eisele TP, Silumbe K, Yukich J, Hamainza B, Keating J, Bennett A, et al. Measuring coverage in MNCH: accuracy of measuring diagnosis and treatment of childhood malaria from household surveys in Zambia. PLoS Med. 2013;10:e1001417.

Measuring coverage in MNCH: current indicators for measuring coverage of diarrhea treatment interventions and opportunities for improvement. Fischer Walker, C L Fontaine, O Black, R E , PLoS Med. 101001385Fischer Walker CL, Fontaine O, Black RE. Measuring coverage in MNCH: current indicators for measuring coverage of diarrhea treatment interventions and opportunities for improvement. PLoS Med. 2013;10:e1001385.

How does context influence performance of communityhealth workers in low-and middle-income countries? Evidence fromthe literature. M C Kok, S S Kane, O Tulloch, H Ormel, S Theobald, M Dieleman, Health Res Pol Syst. 1313Kok MC, Kane SS, Tulloch O, Ormel H, Theobald S, Dieleman M, et al. How does context influence performance of communityhealth workers in low-and middle-income countries? Evidence fromthe literature. Health Res Pol Syst. 2015;13:13.


Ambient Sulphur Dioxide and Female ED Visits for Migraine

Mieczysław Szyszkowicz firstname.lastname@example.com
Population Studies Division
269 Laurier Avenue, Room 3-030K1A 0K9OttawaHealth Canada, ONCanada

Eugeniusz Porada
Population Studies Division
269 Laurier Avenue, Room 3-030K1A 0K9OttawaHealth Canada, ONCanada

Ambient Sulphur Dioxide and Female ED Visits for Migraine

International Scholarly Research Network ISRN Neurology
201210.5402/2012/279051Received 3 December 2011; Accepted 9 January 2012 Academic Editor: A. ArboixResearch Article Correspondence should be addressed to Mieczysław Szyszkowicz,
Ambient sulphur dioxide (SO 2 ) concentrations may affect the number of female emergency department (ED) visits for migraine. ED visits diagnosed as migraine among females in two cities in Canada, Toronto (N = 704) and Ottawa (N = 3, 358), were analyzed. In the study case-crossover design was used. Conditional logistic regression was realized to estimate odds ratios (ORs) and their 95% confidence intervals (CIs) relative to an increase in an interquartile range (IQR, in Toronto IQR = 2.9 ppb, in Ottawa IQR = 3.9 ppb) of sulphur dioxide. In the constructed conditional logistic regression models, temperature and relative humidity were adjusted in the form of natural splines. In Toronto positive and statistically significant associations of sulphur dioxide with migraine ED visits were obtained: all ages, OR = 1.04 (95% CI: 1.00, 1.08); age group[15,50], OR = 1.05 (95% CI: 1.01, 1.09). In Ottawa positive correlations were observed: all ages, OR = 1.05 (95% CI: 0.97, 1.13); age group[15,50], OR = 1.06 (95% CI: 0.97, 1.15). The results suggest that female migraine may be affected by ambient sulphur dioxide.

Introduction

The aim of this paper was to investigate potential correlations between ambient sulphur dioxide and female emergency department (ED) visits for migraine in two Canadian cities, Toronto and Ottawa, two locations with different levels of the pollutant concentrations. Case-crossover (CC) design was used [1]. Conditional logistic regression was applied to realize calculations (PHREG procedure, SAS, v. 9.1) related to the CC technique. Daily events, diagnosed on ED as migraine, were used to represent health outcomes. The considered sulphur dioxide concentrations, temperature (dry bulb), and relative humidity were expressed as daily mean values.

This study was inspired by a few recent publications [2][3][4][5], related to investigation ambient air pollution exposure and ED visits for migraine and headache. The published results suggested the associations of ambient sulphur dioxide concentrations with female ED visits for migraine.

Materials and Methods

Study Population.

Population considered in this study were patients served by two different hospitals in Canada: in Toronto and Ottawa. In Toronto we considered the period between April 1, 1999 andMarch 31, 2002. In this time period we had 898 ED visits diagnosed as migraine, where 704 cases (78%) were females. In Ottawa we considered the period between April 1, 1992 and December 31, 2000, and we identified 4,568 migraine cases, 74% among them (3,358 cases) were female patients. We considered patients of all ages. We applied the ICD-9 codes [6] to identify the cases classified as migraine and we used 346.X rubric in both cities. In our calculations, we considered all patients. The presented graphical figures were produced for the patients of age in the interval [0, 85] years. Thus in this situation we put an age restriction.

Statistical Analysis.

The CC technique is an adaptation of the case-control study [1]. By definition of the case-crossover methodology, the cases acted as their own controls on a set of predefined control days proximate to the time they became cases. A time-stratified approach to determine controls was widely accepted and adopted as it has been shown to produce unbiased conditional logistic regression estimates [7]. In the design, the controls are matched to case periods by day of week for the case period (day), and the control periods are determined as other days in the same month and year (controls ± 7 days). This strategy was used here, thus 3 or 4 controls are present for each case. For example, for the case on Sunday, January 1, 2012, other Sundays in January 2012 will be control periods (January 8, 15, 22, and 29). In this situation, we have 4 controls.

In addition, to analyse less stable ED data in Ottawa (i.e., more cases but lower levels of ambient sulphur dioxide than in Toronto), two other strategies to define control periods for the CC method were used [8,9]. One approach defines control days for a particular case to be every third day within the same month and year that ED visit occurred (control ± 3 days). The second proposed technique uses all days in the same month but 6 days around the case day (all days in a month without 13 days). In this situation, we may have 18, 17, 16, or 15 (February with 28 days) control days (control [−6, 6]) for each case-day. For these sets of time control periods day of week was included in the conditional logistic regression model as an indicator variable.

The generated results were presented as the odds ratios (ORs) and their 95% confidence intervals (95% CIs). The calculated ORs were reported for an increase in the concentration represented as an interquartile range (IQR = Q3-Q1 = 75th-25th values of percentiles, IQR = 2.9 and 3.9 ppb in Toronto and Ottawa, resp.) of the sulphur dioxide level. Temperature and relative humidity were used in form of natural splines with 3 degrees of freedom. All components of the models, sulphur dioxide, and meteorological factors in the models were represented by values on the same day as the visit day. Here we only analyzed the same day exposure.

In addition, the following calculations were executed. The sequence of 57 overlapping age groups, each of length 30 years, was defined in the following way: the first age group was [0, 29], second [1,30], and each next was created from the current one by shifting it by one year. In this sequence, the last element was defined to be the age group [56,85]. The length of the age groups (here 30 years) was chosen rather arbitrarily but in relation to the number of ED visits. For the elements of this sequence, the CC analysis was performed separately. Thus the first OR corresponds to the age group [0, 29], second [1,30], and so on. The calculations were executed for female patients only.

Air Pollution Data.

Hourly air pollution data were obtained from fixed monitoring stations in the considered cities. These data were supplied by Environment Canada. For each air pollution variable, we have 24 measurements recorded at hourly intervals. The daily shared exposures of the population were expressed as mean values among stations in city (Toronto: 4.6, 2.7, 4.0, and 2.9 ppb; Ottawa: 3.9, 3.1, 3.5, and 3.9 ppb; mean, standard deviation, median, and IQR, resp.)

Results and Discussion

In Toronto positive and statistically significant associations of sulphur dioxide with migraine were obtained: all ages, OR = 1.04 (95% CI: 1.00, 1.08); age group [15,50], OR = 1.05 (95% CI: 1.01, 1.09). In Ottawa positive associations (no significant) were observed: all ages, OR = 1.05 (95% CI: 0.97, 1.13); age group [15,50], OR = 1.06 (95% CI: 0.97, 1.15). Our hypothesis is that ambient sulphur dioxide is associated with ED visits for migraine. The above reported results were obtained using the time-stratified (classical, controls ± 7 days) case-crossover technique. It is possible that by increasing the number of control periods, we may improve our estimations [8].

The main results are organized in the form of three figures (Figure 1 for Toronto, Figures 2 and 3 for Ottawa). Two figures, Figures 1 and 2, are composed of two different parts: (a) related to air pollution and (b) related to odds ratios. The top part shows the sulphur dioxide characteristics for controls and cases (the estimated median (Q2), Q1, and Q3) and also indicates the number of cases (divided by 76 and 355, in Toronto and Ottawa, resp.). The numbers of cases were scaled to have values on the same graph with sulphur dioxide characteristics. The bottom part illustrates the values of ORs and the corresponding 95% CIs. The results are shown (and were calculated) for each age group separately.

In Toronto, for many age groups the results were positive and statistically significant (the lower 95% CI limits greater than one, Figure 1). The positive correlations were obtained between the pollutant and migraine in Ottawa (Figure 2). Also should be noted that level of sulphur dioxide (expressed as a median value) was higher in Toronto than that in Ottawa. Figure 3 shows the results for Ottawa for two different variants of the CC design implementations. For example, the following results were obtained in Ottawa for the CC method with control periods as all days in a month without  In this study, statistically significant (in Toronto) shortterm effects were observed for ambient sulphur dioxide exposures on daily ED visits for migraine for females. Ambient air pollutant affects patients of different age ranges. The results suggest that above a specific level of sulphur dioxide concentrations or acute air pollution events (large Q3) might result in elevated number of ED visits for this health condition. The shape of the relations by age group (Figures 1 and 2) in Toronto and Ottawa is different. We identified only 704 cases in Toronto versus 3,358 in Ottawa, but Toronto has higher level of ambient sulphur dioxide than Ottawa. Figure 1 may indicate that in Toronto, specific age group is affected by exposure. In Ottawa (Figure 2), the results may suggest that more biological aspects are involved and dominate; the associations are stronger for the age groups where we observe pick of the frequencies of ED visits.

A few published studies have examined the effect of air pollution on migraine. In previous publication related to ED in Ottawa [10], the authors used the same data and did not find correlations between air pollution and ED visits for migraine (see Figure 2). Probably for the same reason, lower level of exposures and consequently weaker contrast between case and control periods, the authors of the work [11] in USA did not note significant correlations in their study. In both mentioned publications, the results for sulphur dioxide were only positive, and in these two papers, the classical time-stratified CC technique was realized (i.e., 3 or 4 control days). In contrast, the study performed on hospital admission for migraine in Chile reported statistically significant associations between air pollution exposures and hospitalization for headache [5]. The authors estimated relative risk for migraine as 1.10 (95% CI: 1.04, 1.17) for one IQR = 6.20 ppb increase in sulfur dioxide. The animal study on Saharan dust has revealed an unknown environmental factor as a possible trigger for headache [12]. In the epidemiological study performed at the New York University, headaches were observed among persons exposed to the World Trade Center dust clouds after 9/11 [13]. Environmental factors including noise, light/sun, heat, cold, fumes, odor, smoke, and vibration were studied in relation to headache among military persons [14]. The authors of [15] reviewed the literature addressing indoor and outdoor environmental factors which are commonly implicated as migraine triggers. In their conclusion they stated: "There are conflicting studies supporting the validity of patient-reported environmental migraine triggers". In another migraine-related study, headache triggers were questioned in 190 patients. As a result the following factors reported in the study were stressor factors: noise, sleep disturbances, fatigue, hunger, physical effort, light, sun, cold, factors affecting scalp, traveling by motor vehicle, eye strain, crowd, odours, crying, weather, high blood pressure, cigarette smoke, and menstruation, respectively [16].  The limitations of this study are typical of this type of research. They include the adequacy of the model and impact of measurement error in the exposure and outcome variables [8]. The main goal of this work was to investigate female ED visits for migraine in Ottawa. As some other studies indicated the association of ED visits for migraine with exposure to sulphur dioxide, the question was if we have large number of visits (3,358 cases in Ottawa) and relatively low exposure, is it possible to observe associations? We applied the casecrossover methodology with extended number of control periods; this allowed better contrasting exposure in hazard and control periods.

Conclusions

The results support the hypothesis that ED visits for migraine condition are associated with ambient sulphur dioxide. The large majority of this air pollution arises from combustion of fossil fuels (e.g., diesel motor vehicles).

Figure 1 :
1Toronto. (a) Median (Q2), Q1, Q3, cases scaled by 76. (b) ORs with their 95% CIs.

Figure 2 :
2Ottawa. (a) Median (Q2), Q1, Q3, cases scaled by 355. (b) ORs with their 95% CIs.

Figure 3 :
3Ottawa. ORs with their 95% CIs. (a) Controls ± 3 days. (b) Controls [−6, 6].

13 days ([−6, 6]): age group [15, 50], OR = 1.07 (95% CI: 1.00, 1.16); age group [25, 50], OR = 1.09 (95% CI: 1.00, 1.19); age group [25, 60], OR = 1.08 (95% CI: 1.00, 1.18). Thus the results are close to be statistically significant (P value < 0.05).

Age group (first: [0, 29], last: [56, 85]) OR [95%Cl] for ED visits for migraine Ottawa, control: [−6, 6]1.3

1.2

1.1

1

0.9

0.8

0
1 0
2 0
3 0
4 0
5 0

Ottawa, control: +/−3

Age group (first: [0, 29], last: [56, 85])
OR [95%Cl] for ED visits for migraine

95%Cl

SO 2 , female: OR

(a)

1.3

1.2

1.1

1

0.9

0.8

0
1 0
2 0
3 0
4 0
5 0
95%Cl

SO 2 , female: OR

(b)

AcknowledgmentsThe authors appreciate the efforts of Health Canada for securing these data and for funding data acquisition. The authors acknowledge Environment Canada for providing the air pollution data from the National Air Pollution Surveillance network that was used in this paper.
The case-crossover design: a method for studying transient effects on the risk of acute events. M Maclure, American Journal of Epidemiology. 1332M. Maclure, "The case-crossover design: a method for study- ing transient effects on the risk of acute events," American Journal of Epidemiology, vol. 133, no. 2, pp. 144-153, 1991.

Air pollution and daily ED visits for migraine. M Szyszkowicz, D M Stieb, B H Rowe, American Journal of Emergency Medicine. 274M. Szyszkowicz, D. M. Stieb, and B. H. Rowe, "Air pollution and daily ED visits for migraine and headache in Edmonton, Canada," American Journal of Emergency Medicine, vol. 27, no. 4, pp. 391-396, 2009.

Ambient sulphur dioxide exposure and emergency department visits for migraine in Vancouver, Canada. M Szyszkowicz, B H Rowe, G G Kaplan, International Journal of Occupational Medicine and Environmental Health. 221M. Szyszkowicz, B. H. Rowe, and G. G. Kaplan, "Ambient sul- phur dioxide exposure and emergency department visits for migraine in Vancouver, Canada," International Journal of Occupational Medicine and Environmental Health, vol. 22, no. 1, pp. 7-12, 2009.

Emergency department visits for migraine and headache: a multi-city study. M Szyszkowicz, G G Kaplan, E Grafstein, B H Rowe, International Journal of Occupational Medicine and Environmental Health. 223M. Szyszkowicz, G. G. Kaplan, E. Grafstein, and B. H. Rowe, "Emergency department visits for migraine and headache: a multi-city study," International Journal of Occupational Med- icine and Environmental Health, vol. 22, no. 3, pp. 235-242, 2009.

Air pollution and hospitalization for headache in Chile. R E Dales, S Cakmak, C B Vidal, American Journal of Epidemiology. 1708R. E. Dales, S. Cakmak, and C. B. Vidal, "Air pollution and hospitalization for headache in Chile," American Journal of Epidemiology, vol. 170, no. 8, pp. 1057-1066, 2009.

World Health Organization, The International Classification of Diseases. Geneva, SwitzerlandWHO9th editionWorld Health Organization, The International Classification of Diseases, WHO, Geneva, Switzerland, 9th edition, 1997.

Case-crossover analyses of air pollution exposure data: referent selection strategies and their implications for bias. H Janes, L Sheppard, T Lumley, Epidemiology. 166H. Janes, L. Sheppard, and T. Lumley, "Case-crossover analyses of air pollution exposure data: referent selection strategies and their implications for bias," Epidemiology, vol. 16, no. 6, pp. 717-726, 2005.

Case-crossover design: air pollution and health outcomes. M Szyszkowicz, N Tremblay, International Journal of Occupational Medicine and Environmental Health. 243M. Szyszkowicz and N. Tremblay, "Case-crossover design: air pollution and health outcomes," International Journal of Occupational Medicine and Environmental Health, vol. 24, no. 3, pp. 249-255, 2011.

Potential for bias in case-crossover studies with shared exposures analyzed using SAS. S V Wang, B A Coull, J Schwartz, M A Mittleman, G A Wellenius, American Journal of Epidemiology. 1741S. V. Wang, B. A. Coull, J. Schwartz, M. A. Mittleman, and G. A. Wellenius, "Potential for bias in case-crossover studies with shared exposures analyzed using SAS," American Journal of Epidemiology, vol. 174, no. 1, pp. 118-124, 2011.

Weather and emergency room visits for migraine headaches in Ottawa, Canada. P J Villeneuve, M Szyszkowicz, D Stieb, D A Bourque, Headache. 461P. J. Villeneuve, M. Szyszkowicz, D. Stieb, and D. A. Bourque, "Weather and emergency room visits for migraine headaches in Ottawa, Canada," Headache, vol. 46, no. 1, pp. 64-72, 2006.

Weather and air pollution as triggers of severe headaches. K J Mukamal, G A Wellenius, H H Suh, M A Mittleman, Neurology. 7210K. J. Mukamal, G. A. Wellenius, H. H. Suh, and M. A. Mit- tleman, "Weather and air pollution as triggers of severe head- aches," Neurology, vol. 72, no. 10, pp. 922-927, 2009.

African dust-laden atmospheric conditions activate the trigeminovascular system. H Doganay, D Akcali, T Goktaş, Cephalalgia. 2910H. Doganay, D. Akcali, T. Goktaş et al., "African dust-lad- en atmospheric conditions activate the trigeminovascular sys- tem," Cephalalgia, vol. 29, no. 10, pp. 1059-1068, 2009.

Headaches reported in people exposed to WTC dust cloud post 9/11. M B Nierengarten, Neurology Today. 106M. B. Nierengarten, "Headaches reported in people exposed to WTC dust cloud post 9/11," Neurology Today, vol. 10, no. 6, pp. 1,16-17, 2009.

Headache triggers in the US military. B J Theeler, K Kenney, O A Prokhorenko, U S Fideli, W Campbell, J C Erickson, Headache. 505B. J. Theeler, K. Kenney, O. A. Prokhorenko, U. S. Fideli, W. Campbell, and J. C. Erickson, "Headache triggers in the US military," Headache, vol. 50, no. 5, pp. 790-794, 2010.

Migraine and the environment. D I Friedman, T De Ver Dye, Headache. 496D. I. Friedman and T. de ver Dye, "Migraine and the environ- ment," Headache, vol. 49, no. 6, pp. 941-952, 2009.

Trigger factors of migraine. A Kutlu, I Yaluǧ, S Mülayim, O Obuz, M Selekler, Archives of Neuropsychiatry. 471A. Kutlu, I. Yaluǧ, S. Mülayim, O. Temel Obuz, and M. Selekler, "Trigger factors of migraine," Archives of Neuropsychiatry, vol. 47, no. 1, pp. 58-63, 2010.