Efficacy of Virtual Reality Exposure Therapy in the Treatment of PTSD: A Systematic Review
2012

R Gonçalves
A L Pedrozo
Esf Coutinho
I Figueira
P Ventura
Efficacy of Virtual Reality Exposure Therapy in the Treatment of PTSD: A Systematic Review

PLoS ONE
712484692012Received June 26, 2012; Accepted September 25, 2012;Editor: Mel Slater, ICREA-University of Barcelona, Spain Funding: The study was partially supported by CNPq (Brazilian Research Council). No additional external funding was received for this study. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript. Competing Interests: The authors have declared that no competing interests exist. *
The use of Information and Communication Technologies, such as virtual reality, has been employed in the treatment of anxiety disorders with the goal of augmenting exposure treatment, which is already considered to be the first-line treatment for Post-traumatic Stress Disorder (PTSD). To evaluate the efficacy of virtual reality exposure therapy (VRET) in the treatment of PTSD, we performed a systematic review of published articles using the following electronic databases: Web of Science, PubMed, PsycINFO, and PILOTS. Eligibility criteria included the use of patients diagnosed with PTSD according to DSM-IV, the use of cognitive behavioral therapy (CBT) and the use of virtual reality for performing exposure. 10 articles were selected, seven of which showed that VRET produced statistically significant results in comparison to the waiting list. However, no difference was found between VRET and exposure treatment. Of these 10, four were randomized, two were controlled but not randomized and four were non-controlled. The majority of the articles used head-mounted display virtual reality (VR) equipment and VR systems specific for the population that was being treated. Dropout rates do not seem to be lower than in traditional exposure treatment. However, there are a few limitations. Because this is a new field of research, there are few studies in the literature. There is also a need to standardize the number of sessions used. The randomized studies were analyzed to assess the quality of the methodology, and important deficiencies were noted, such as the non-use of intent-to-treat-analysis and the absence of description of possible concomitant treatments and comorbidities. Preliminary data suggest that VRET is as efficacious as traditional exposure treatment and can be especially useful in the treatment of patients who are resistant to traditional exposure.

Introduction

Post-traumatic stress disorder (PTSD) involves a constant feeling of fear generated by inadequate consolidation of the autobiographical trauma memory [1]. Foa & Kozak [2] suggested that for there to be adequate processing of the traumatic memory and the consequent extinction of the fear, the memory must be activated and safe components must be inserted. Prolonged Exposure Therapy, proved to be highly eficacious in the treatment of PTSD, aims to access the traumatic memory, including information about the traumatic situation and related emotions, thoughts and behaviors. It helps the patient to understand the context of the traumatic experience as well as its impact in the patient's life. It also enables the patient to achieve a realistic perspective on the traumatic event and its aftermath [3].

Despite the fact that exposure therapy stimulates emotional engagement during imaginal exposure, some patients find it difficult to immerse themselves in the traumatic scene and, therefore, may quit the treatment. In some studies, dropouts and non-response rates may reach 50% of the cases [4]. Therefore, the use of Information and Communication Technologies may facilitate exposure for avoidant patients.

In this sense, virtual reality has been used as a tool for exposure and has achieved positive results in the treatment of various anxiety disorders including specific phobias, social phobia, panic disorder and PTSD [5]. Even though this resource has encountered some difficulties because it raises questions about affecting the therapeutic relationship and struggles with personalizing exposure for individuals with different traumas [5], its use has yielded many benefits. Virtual reality exposure therapy (VRET) facilitates the emotional engagement of patients with PTSD during exposures to the multiple sensory stimuli made possible by the virtual environment, bypassing symptoms of avoidance and facilitating control on the part of the therapist [6][7]. The sense of presence provided by a virtual environment that is rich in sensory stimuli facilitates the emotional processing of memories related to the trauma [8]. This technological apparatus allows gradual exposure to the feared environment according to the needs of each patient. In addition, it can be used in situations where time is limited, as well as in situations that are difficult to control or unpredictable [9] or that could put the patient at risk if the exposure were performed in a real situation. Finally, exposure in a virtual reality environment allows for greater methodological rigor in clinical studies as it allows for the standardization of the duration and type of exposure for all patients [10].

The objective of this article is to conduct a systematic review of studies that have used virtual reality in the treatment of PTSD. We aim to verify the efficacy of Virtual Reality Exposure Therapy for patients diagnosed with PTSD.

Methodology

We performed an electronic search in the following databases: ISI/Web of Knowledge, PUBMED/MEDLINE, PILOTS and PsycINFO in May 2011 with keywords that included the terms PTSD OR ''stress disorder*'' and ''virtual reality''. In the ISI/ Web of Knowledge, we restricted the search criteria to include only ''articles'' and ''notes''. Articles found in the references of the articles selected for this review were also considered for analysis. Independently data collection process was applied. The articles included in the final selection had to fulfill the following criteria.

Criteria for inclusion: the use of patients diagnosed with PTSD according to DSM-IV, the use of cognitive-behavioral therapy (CBT) and the use of virtual reality for performing exposure.

Criteria for exclusion: theoretical articles, reviews, theses, dissertations and book chapters, articles not published in a peerreviewed journal, articles that used patients who were not diagnosed with full PTSD or techniques that were not part of the repertoire of CBT, case studies, articles that did not include outcome criteria and trials published in languages other than English, Portuguese or Spanish.

After the search phase, we did an analysis of the methodological quality of the randomized controlled trials based on the Cochrane Collaboration Tool for Assessing the Risk of Bias [11]. This tool analyzes the risk of bias for each study that might interfere with the results. In addition, six items were added that also represent risks for biased results. These items were based on the scale developed by Kocsis et al [12] and include description of relevant comorbidities; information on screened, included and excluded subjects; description of treatment (or reference); description of concurrent treatment; an intent-to-treat analysis; and description of dropouts.

Results

The final result of the search can be observed in the flowchart ( Figure 1).

Of the 300 articles obtained through the electronic search summed from all the databases searched, 27 were excluded for using samples including non-PTSD disorders or using patients with subclinical PTSD, 36 were excluded for being case studies, 192 were excluded for being theoretical articles, reviews or trials that did not include CBT, two were excluded for using outcome measures that did not include symptoms of PTSD and seven were excluded for using a sample compiled and used in a later study. Thus, 10 articles were considered for the final analysis. This count includes articles found in more than one electronic database, which were, therefore, repeated. Additionally, some studies met more than one exclusion criterion. Each of the studies selected is described below.

Ready et al [7] performed an open trial with 21 Vietnam veterans. The final result combined the samples with those from an earlier study by Rothbaum et al [8], that consisted of a sample of five participants. Nine patients in the most recent study completed the treatment, and six patients dropped out during it. Of the 21 initial participants, 14 completed at least one posttreatment assessment. The questionnaires used included the Clinician Administered PTSD Scale (CAPS), Impact of Events Scale (IES), Beck Depression Inventory (BDI) and Subjective Units of Discomfort Scale (SUDS). The protocol included 8 to 20 weekly sessions, and each lasted 90 minutes. The techniques used included psychoeducation, anxiety management techniques, exposure using VR and imaginal exposure. Follow-ups were performed at three and six months. There was a significant reduction in total CAPS and IES after treatment and during the follow ups. However, for the intrusion cluster on the CAPS scale, no statistical significance was observed at the first measure posttreatment; however, at the follow-ups, there was significant improvement compared to the beginning of treatment. Although there was a significant reduction in BDI during post-treatment and at the follow-up at six months, this was not evident at the threemonth follow-up. Gains were maintained for 10 of the eleven patients who completed all of the follow-ups.

Difede et al [13] evaluated the efficacy of treatment with VRET in-patients who developed PTSD after the terrorist attack on the World Trade Center. The participants were assigned to two groups. In the first group, thirteen patients underwent intervention with VRET, and eight matched subjects were assigned to the waiting list. Five of the thirteen participants in the first group were resistant to previous treatments, and four were resistant to imaginal treatment. Because it was decided that these patients would be assigned to treatment, the study became quasiexperimental. The participants in the VRET group were paired with the participants on the waiting list according to the severity of their CAPS scores and socio-demographic characteristics. The study sample was composed entirely of males with severe PTSD. In addition to the use of CAPS, the participants were evaluated by an independent assessor using BDI and Clinical Global Impression (CGI). The treatment combined CBT with VR using techniques including psychoeducation, relaxation training, cognitive restructuring and VRET. The number of sessions was less than 14, with a mean of 7.5. All of the patients received at least six weeks of VRET. The results showed that nine of the 10 participants who completed the training obtained clinically and statistically significant improvements. A large effect size was obtained (Cohen's d = 1.54). Compared to the waiting list, the group treated with VRET demonstrated statistically significant improvement based on the CAPS score. The same did not occur for BDI and CGI, for which an interaction between time and group was not noticed. However, the authors stated that the baseline scores were low. At the end of the study, six of the 10 participants in the VRET group no longer met the diagnostic criteria for PTSD. Three patients from the group undergoing VRET did not complete the treatment, and one dropped out due to the anxiety related to the exposure. The dropout rate in the control group was not reported. The results remained the same in the follow-up at six months.

Wood et al [14] examined the effects of VRET on twelve participants in a non-controlled study. 10 sessions that lasted 90 to 100 minutes each were performed once or twice a week. The Post Traumatic Stress Disorder Checklist (PCL-M), the BAI and PHQ-9 questionnaires were used. The techniques and procedures used in the study included psychoeducation, meditation for anxiety management, cognitive restructuring, biofeedback training and gradual exposure using virtual reality. There was a significant reduction in the PCL in 75% of the participants, and these cases no longer fulfilled the criteria for PTSD. The study did not report if there were dropouts over the course of the treatment, as well as averages or standard-deviations and only showed the results with a graph. The study did not perform a follow-up. The focus of this article was on the cost-benefit analysis of using VR; thus, the article may be limited in terms of methodology.

A randomized study by Ready et al [15] compared VRET with Therapy Centered on the Present (TCP) in patients who are veterans of the Vietnam War. Of eleven male participants, nine completed the treatment, with five in the VRET group and four in the TCP group. Protocols for both treatments consisted of 10 sessions. The CAPS and BDI scales were administered before and after treatment and during a follow-up six months later by an independent assessor who was unaware of the treatment. There was an improvement of symptoms in both conditions from pretreatment to post-treatment and at the follow-up, performed six months after the end of the study, with scores on the CAPS scale showing greater variations for VRET. However, there was not a statistically significant improvement on the CAPS and BDI scales either within or between groups. It was not possible to compare and find statistically significant differences due to the small sample. There were two dropouts, one from each group.

Roy et al [16] studied soldiers diagnosed with PTSD using functional magnetic resonance imaging (fMRI) in addition to evaluating the effects of VRET. In the first phase of the study, there were 29 subjects who were assigned to four groups: fifteen with both PTSD and Traumatic Brain Injury (TBI), nine with PTSD only, one with TBI only, and four controls. The fMRI was performed before and after treatment and during follow-up periods. In the treatment phase, fifteen participants with PTSD were initially selected at random for VRET (seven) or for Prolonged Exposure (eight). At least twelve sessions lasting 90 min were performed over $ six weeks. Both the PE and VRET treatments used psychoeducation, relaxation techniques and some elements of imaginal and in vivo exposure. The groups received equivalent times of exposure (with VR or imaginal exposure), which lasted up to half of each session. The CAPS, PCL-M and BDI scales were administered at the same time as the cerebral scans. It was an ongoing study in which eight participants with PTSD had completed the final scan, and six had dropped out. As the authors considered the number of participants in each group insufficient for a comparison of the efficacy of treatments, the results of the two groups were combined, meaning, in this respect, that the study was non-controlled. There was a nonsignificant improvement in the clinical symptoms, which could be measured through reductions in CAPS, PCL, CGI and BDI. The results were maintained in the follow-ups performed at one, two and three months with eight patients with PTSD completing the post-treatment scan (five with both PTSD and TBI and three with PTSD only). The brain regions of interest were the amygdala, the hippocampus and the anterior cingulate cortex. Significant improvements are evident on the fMRI scans, which showed a reduced activation of the amygdala, subcallosal gyrus and lateral prefrontal cortex as well as a less-accentuated reduction for the anterior cingulate cortex. These results are corroborated by CGI scores; however, the CAPS score improvements are not statistically significant.

The study by McLay et al [17] is a controlled non-randomized study that shows the results of treatment for PTSD in soldiers from the Iraq War, all male. They compared six cases treated with VRET and four cases with traditional exposure therapy. The treatments were conducted once or twice per week, and the number of sessions varied between six and twelve. There was no standardization in the administration of the techniques or in the number of sessions. The patients were treated according to the clinical judgment of what they required, and they typically received the treatment to which they were assigned. The scales administered included PCL-M, CAPS, PHQ-9 and BAI. All of the patients who were treated with VRET significantly reduced their scores on the PCL-M (by 67% on average), BAI and PHQ-9. At the end of the treatment, five no longer met the diagnostic criteria for PTSD. The patients who received traditional exposure therapy also showed statistically significant improvement at the end of the treatment (a 74% decrease on the PCL-M), and none showed the diagnostic criteria for PTSD at this stage. There was no statistically significant difference between the treatments. The study didǹt perform a follow up. There were no dropouts.

Rizzo et al [18] conducted an open trial with 40 war veterans who were resistant to pharmacological treatment and counseling using virtual Iraq. Sessions lasting 90 to 120 minutes were held twice per week for five weeks. The average number of sessions was less than 11. The techniques used included psychoeducation, anxiety management techniques and imaginal, in vivo and VR exposure. Participants were evaluated based on the PCL-M, BAI and Patient Health Questionnaire (PHQ-9), in addition to physiological monitoring, which included heart rate, galvanic skin response and respiration frequency. However, the results of the psychophysiological monitoring were not reported. A follow-up was performed at three months. Of the 20 participants who completed the treatment, there was a clinical and statistically significant reduction in the PCL-M, with 17 participants showing a reduction of more than 50%. Furthermore, sixteen of the participants did not fulfill the criteria for PTSD after treatment. When looking at the average scores for the BAI and the PHQ-9, there was a statistically significant reduction at the end of treatment. Half of the patients did not complete the treatment.

Seven soldiers dropped out before the first session, six quit after the first session, and seven at the beginning of the VR treatment. Gains were maintained at follow-ups.

The study by Botella et al [19] included subjects who had experienced a broad variety of traumatic events and used EMMA's World as a VR tool. This system used symbols to represent various traumatic events. Their sample included eight men and two women. This was a randomized clinical trial with two groups that used VRET or CBT with in vivo and imaginal exposure. Participants on the waiting list were not included. The study included 10 participants. The authors did not report the number of participants in each group or whether any participant did not complete treatment. Both protocols used from nine to twelve sessions. There was a statistically significant reduction on all the scales administered (CAPS, Davidson Trauma Scale (DTS) and Posttraumatic Cognitions Inventory (PTCI) between pre-and post-treatment conditions for the group that received VRET. VRET proved to be more effective than traditional therapy on the CAPS scale, but the difference was not statistically significant for the CAPS scale or for the others administered. The researchers did not perform follow-ups and did not report the dropout rate for either of the groups.

The study by Gamito et al [20] evaluated VRET as an alternative procedure for reducing symptoms of chronic PTSD in elderly Portuguese veterans who had participated in the colonial war in Africa. 10 participants, all male, were randomly assigned to three groups: five for VRET, two for imaginal exposure, and three for the waiting list. The protocols for VRET and imaginal exposure were based on cognitive desensitization with twelve sessions of exposure with virtual reality for VRET and twelve sessions of traditional imaginal exposure for imaginal exposure group. The first session for both protocols was dedicated to psychoeducation. The CAPS, Impact of Events Scale Revised (IES-R), Symptoms Checklist Revised (SCL-90-R) and BDI scales (only for the VR group) were administered before and after treatment. There was no statistically significant difference in the analysis of the CAPS scale. However, the group treated with VRET obtained a reduction of 8% when measured with CAPS, although there was only a reduction of 1% for the group assigned to imaginal exposure and 6% for the WL. There was a reduction in IES-R for the group with VRET, whereas an increase was observed for the other two groups. There was a significant reduction in symptoms of depression (reduction of 40% on the BDI) and on the SCL-90-R for the VRET group. No follow-up was performed. One dropout was reported in the VRET group, and the dropout rate in the control group was not reported.

The study by McLay et al [21] evaluated the efficacy of VRET in the treatment of 20 Active Duty military personnel in a randomized trial. 10 of these patients were assigned to VRET and 10 to treatment as usual (TAU). The participants of the latter group received a combination of typical treatments for PTSD, which included Prolonged Exposure, Cognitive Processing Therapy, Eye Movement Desensitization and Reprocessing, group therapy, psychiatric medication, substance rehabilitation and inpatient service. The VRET protocol included sessions of psychoeducation, relaxation, attentional and autonomic control training and exposure to a VR simulation of Iraq or Afghanistan. There was no standardization of the treatment protocol, and the number of sessions varied from 10 to 13 weekly sessions for the two groups. The participants were monitored using the CAPS scale before and after treatment. At the end of the treatment, seven of the 10 participants in the VRET protocol showed improvements of greater than 30% on the CAPS scale, whereas only one of the nine participants who completed the TAU showed an improvement greater than 30%. Both treatments showed efficacy and there was no statistically significant difference between the groups. Only one dropout in the TAU group was reported. The researchers did not perform a follow-up. Figures 2 and 3 summarize the different aspects concerning the methodological quality of the randomized studies. Among these investigations, only one study reported the method used for generating the random sequence and whether the method used to hide the sorting did not allow a prediction of the distribution of the patients into groups. Two of the studies did not use a blind evaluator; one did not report on this, and one made use of a blind evaluator. In addition, only one of the studies included data for all the subjects in the final analysis. In all of the articles, the main scales were also administered at the final evaluation. In one study, the presence of comorbidities was not described, and in three studies, they were partially described. The numbers of subjects screened, included and excluded were described completely in only one study. In three studies, the treatments were sufficiently described or referenced to allow replication. Only two studies clearly provided information on concurrent treatments allowed and administered during the course of the study, and they did not permit concomitant treatments. In one study, this description was partially completed, and in another study, there was the possibility that the participants in group therapy were concurrently taking non-stabilized psychotropic medication. None of the articles performed an intent-to-treat analysis because there were no evaluations over the course of treatment. In fact, there were only pre-and post-treatment evaluations.

Discussion

General results

To the best of our knowledge and despite the current relevance of this subject, this is the first systematic review that evaluates the efficacy of VRET focused exclusively on patients diagnosed with PTSD. A study conducted by Meyerbroker & Emmelkamp [5] included a systematic review of VRET in various anxiety disorders including PTSD. However, only two studies were considered in their analysis [7][8][9][10][11][12][13]. A meta-analysis was performed by Parsons et al [22], and two studies with PTSD were selected. The metaanalysis was based on fewer than 20 subjects. Two different metaanalyses in VRET and anxiety disorders were published [23][24], including only one paper in PTSD [13]. The lack of systematic reviews and meta-analyses on the subject is probably due to the small number of studies in the literature that use VR as a tool for exposure. However, the few published studies are recent, with the first dating from 2006 [7], which indicates rapid growth in this area.

Based on the evaluations of the 10 studies, the results point to the potential efficacy of VRET in the treatment of PTSD. Among the six studies that included a control group, a statistically significant reduction in questionnaires evaluating symptoms of PTSD was observed in four. In comparison to the waiting list, results were significantly better for VRET. However, no difference was found between VRET and exposure treatment. The same result was obtained in the meta-analyses on the efficacy of VRET and classical cognitive-behavioral evidence-based treatment in the treatment of anxiety disorders, performed by Opris & colleagues [24]. We cannot discard, nonetheless, the possibility of a Type 2 error due to low sampling power; that is, if the samples were larger, a difference in efficacy in favor of one of the treatments might have been found. In three of the four non-controlled studies, VRET showed efficacy, which confirms the potential efficacy of this treatment.

Eight of the 10 studies used samples of war veterans. The high utilization of this population was probably due to the high financial and social costs of wars in terms of the number of soldiers who return home with PTSD [14]. In two studies, samples included victims of various traumas, and the remaining study was conducted with victims of the terrorist attack of September 11, 2001. War veterans are considered to be a more resistant sample 7 . Thus, the results obtained might have been even more promising if the studies had used subjects exposed to other types of trauma. A problem involved in using VRET in comparison to traditional exposure is related to the difficulty of constructing a context that refers precisely to the traumatic memory of the patient [5]. Due to its high financial cost, it would be almost impossible to follow the idiosyncratic method, which means personalizing the virtual environment according to the perspective of each patient related to the trauma. By contrast, with a homogeneous sample, only some stimuli elicited by the virtual environment are necessary to produce anxiety sufficient for activation of the traumatic memory [7]. In a study which used individuals that had suffered various types of trauma [19], the virtual environment was even more general, but it still evoked recollections of the traumatic event.  The number of sessions varied between 5 and 20. Only four studies set a fixed number of sessions. The others proposed only a minimum number that could be exceeded depending on the needs of the patient. The lack of standardization makes it difficult to evaluate the efficacy of the protocol, although it is understandable considering that these were the first studies to test the efficacy of VRET in the treatment of PTSD. Therefore, there is no standard to follow. The sessions basically consisted of techniques that included psychoeducation, anxiety management (such as diaphragmatic breathing, progressive muscle relaxation and meditation) and in vivo and imaginal exposure. Only three studies reported using cognitive restructuring [13,[14][15][16][17][18][19][20][21].

The majority of the articles used head-mounted display VR equipment and VR systems specific for the samples. Only one study used an automatic virtual environment, an adaptive VR display called EMMA's World [19]. This system allows for deep immersion into the virtual environment for both the patient and the therapist, which offers a better perception of what the patient is seeing. This VR equipment is non-specific, and different environments could bring the patient into contact with different feelings that serve as triggers for traumatic memories. No other studies used the automatic virtual environment methodology. So, it was not possible to test which of the two methodologies was more efficacious in the treatment of PTSD.

Methodological Issues

Randomized clinical trials. We found that the four randomized studies were not conducted in accordance with rigorous criteria for quality. All of them had methodological limitations that introduced a risk of bias for the final results. Proper reporting of the items that constitute a randomized trial assists in the critical evaluation of internal validity and the generalization of the results [25].

It is striking the absence of information to judge the quality of the trials. The majority of studies omitted data concerning random sequence generation, allocation concealment, description of relevant comorbidities, information on screened, included and excluded subjects and description of incomplete outcome data. Besides, even in the study with the higher amount of items evaluated favorably [15], those items account for less than half of the total of the evaluated items. The items with the lower risk of bias were selective reporting and description of treatment.

Another serious limitation concerns the use of extremely small samples in each of the randomized trials. For example, two studies [19][20], despite being randomized, involved only 10 participants each. The randomized study with the largest sample included 20 subjects [21]. These small samples make a meta-analysis focused on those data impossible. Finally, the absence of effect size makes it difficult to understand to what extent the result is explained by the predictor variable [26]. In addition, none of the studies used the number needed to treat (NTT), which is a more powerful and intelligible statistical tool because it establishes the necessary minimum number of individuals to be treated in the event that one of them no longer fulfills the diagnostic criteria after the treatment [27].

Future studies should take into account the methodological issues mentioned above and presented in figures 2 and 3 in order to improve the quality of the trials in VRET.

All Studies. We observed an absence of key information that is required to understand the evaluation of VRET efficacy because some studies focused on the description of the VR system configured for exposures, thereby placing less emphasis on evaluating the efficacy of the treatment. Some studies did not even present quantitative results but only qualitative information about efficacy [14][15][16][17][18].

In spite of these limitations, all of the articles reported the results of the most important scale at the final evaluation. Therefore, there was no risk of hidden possible negative results. In addition, half of the articles performed follow-ups. Those data were collected from one month to one year after the end of treatment. Although the ideal time suggested for a follow-up begins at twelve months [12], the fact that there was follow-up assessment shows an effort to incorporate a more robust methodology. We believe that, with advances in the study of VRET efficacy in the treatment of PTSD, future studies will achieve better methodology quality.

Another important consideration is treatment adherence. It is known that exposure techniques can have a high rate of nonadherence due to the initial worsening of the symptoms. For there to be extinction of the fear associated with the memory of the traumatic event, it is necessary for the patient to experience anxiety and feel its reduction after a specified period of repetition of the traumatic memory. Because the technique is highly anxiogenic, many patients do not agree to undergo the treatment or give up at the beginning of treatment. Theoretically, VRET appears to partially bypass this problem by providing exposure in a more controlled and safer environment. However, except for two studies [17][18][19][20][21], there were dropouts from the VRET group for all of the studies that supplied this information. Two studies did not report the dropout rate [14][15][16][17][18][19]. The highest dropout rate occurred in the study by Rizzo et al [18], where half of the total sample of 40 participants did not complete the treatment. Two other studies showed high dropout rates , with six dropouts in each (out of 21 and 29, respectively).

It is important to mention the low treatment adherence rates found in the active duty military populations. Out of 49,425 veterans diagnosed with PTSD, only 9.5% attended nine or more VA mental health sessions in 15 weeks or less in the first year of diagnosis [28]. This population has issues regarding stigma associated with mental health problems, higher rates of alcohol related problems, and competing responsibilities for preparing for the next deployment that have a dramatic negative effect on adherence to treatment [29][30].

Despite the fact that exposure therapy is considered to be highly aversive [31], in a study by Hembree et al [32], the dropout rates for exposure therapy were not different from the rates found for techniques considered to be less aversive, such as Cognitive Therapy, Stress Inoculation Training and EMDR during the treatment of PTSD. However, due to the high dropout rate that was evident in the selected articles, we believe that there remains a certain loss of control of anxiogenic stimuli during VRET. The loss of control might be related to the need for the therapist or the patient himself to manage the equipment so as to reduce the intensity of the stimulus when anxiety is very high. The use of new technologies, such as physiology-driven adaptive VR stimulation that automatizes the system to make it less dependent on the intervention of the psychotherapist during exposure, may be able to bypass this limitation.

Finally, one potential application of VRET would be for patients who are resistant to traditional exposure. For example, Difede et al [13] included five patients who had not responded initially to traditional exposure, and the results showed a positive response to VRET (for three of them, there was a reduction of at least 25% in CAPS, whereas for the other two, the reduction was greater than 50%). Replicating this finding would provide evidence of the mechanisms by which VRET could be more advantageous than traditional exposure. We believe that the results would be promising because VRET prevents patients from using safety signals that make engagement difficult, which compromises the adequate extinction of conditioned fear.

Conclusions

The results of this systematic review suggest the potential efficacy of VRET in the treatment of PTSD for different types of trauma. VRET proved to be as efficacious as exposure therapy. VRET can be particularly useful in the treatment of PTSD that is resistant to traditional exposure because it allows for greater engagement by the patient and, consequently, greater activation of the traumatic memory, which is necessary for the extinction of the conditioned fear.

However, there remains a vast field to be explored that requires methodologically stronger trials and replications of those trials, the standardization of treatment, empirical studies with different VR systems and attempts to increase the rate of adherence to treatment. In future studies, the use of VRET for traditional exposure resistant patients would be interesting, as would the use of artificial intelligence to make the system less dependent on the intervention of the psychotherapist at the moment of exposure.

Supporting Information

(DOC)

Figure 1 .
1Flowchart of the process of identifying and selecting studies. doi:10.1371/journal.pone.0048469.g001

Figure 2 .
2Methodological Analysis of Randomized Controlled Trials; + Low Risk of Bias; -High Risk of Bias; ? Unclear Risk of Bias. doi:10.1371/journal.pone.0048469.g002

Figure 3 .
3Methodological Analysis of Randomized Controlled Trials. doi:10.1371/journal.pone.0048469.g003
PLOS ONE | www.plosone.org
December 2012 | Volume 7 | Issue 12 | e48469

A dual representation theory of post traumatic stress disorder. C R Brewin, T Dalgleish, S Joseph, Psychological Review. 103Brewin CR, Dalgleish T, Joseph S (1996) A dual representation theory of post traumatic stress disorder. Psychological Review 103: 670-686.

Emotional processing: exposure to corrective information. E Foa, M Kozac, Psychological Bulletin. 99Foa E, Kozac M (1986) Emotional processing: exposure to corrective information. Psychological Bulletin 99: 20-35.

Prolonged Exposure Therapy for PTSD: Emotional Processing of Traumatic Experiences Therapist Guide. E Foa, E Hembree, B Rothbaum, Oxford University PressNew YorkFoa E, Hembree E, Rothbaum B (2007) Prolonged Exposure Therapy for PTSD: Emotional Processing of Traumatic Experiences Therapist Guide. New York: Oxford University Press.

Nonresponse and Dropout Rates in Outcome Studies on PTSD: Review and Methodological Considerations. M A Schottenbauer, D B Glass, D B Arnkoff, A V Tendick, S H Gray, Psychiatry. 712Schottenbauer MA, Glass DB, Arnkoff DB, Tendick AV, Gray SH (2008) Nonresponse and Dropout Rates in Outcome Studies on PTSD: Review and Methodological Considerations. Psychiatry 71(2): 134-168.

Virtual reality exposure therapy in anxiety disorders: A systematic review of process-and-outcomes studies. K Meyerbröker, P M Emmelkamp, Depress Anxiety. 2710Meyerbröker K, Emmelkamp PM (2010) Virtual reality exposure therapy in anxiety disorders: A systematic review of process-and-outcomes studies. Depress Anxiety 27(10): 933-944.

Innovative Use of Virtual Reality Technology in the Treatment of PTSD in the Aftermath of. J Difede, H Hoffman, N Jaysinghe, Psychiat Serv. 539Difede J, Hoffman H, Jaysinghe N (2002) Innovative Use of Virtual Reality Technology in the Treatment of PTSD in the Aftermath of September 11. Psychiat Serv 53(9): 1083-1085.

Virtual reality exposure for veterans with posttraumatic stress disorder. D J Ready, S Pollack, B O Rothbaum, R D Alarcon, J Aggression, Maltreatment, & Trauma. 12Ready DJ, Pollack S, Rothbaum BO, Alarcon RD (2006) Virtual reality exposure for veterans with posttraumatic stress disorder. J Aggression, Maltreatment, & Trauma 12: 199-220.

Virtual reality exposure therapy for Vietnam veterans with posttraumatic stress disorder. B O Rothbaum, L Hodges, D Ready, K Graap, R Alarcon, J Clin Psychiatr. 62Rothbaum BO, Hodges L, Ready D, Graap K, Alarcon R (2001) Virtual reality exposure therapy for Vietnam veterans with posttraumatic stress disorder. J Clin Psychiatr 62: 617-622.

Efficacy of virtual reality exposure therapy for driving phobia: a multiple baseline across-subjects design. J Wald, Behav Ther. 35Wald J (2004) Efficacy of virtual reality exposure therapy for driving phobia: a multiple baseline across-subjects design. Behav Ther 35: 621-635.

Using virtual reality to help our patients in the real world. B O Rothbaum, Depress anxiety. 26Rothbaum BO (2009) Using virtual reality to help our patients in the real world. Depress anxiety 26: 209-211.

The Cochrane Collaboration's tool for assessing risk of bias in randomized trials. Version 5.1.0. Chapter 8: Assessing risk of bias in included studies. J Higgins, S Green, Higgins J, Green S (2011) The Cochrane Collaboration's tool for assessing risk of bias in randomized trials. Version 5.1.0. Chapter 8: Assessing risk of bias in included studies. Available: http//www.cochrane-handbook.org. Accessed 2011 Sep 10.

A new scale for assessing the quality of randomized clinical trials of psychotherapy. J H Kocsis, A J Gerberb, B Milroda, S P Roose, J Barber, Comprehensive Psychiatry. 513Kocsis JH, Gerberb AJ, Milroda B, Roose SP, Barber J, et al. (2009) A new scale for assessing the quality of randomized clinical trials of psychotherapy. Comprehensive Psychiatry 51(3): 319-24.

Virtual reality exposure therapy for the treatment of posttraumatic stress disorder following September 11. J Difede, J Cukor, N Jayasinghe, I Patt, S Jedel, J Clin Psychiatry. 6811Difede J, Cukor J, Jayasinghe N, Patt I, Jedel S, et al. (2007) Virtual reality exposure therapy for the treatment of posttraumatic stress disorder following September 11, 2001. J Clin Psychiatry 68(11): 1639-1647.

Cost effectiveness of virtual reality graded exposure therapy with physiological monitoring for the treatment of combat related posttraumatic stress disorder. D Wood, J Murphy, R Mclay, R Koffman, J Spira, Studies in Health Technology & Informatics. 1447Wood D, Murphy J, McLay R, Koffman R, Spira J, et al. (2009) Cost effectiveness of virtual reality graded exposure therapy with physiological monitoring for the treatment of combat related posttraumatic stress disorder. Studies in Health Technology & Informatics 144(7): 223-229.

Comparing virtual reality exposure therapy to present-centered therapy with 11 U.S. Vietnam veterans with PTSD. D J Ready, R J Gerardi, A G Backscheider, N Mascaro, B O Rothbaum, Cyberpsychology, Behavior, and Social Network. 131Ready DJ, Gerardi RJ, Backscheider AG, Mascaro N, Rothbaum BO (2010) Comparing virtual reality exposure therapy to present-centered therapy with 11 U.S. Vietnam veterans with PTSD. Cyberpsychology, Behavior, and Social Network 13(1): 49-54.

Improvement in cerebral function with treatment of posttraumatic stress disorder. M J Roy, J Francis, J Friedlander, L Banks-Williams, R G Lande, Ann NY Acad Sci. 1208Roy MJ, Francis J, Friedlander J, Banks-Williams L, Lande RG, et al. (2010) Improvement in cerebral function with treatment of posttraumatic stress disorder. Ann NY Acad Sci 1208: 142-149.

Exposure Therapy with and without Virtual Reality to Treat PTSD while in the Combat Theater: A Parallel Case Series. R N Mclay, C Mcbrian, M D Wiederhold, B K Wiederhold, Cyberpsychology, Behavior, and Social Network. 131McLay RN, McBrian C, Wiederhold MD, Wiederhold BK (2010) Exposure Therapy with and without Virtual Reality to Treat PTSD while in the Combat Theater: A Parallel Case Series. Cyberpsychology, Behavior, and Social Network 13(1): 37-42.

Development and early evaluation of the Virtual Iraq/Afghanistan exposure therapy system for combat related PTSD. A S Rizzo, J Difede, B O Rothbaum, G Reger, J Spitalnick, Ann N Y Acad Sci. 1208Rizzo AS, Difede J, Rothbaum BO, Reger G, Spitalnick J, et al. (2010) Development and early evaluation of the Virtual Iraq/Afghanistan exposure therapy system for combat related PTSD. Ann N Y Acad Sci 1208: 114-125.

An Adaptive Display for the Treatment of Diverse Trauma PTSD Victims. C Botella, A García-Palacios, V Guillen, R M Bañ Os, S Quero, Cyberpsychology, Behavior, and Social Networking. 131Botella C, García-Palacios A, Guillen V, Bañ os RM, Quero S, et al. (2010) An Adaptive Display for the Treatment of Diverse Trauma PTSD Victims. Cyberpsychology, Behavior, and Social Networking 13(1): 67-71.

PTSD Elderly War Veterans: A Clinical Controlled Pilot Study. P Gamito, J Oliveira, P Rosa, D Morais, N Duarte, Cyberpsychology, Behavior, and Social Networking. 131Gamito P, Oliveira J, Rosa P, Morais D, Duarte N, et al. (2010) PTSD Elderly War Veterans: A Clinical Controlled Pilot Study. Cyberpsychology, Behavior, and Social Networking 13(1): 43-48.

A Randomized, Controlled Trial of Virtual Reality-Graded Exposure Therapy for Post-Traumatic Stress Disorder in Active Duty Service Members with Combat-Related Post-Traumatic Stress Disorder. R N Mclay, D P Wood, J A Webb-Murphy, J L Spira, M D Wiederhold, Cyberpsychology, Behavior, and Social Network. 144McLay RN, Wood DP, Webb-Murphy JA, Spira JL, Wiederhold MD, et al. (2011) A Randomized, Controlled Trial of Virtual Reality-Graded Exposure Therapy for Post-Traumatic Stress Disorder in Active Duty Service Members with Combat-Related Post-Traumatic Stress Disorder. Cyberpsychology, Behavior, and Social Network 14(4): 223-229.

Affective outcomes of virtual reality exposure therapy for anxiety: A meta-analysis. T Parsons, A Rizzo, J Behav Ther Exp Psychiatry. 39Parsons T, Rizzo A (2008) Affective outcomes of virtual reality exposure therapy for anxiety: A meta-analysis. J Behav Ther Exp Psychiatry 39: 250-261.

Virtual reality exposure therapy for anxiety disorders: a meta-analysis. M Powers, P Emmelkamp, Journal of Anxiety Disorders. 22Powers M, Emmelkamp P (2008) Virtual reality exposure therapy for anxiety disorders: a meta-analysis. Journal of Anxiety Disorders 22: 561-569.

Virtual reality exposure therapy in anxiety disorders: a quantitative metaanalysis. D Opris, S Pintea, A García-Palacios, C Botella, S Szamosközi, Depression and Anxiety. 292Opris D, Pintea S, García-Palacios A, Botella C, Szamosközi S, et al. (2012) Virtual reality exposure therapy in anxiety disorders: a quantitative meta- analysis. Depression and Anxiety 29(2): 85-93.

Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration. I Boutron, D Moher, G Douglas, K F Schulz, P Ravaud, Annals of Internal Medicine. 148Boutron I, Moher D, Douglas G, Schulz KF, Ravaud P (2008) Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: Explanation and elaboration. Annals of Internal Medicine 148: 295-310.

When is the difference significant? Estimates of meaningfulnes in clinical research. B T Conner, Clin Psychol -Sci Pr. 171Conner BT (2010) When is the difference significant? Estimates of mean- ingfulnes in clinical research. Clin Psychol -Sci Pr 17(1): 52-57.

The use of the number needed to treat (NNT) in randomized clinical trials in psychological treatment. C Shearer-Underhill, C Marker, Clinical Psychology: Science and Practice. 17Shearer-Underhill C, Marker C (2010) The use of the number needed to treat (NNT) in randomized clinical trials in psychological treatment. Clinical Psychology: Science and Practice 17: 41-47.

VA Mental Health Services Utilization in Iraq and Afghanistan Veterans in the First Year of Receiving New Mental Health Diagnoses. K H Seal, S Maguem, B Cohen, K S Gima, T J Metzler, Journal of Traumatic Stress. 231Seal KH, Maguem S, Cohen B, Gima KS, Metzler TJ, et al. (2010) VA Mental Health Services Utilization in Iraq and Afghanistan Veterans in the First Year of Receiving New Mental Health Diagnoses. Journal of Traumatic Stress 23(1): 5- 16.

Treatment Presentation and Adherence of Iraq/Afghanistan EraVeterans in Outpatient Care for Posttraumatic Stress Disorder. C R Erbes, K T Curry, J Leskela, Psychological Services. 63Erbes CR, Curry KT, Leskela J (2009) Treatment Presentation and Adherence of Iraq/Afghanistan EraVeterans in Outpatient Care for Posttraumatic Stress Disorder. Psychological Services 6(3): 175-183.

Combat Duty in Iraq and Afghanistan, Mental Health Problems, and Barriers to Care. C W Hoge, C A Castro, S C Messer, D Mcgurk, D I Cotting, The New England Journal of Medicine. 3511Hoge CW, Castro CA, Messer SC, McGurk D, Cotting DI, et al. (2004) Combat Duty in Iraq and Afghanistan, Mental Health Problems, and Barriers to Care. The New England Journal of Medicine 351(1): 13-22.

A virtual reality system for the treatment of stress related disorders: A preliminary analysis of efficacy compared to a standard cognitive behavioral program. R M Bañ Os, V Guillen, S Quero, A García-Palacios, M Alcaniz, Int. J. Human-Computer Studies. 699Bañ os RM, Guillen V, Quero S, García-Palacios A, Alcaniz M, et al. (2011) A virtual reality system for the treatment of stress related disorders: A preliminary analysis of efficacy compared to a standard cognitive behavioral program. Int. J. Human-Computer Studies 69(9): 602-613.

Do patients drop out prematurely from exposure therapy for PTSD?. E Hembree, E Foa, N Dorfan, G Street, J Kowalski, J Trauma Stress. 163Hembree E, Foa E, Dorfan N, Street G, Kowalski J, et al. (2003) Do patients drop out prematurely from exposure therapy for PTSD? J Trauma Stress 16(3): 555-562.


Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in Japan

Erminia Colucci
Claire M Kelly
Harry Minas
Anthony F Jorm
Yuriko Suzuki
Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in Japan
R E S E A R C H Open Access
Background: This study aimed to develop guidelines for how a member of the Japanese public should provide mental health first aid to a person who is suicidal.Methods: The guidelines were produced by developing a questionnaire containing possible first aid actions and asking an expert panel of 32 Japanese mental health professionals to rate whether each action should be included in the guidelines. The content of the questionnaire was based on a systematic search of the relevant evidence and claims made by authors of consumer and carer guides and websites. The panel members were asked to complete the questionnaire by web survey. Three rounds of the rating were carried and, at the end of each round, items that reached the consensus criterion were selected for inclusion in the guidelines. During the first round, panel members were also asked to suggest any additional actions that were not covered in the original questionnaire (to include items that are relevant to local cultural circumstances, values, and social norms).Responses to these openended questions were used to generate new items. Results: The output from the Delphi process was a set of agreed upon action statements. The Delphi process started with 138 statements, 38 new items were written based on suggestions from panel members and, of these 176 items, 56 met the consensus criterion. These statements were used to develop the guidelines appended to this article.Conclusions: There are a number of actions that are considered to be useful for members of the Japanese public when they encounter someone who is experiencing suicidal thoughts or engaging in suicidal behaviour. Although the guidelines are designed for members of the public, they may also be helpful to health professionals working in health and welfare settings who do not have clinical mental health training.

Background

As reported by WHO [1], suicide is a huge but largely preventable public health problem, causing almost half of all violent deaths and resulting in one million fatalities every year (almost 3000 every day) worldwide, as well as economic costs in the billions of dollars. Estimates suggest fatalities could rise to 1.5 million by 2020 [2]. For every person who dies by suicide, 20 or more make a suicide attempt, resulting in injury, hospitalization, and emotional and mental trauma, although no reliable data are available on the full extent of suicide attempts [1,2].

In Japan, the annual number of suicides has been over 30,000, since a drastic increase in 1998, and suicide rates have been over 25.0 per 100,000 people, which is above the rate in other developed nations [3]. Suicide rates are particularly high among older men in rural areas, and are rapidly growing among middle aged men in urban areas [4]. As a counter-measure to this situation, the "Basic Act on Suicide Prevention" was issued in 2006, which emphasises building the capacity of gatekeepers to recognise and respond appropriately when a person is at risk of suicide [5].

For every suicide death, there are scores of family and friends whose lives are devastated emotionally, socially and economically [2]. For family members and friends affected by suicide or attempted suicide, the emotional impact may last for many years [1]. Also when the suicide attempt does not result in death, this act will have several consequences (emotional, physical, social and so on) for the individual as well as family, friends and others.

A member of the community who is close to the suicidal person, such as a friend, family member, co-worker or classmate, is likely to be the first person to notice suicide warning signs. However, few have the knowledge and skills required to recognize the imminent risk of suicide and to assist in preventing suicide. If members of the public were provided with simple and practical guidelines on how they should provide first aid to a person who is suicidal (i.e. has expressed suicidal thoughts or intent, whether overt or covert, or has taken action toward making a suicide attempt), they might be able to help a suicidal individual to seek professional help or decide against suicide. Such guidelines could be applied in training courses for the public, as for other first aid training courses.

First aid training is widespread throughout the world, giving members of the public skills to help an injured person before medical help arrives. There are many organizations offering first aid training, but the first aid practices taught in these courses generally conform to national guidelines. While first aid training is common, it generally ignores mental health crises, such as how to assist a suicidal person. Nevertheless, there have been efforts to develop training for the public that does cover these issues, such as Applied Suicide Intervention Skills Training (ASIST) [6] and Mental Health First Aid (MHFA) training [7]. In Japan, a brief and structured training course for medical residents on how to manage a suicidal patient has been developed and implemented by modifying the depression module of the MHFA program. The feasibility and preliminary effectiveness of the training were demonstrated, although some limitations were also highlighted [8]. The program is currently being extended to other human service workers who may encounter suicidal persons in their daily practice.

Unfortunately, there is limited evidence to guide the content of such trainings. While randomized controlled trials provide the highest standard of evidence, it is not feasible or ethical to carry out such trials to evaluate specific suicide first aid strategies. In the absence of high quality evidence, the best option for developing guidelines is expert consensus [9]. There are formal methods for assessing expert consensus that have been used in several areas of health research. One of the most commonly used consensus method is the Delphi process [10]. There are many variants but all involve a group of experts making private ratings of level of agreement with a series of statements, feedback to the group of a statistical summary of the ratings, and then another round of rating. Delphi group members do not meet, so it is possible to do studies using mail or the Internet. The output from the process is statements for which there is substantial consensus in ratings. The Delphi method has been used in health research since the mid-1970s [11]. The Delphi method has been used to develop suicide first aid guidelines for developed English-speaking countries [12], as well as mental health first aid guidelines for non-suicidal self injury [13], panic attacks [14], psychosis [15], depression [16], eating disorders [17], problem drinking [18], problem drug use [19] and traumatic events [20]. However, guidelines produced in these contexts will not necessarily be applicable in countries with different cultures and health systems. We therefore wished to explore the possibility of developing suicide first aid guidelines for a number of Asian countries. This project was undertaken to establish whether the use of the Delphi method is feasible in the development of suicide first aid consensus guidelines for these countries. This method was previously successfully implemented in the production of first aid guidelines for psychosis in Asia [15].

The aim of this project was to produce guidelines for use in particular Asian countries on how a member of the public should provide first aid to a person who is suicidal. The project did not aim to test hypotheses, rather to develop guidelines on first aid actions based on the consensus of expert clinicians. The project involved undertaking separate studies in three countries: Japan, Philippines and India. These three countries were chosen because they are Asian countries with different cultural and religious contexts, different rates of suicide, different levels of economic development, and different levels of availability of mental health services. We, therefore, expected that there would be differences across countries in the views expressed by the expert panels about appropriate guidelines for mental health first aid in relation to suicide [21][22][23][24]. This article presents the results of the study in Japan. The results from India and the Philippines have already been published [25,26]. To the best of the authors' knowledge, no study of this kind has been conducted in these countries before.

Methods

The first aid guidelines were produced using a threestage process: (a) a systematic search of the relevant evidence and claims made by authors of consumer and carer guides and websites; (b) the development of a questionnaire on possible first aid actions which was based on the search; (c) and the consensus of panels of clinicians from each of the countries on which first aid actions should be included in the guidelines.

Systematic search for possible suicide first aid actions in the literature

As part of the project to develop suicide first aid guidelines for developed English-speaking countries [12], a systematic search for possible first aid actions was carried out in the formal professional literature listed in PubMed and PsycLit and other sources such as existing general mental health first aid manuals [7], other relevant manuals and guides on suicide prevention (e.g. Suicide Prevention Skills Training, [27]; Mental Health for Emergency Departments [28]) and relevant web sites (e. g. Samaritans).

Construction of the questionnaire

A questionnaire was constructed from a content analysis of the actions indicated in the literature. Only statements that suggested a potential first aid action (i.e. what the first aider should do) or relevant awareness statements (what the first aider should know) were included in the questionnaire. These statements were grouped into common themes and used by a working group to generate questionnaire items specifying what actions a first aider should take. No judgments were made by the working group about the potential usefulness of the statements. Anything was included that fitted the definition of first aid, even if contradictory to other statements.

The questionnaire developed for English-speaking countries had 114 items, each describing a potential action that a first aider could do, which could be put to the panel for rating. These items covered the following broad areas: identification of suicide risk, assessing seriousness of suicide risk, initial assistance, talking with a suicidal person, no-suicide contracts, ensuring safety, confidentiality, and passing time during the crisis. The items are shown in Additional File 1. For the Asian guidelines, we added a few other items based on previous work on suicide prevention in Asian countries [22,29]. The initial questionnaire thus contained 138 first aid action items, plus 13 questions on participants' socio-demographics, experience/training, and opinions on suicide first aid. Open-ended questions to generate additional culturally-specific items were also included. Given that this was an exploratory project, we used English-language questionnaires, because the cost of doing it in the experts' native languages would have been prohibitive.

Forming the expert panel Professionals in Medicine, Public health, Social work and Psychology working in Japan were recruited by YS and EC (see Figure 1) to form the panel. Professionals recruited were actively involved in suicide prevention programs with the Ministry of Health, Labor and Welfare, or the Cabinet Office. The majority of the participants were local practitioners and some of them served in developing the Japanese national suicide prevention strategy.

In order to increase the cultural appropriateness of the guidelines, when forming the expert panels in each country we were careful to include as wide a representation (cultural and geographical) of professionals as possible.

When invitation letters, together with the Plain Language Statement, were sent to professionals asking them to be involved, they were also invited to nominate any colleagues who they felt would be appropriate panel members. During the recruitment process, potential participants were informed that one of the selection criteria was fluency in written English.

No attempt was made to make panels representative. The Delphi method does not require representative sampling; it requires panel members who are information and experience-rich [15].

The number of panel members in previous Delphi studies has varied considerably from 15 to 60 [30]. The presence of heterogeneous or homogeneous samples influences the decision of the appropriate number of participants. For a homogeneous group, 10 to 15 people can be sufficient, whereas heterogeneous group needs several hundred participants [31]. De Villiers and colleagues [32] argued that the number of participants can vary according to the purpose of the study, its complexity and resources, but a panel usually consists of 15 to 30 participants from the same discipline, or five to 10 per category from different professional groupings. The authors also observed that increasing the group size beyond 30 has seldom been found to improve the results. The participants in this study were all professionals with expertise in suicide prevention, therefore we aimed to have approximately 25-30 members in the panel.

Delphi process

In Round 1 of the Delphi process, panel members were asked to complete an on-line questionnaire. This was administered using SurveyMonkey [33], with the option to complete it by email or paper mail if this was not possible (although no participant opted for this alternative). The questionnaire consisted of a list of first aid actions to rate. Only actions that are do-able by mental health first aiders were included in the list of items to be rated. Members of the expert panel were given the following instructions to guide their judgments:

"The following questionnaire asks about the best way a member of the public can help someone who is thinking about, or planning to suicide. Mental health first aid is defined as help given to someone who is experiencing a mental health problem, or is in a mental health-related crisis, until professional help is received or the crisis resolves. It does not include counseling or therapy. In the case of suicide, mental health first aid is given until the person decides to accept professional help, or decides against suicide. People who offer mental health first aid may be friends, family members, colleagues or acquaintances. They may or may not be involved in the  person's life before or after offering first aid. For brevity, we will refer to the person offering assistance as "the first aider". When completing this questionnaire, you will read statements describing possible actions that the first aider can take to assist a suicidal individual. You will be asked to rate how important each item is as a guideline for a first aider. Please rate as "essential" or "important" those items which you feel should guide most people, most of the time, when assisting a suicidal person. The statements in this questionnaire were derived from a search of both professional and lay literature in English-speaking western countries. Therefore, there will be actions which would be appropriate for members of the public in your country which are not included and there will be actions that may be appropriate in Western countries but not in your cultural context. At the bottom of each page, there is room for you to add suggestions. Please consider the cultural, social and religious environment where you live, and try to add some relevant suggestions on each page. The more panel members add to this questionnaire, the more relevant and useful the guidelines will be for each individual country. Thank you for taking the time to assist us in this important suicide prevention project!".

The definition of mental health first aid given to the panel in the above instructions distinguishes a first aider's role from that of a clinician. In the case of a suicidal person, the first aider responds by getting professional help for the person, and supporting the person and ensuring their safety until the crisis has passed or until the suicidal person is receiving professional treatment and care. The guidelines needed to focus on the immediate prevention of suicide and not on solving the issues that led to the crisis.

Panel members were asked to rate each statement according to how important they believed it was as a potential first aid action for helping a suicidal person. The response scale was: 1. Essential; 2. Important; 3. Don't know/Depends; 4. Unimportant; 5. Should not be included. The scale was purposefully asymmetric because only items with positive ratings were of interest for the guidelines. This scale has worked well in previous guideline development work [12].

At the end of each block of items, the participants were asked to give any comments. In particular, they were invited to comment on items that were in the initial questionnaire that they considered to be culturally irrelevant or unacceptable, or that would not be feasible because of the local health system and other resources. Panel members were also invited to add any additional actions that were not included in the questionnaire.

This was the place where culturally specific material could be introduced. The suggestions made by the panel members in response to the open-ended questions were reviewed by the research team and used to construct new items. Suggestions were accepted and added to Round 2 if they represented a new idea, could be interpreted unambiguously and were actions. Suggestions were rejected if they were near-duplicates of items in the questionnaire, if they were too specific, too general or were more appropriate to therapy than first aid.

One of the main advantages of using the Internet to collect data was that it made it easy for the survey manager to identify those participants who were late in completing the questionnaire or had completed only part of it and, thus, send reminders (with no need to send extra copies of the questionnaire, which can increase waiting times). Furthermore, no questions were inadvertently missed as the web survey was set up so that each question was mandatory. In addition, such survey software allows for branching (so that participants who did not endorse the use of no-suicide contracts were not asked to answer questions about what such a contract should contain).

In previous Delphi studies on suicide first aid, items were included in the guidelines if at least 80% of the panel rated them as "Essential" or "Important, and items were re-rated if 70-79% of the panel endorsed them. However, the research team made a decision to lower the threshold for the Japanese expert panel, because of the small number of items endorsed (mainly due to a large number of "Don't know/depends" answers). Thus, items rated as "Essential" or "Important" by 70% or more of the panel were included in the guidelines, and re-rated if there was 60-69% endorsement.

In Round 2, a second questionnaire was prepared. This consisted of items that met the 60-69% endorsement criterion for re-rating and new items that were generated from the comments in Round 1. In addition, a small number of items that received more than 50% of "Don't know/Depends" or "Not sure" answers were reworded, clarified or specified, and re-rated. In particular, following the comments made by a few participants, some of the items on the "no-suicide contracts" were reworded using the term "agreement" instead. Participants received an email with an individualized link to the online survey and a Word file that fed back a statistical summary of the items that were to be re-rated (i.e. their own original response to the item together with total percentages of endorsement of the item). They were told that they were able to change their responses when re-rating an item if they wished to do so. At the end of this round, any item that reached the 70% consensus criterion was selected for inclusion in the guidelines, any new item reaching the 60-69% consensus criterion was re-rated in Round 3, and the rest were rejected.

Ethics

Ethics approval for the study was obtained from the University of Melbourne Human Research Ethics Committee (Project No. HREC 0605537).

Results

Sample

In Japan, 32 panel members were involved in Round 1 (i.e. 71% of the experts who were invited to participate), 28 in Round 2 and 26 in Round 3. All panel members were currently working in Japan. The majority were psychiatrists (N = 15) and psychologists (N = 5). The remaining were psychiatrist nurses and social workers. The panel was comprised of 18 males and 14 females. The majority of the participants were in the age range 30-39 years (N = 19) and 40-49 years (N = 5), four participants were aged 18-29 years, three were aged 50-59 years, and one was aged 60 years or over.

Some information was also collected on the clinical experience of the panel members. On average, participants reported that they had practiced in mental health/ psychiatry for 12 years (the shortest time was 3 years and the longest 32 years). Just over a quarter of the participants (N = 9) received some formal education related to their profession overseas (mainly in USA). Slightly over a third of the participants (N = 12) reported having received formal training specifically on suicide prevention or intervention. However, when asked to state how well prepared they felt to assist a suicidal person, 16 answered "somewhat prepared", 12 "mostly prepared" and 2 "very prepared". Two participants reported being "not at all prepared". Although participants generally felt prepared to assist a suicidal person, in their opinion most people in Japan are not at all prepared (N = 19) or somewhat prepared (N = 12). Only one participant believed others are "mostly prepared" and no one believed that most people are "very prepared" to assist.

Item endorsement

After three Delphi rounds, there were 56 items that were rated as "essential" or "important" by 70% or more of the panel members.

At Round 2, 38 new items suggested by participants were added to the questionnaire. The following are examples of such items:

• An important warning sign for suicide is if a person is saying they wish or intend to see or speak to someone who is dead (e.g., a deceased family member).

• An important warning sign for suicide is if a person is suddenly cleaning up their things (e.g. putting their room or desk in order).

• An important warning sign for suicide is if a person expresses in words or actions a sense of shame (e.g. from failure or loss).

• The first aider should be aware that suicidal people differ in their chosen suicide methods, so they should pay attention to the presence of any sort of potential suicidal means (not just guns, rope, pills but also knives, any kind of poison, kerosene and so on).

• The first aider should tell the person that they have done the right thing in talking about their suicidal thoughts.

• The first aider should try to convince the suicidal person that it is better to not keep their suicidal intentions a secret but involve someone else (e.g. a professional or a family member).

A number of responses to the Round 1 open-ended questions did not meet criteria for creation of a new item (e.g. they did not fit the definition of first aid or did not suggest a clear action) or were comments/suggestions. The following are examples (quoted verbatim, with minimal editing) of the comments and suggestions that did not generate new items but, nevertheless, are worthy of consideration:

• First Aid guidelines and the training program will not only reduce suicide, but also promote understanding of suicide and mental illness in public. In future, this First Aid should be taught to junior or high school student. I hope you will develop the guideline both in English and in our language so that we can use it.

• In our country (Japan), even mental health professionals are not always familiar with first-aid to suicide, and treatments vary from person to person. So the first step to disseminate first-aid actions should be started with the mental health professionals.

• In Japan, the social beliefs are somehow different from Western countries. Therefore, if we import the MHFA acts, we need to modify it.

• Thank you to request me to answer these questions. So interesting. We Japanese need mental health first aid for suicide, but in our culture/society this is a more ambiguous phrase, I think. I will at first stage focus on giving training to the possible first aider concerning building attitude to listen to others (not only suicidal persons) with empathy.

• The action and commitment of first aider seems to depend much on the background of first aiders (their position, profession and so on).

See the Additional File 1 for a complete list of rated statements, including the percentage of panel members endorsing each item.

At the end of the survey, participants were asked their opinions about the likely effectiveness of suicide first aid, using a 5-point Likert scale (from "definitely yes" to "definitely no"). Twenty-seven of the 29 participants who answered the question believed that if the first aider does the right thing, the risk of suicide can be reduced, whereas two answered "don't know/depends". The majority of the respondents (i.e. 19/29) thought that if the first aider does the wrong thing, the risk of suicide can be increased, whereas 7 answered "don't know/depends" and three "probably no".

The longer-term goal of the project is to use the guidelines to develop, implement and evaluate a training program on suicide first aid in Japan. This goal reflects the opinion of the expert panel. When asked if they thought members of the public should receive such training, 25 participants responded affirmatively. Only three respondents answered "don't know/depends" and one "probably no".

Development of a guidelines document

The output from the Delphi process was a set of 56 agreed upon statements referring to actions that can be done by a mental health first aider. To be usefully communicated, this list of action statements was transformed into narrative text -the Suicide First Aid Guidelines (see Additional File 2).

The draft guidelines were sent to all panel members for their comments and final endorsement. Since the guidelines were meant to be useful to members of the public, it was important to ensure that they were written to be comprehensible to the target non-professional readership. Feedback from panel members was explicitly sought on the structure and readability of the guidelines, and suggested improvements were incorporated in the final version.

Discussion

This project has demonstrated that it is possible to achieve consensus among mental health professionals on first aid strategies for suicidal thoughts and behaviour, and that the Delphi method is suitable for developing consensus guidelines in Japan. The method has been similarly successful in the Philippines [25] and India [26]. We would suggest that guideline development studies, using a similar method, could be carried in a number of other countries. As well as developing country-specific guidelines, it will also be possible to develop guidelines that are appropriate for cultural minorities within a country. This approach has been used in Australia, with a separate Delphi study undertaken to develop guidelines for Indigenous Australians, using Aboriginal mental health experts as panel members [34,35], and specific teaching programs have been developed for non-English speaking immigrant communities [36,37]. Inclusion and exclusion criteria, however, must consider cultural attitudes towards expressing opinions. For instance, while the threshold for items acceptance had to be lowered in the current study (from 80% to 70%) because of the large presence of "don't know/depends" answers, the opposite issue emerged in the study with Indigenous Australians where participants expressed agreement with the majority of items and the acceptance criteria was thus increased to 90%.

The Japanese experts expressed higher agreement on the item "The first aider should keep in mind that asking too many questions can provoke anxiety in the suicidal person" compared to Indian and Filipino experts. Furthermore, other items that could express a more direct or invasive involvement of the first aider were rejected in this country. Japan was also the only country where no statement in the section "Passing time" was agreed upon by the panel. Such findings raise the possibility that the role of a first aider might partially be culturally-bound and, for instance, in countries characterized by greater personal distance such as Japan, some behaviours that could be appropriate from the perspective of first aid strategies could be seen as socially inappropriate. Such differences should be further investigated and considered when developing training programs.

The next steps will be dissemination and use of the guidelines for the purpose of increasing community members' ability to recognize the risk of suicide and undertake basic first aid actions. These guidelines can be used as a source of advice to the public, as a basis for determining the curriculum of first aid training courses, and as a standard against which to evaluate the quality of existing materials and programs. The guidelines can inform the development of culturally appropriate training programs and information materials for how a member of the public can assist someone who is suicidal. At present there are limited structured training programs on suicide management skills available in Japan. Although much training is implemented throughout Japan, most of them mainly consist of didactic lectures with a focus on knowledge of facts about suicide. Provision of information alone may increase the knowledge of the audience; however, it does not necessary change attitudes or helping behaviour towards people with suicidal thoughts [38]. Training on skills enhancement based on these guidelines will certainly be beneficial for raising awareness and strengthening capacity at a community level, and preliminary work to further expand and develop a structured training program is currently underway. It will, of course, be necessary to evaluate the impact of such a training program [36,39,40]. This is the last of the three guidelines that were developed as part of this research project and it is relevant to observe that, despite the diversity of the countries involved, there was consensus on a core set of first aid items that were considered suitable for assisting a suicidal person, and there were other items that were consistently rejected across countries. Nevertheless, there were also differences between these three countries (such as on keeping the suicidal risk a secret in India, suggestion and/or acceptance of items concerning religion in the Philippines, and a greater personal distance expected by the first aider in Japan,) as there are differences with the guidelines for developed Englishspeaking countries [12]. This observation highlights the presence of a number of core basic first aid strategies that might have a cross-national and cultural validity and should form the basis of any training program, and the importance of developing culturally-specific guidelines with the help of local experts.

Limitations

One limitation of this study is the relatively small number of panel members, although Delphi studies have been successfully run even with smaller groups and a size of 20-30 experts is common for this sort of study. Three quarters of the professionals approached agreed to participate. It is likely that some professionals refused participation because of limited proficiency in English, the language used throughout the research project. It is also possible that selecting participants who were fluent in English might have led to the exclusion of experts who were less exposed to the international literature and therefore held more traditional views on suicide.

Administering the questionnaire in English rather than in Japanese limits the general applicability of the findings. It is possible that there would have been more culturally specific responses if panelists had used their native language.

Future studies should recruit broader and more representative expert panels including, where possible, professionals from all the relevant mental health disciplines and consumer and carer representatives. Participation of consumers and carers in such research, as members of an 'expert panel' rather than as research subjects, is still an uncommon practice [15], which also requires additional funding that was not available for the current project. Nevertheless, the local collaborator in this study (YS) and her team are planning to also have consultations with carers and people who have made a suicide attempt.

Another limitation is that the inclusion of culturally relevant material was dependent on panelists responding to the open-ended questions and not every participant did this. In some cases, this may have been due to lack of time or because the participants in this study were better able to read English than to write it. It may also be because, in questionnaires of all kinds that require ratings to be made, respondents rarely take the opportunity to write comments or to make suggestions when the opportunity is given [15]. However, it must be noted that, compared to other similar mental health first aid guideline research, a considerable number of suggestions for new items were given in this study. This might have been because of the emphasis in the instructions to participants in each section of the questionnaire to provide suggestions based on their cultural, social and religious settings.

Finally, while these guidelines have been developed for Japan, it is recognised that this country is characterised by diversity, for instance between rural and urban areas, and some prefectures are more affected by suicide than others.

Conclusions

There is a growing awareness of suicide as a major public health problem, even though there is a taboo in many societies against discussing it openly [1]. Developing suicide first aid guidelines for community members, and training programs based on these, might also contribute towards changing society's attitudes towards suicide and towards those members of the community who are dealing with suicidal intentions.

This study has demonstrated that it is possible to reach consensus on the development of guidelines for Japan. Although the guidelines were designed for the public, they may also contain advice that might be helpful to people working in health and welfare professions.

Where the guidelines are used as the basis for first aid training, efforts need to be made to evaluate their impact on the first aider's helping behaviour and on the recipients of the first aid. This will assist researchers to develop an evidence base for mental health first aid and suicide prevention initiatives.

In a WHO news release [2] it was stated that "It's important to realise that suicide is preventable". By collaborating with local experts to agree on a minimum set of suicide first aid actions, and by making such guidelines freely and easily accessible to everyone, we hope to convey the message that suicide is preventable, suicide is everyone's business, and everyone can contribute to its reduction. Members of the general public have a crucial role to play in suicide prevention. Creating opportunities for the public to learn basic first aid actions, and how to implement them when needed, are important steps towards more effective and global suicide prevention activities.

Additional file 2: First aid guidelines for Japan. This file may be distributed freely, with the authorship and copyright details intact. Please do not alter the text or remove the authorship and copyright details.

Figure 1
1Stages in the guidelines development.
Colucci et al. International Journal of Mental Health Systems 2011, 5:12 http://www.ijmhs.com/content/5/1/12
Colucci et al. International Journal of Mental Health Systems 2011, 5:12 http://www.ijmhs.com/content/5/1/12 Page 6 of 10
AcknowledgementsThe authors gratefully acknowledge the contribution of the following panel members:NaokiAdditional materialAdditional file 1: Authors' contributions EC recruited clinical experts, revised the existing questionnaire, prepared and administered the on-line surveys for Round 1-3, analysed the data, prepared the draft and final guidelines and wrote the first draft of the manuscript. CMK wrote the former questionnaire and contributed to the revised version, supervised every stage of the project and co-wrote the guidelines. HM cowrote the grant application, recruited the main local collaborators and clinical experts, and reviewed the Round 1 questionnaire and the guidelines. AFJ co-wrote the grant application and developed the method. YS was the main local collaborator in the study and recruited the majority of the clinical experts. All authors contributed to the writing of the manuscript and approved the final version.Competing interestsThe authors declare that they have no competing interests.
WHO: Suicide huge but preventable public health problem. 29/04/09WHO: World Suicide Prevention DayWHO: World Suicide Prevention Day. 2007 [http://www.who.int/ mediacentre/news/statements/2007/s16/en], Accessed on the 24/04/09. 2. WHO: Suicide huge but preventable public health problem. 2004 [http://www.who.int/mediacentre/news/releases/2004/pr61/en], Accessed on the 29/04/09.

A community intervention trial of multimodal suicide prevention program in Japan: A Novel multimodal Community Intervention program to prevent suicide and suicide attempt in Japan. Y Ono, NOCOMIT-J. BMC Public Health. 8315Ono Y, et al: A community intervention trial of multimodal suicide prevention program in Japan: A Novel multimodal Community Intervention program to prevent suicide and suicide attempt in Japan, NOCOMIT-J. BMC Public Health 2008, 8:315.

Rapid increases of suicide deaths in metropolitan areas. T Fujita, J Natl Inst Public Health. 52Fujita T: Rapid increases of suicide deaths in metropolitan areas. J Natl Inst Public Health 2003, 52:295-301.

Cabinet Office: Suicide comprehensive guidelines. Cabinet Office: Suicide comprehensive guidelines. 2008 [http://www8.cao. go.jp/jisatsutaisaku/taikou/pdf/20081031taikou.pdf].

Suicide intervention training evaluation: a preliminary report. R J Tierney, Crisis. 15Tierney RJ: Suicide intervention training evaluation: a preliminary report. Crisis 1994, 15:69-76.

Mental health first aid training for the public: evaluation of effects on knowledge, attitudes and helping behavior. B A Kitchener, A F Jorm, BMC Psychiatry. 210Kitchener BA, Jorm AF: Mental health first aid training for the public: evaluation of effects on knowledge, attitudes and helping behavior. BMC Psychiatry 2002, 2:10.

Development of 2-hour suicide intervention program among medical residents: First pilot trial. A T Kato, Psychiatry and Clinical Neurosciences. 64Kato AT, et al: Development of 2-hour suicide intervention program among medical residents: First pilot trial. Psychiatry and Clinical Neurosciences 2010, 64:531-540.

Where there is no evidence: use of expert consensus methods to fill the evidence gap in low-income countries and cultural minorities. H Minas, A F Jorm, Int J Ment Health Syst. 4Minas H, Jorm AF: Where there is no evidence: use of expert consensus methods to fill the evidence gap in low-income countries and cultural minorities. Int J Ment Health Syst 2010, 4.

The Delphi method: techniques and applications. H A Linstone, M Turoff, 10/10/2010Linstone HA, Turoff M: The Delphi method: techniques and applications. 2002 [http://is.njit.edu/pubs/delphibook/], Accessed on the 10/10/2010.

Comparison of mailed vs. Internet applications of the Delphi technique in clinical informatics research. R Snyder-Halpern, C B Thompson, J Schaffer, Proc AMIA Symp. Snyder-Halpern R, Thompson CB, Schaffer J: Comparison of mailed vs. Internet applications of the Delphi technique in clinical informatics research. Proc AMIA Symp 2000, 809-813.

Development of mental health first aid guidelines for suicidal ideation and behaviour: a Delphi study. C M Kelly, A F Jorm, B A Kitchener, R L Langlands, BMC Psychiatry. 817Kelly CM, Jorm AF, Kitchener BA, Langlands RL: Development of mental health first aid guidelines for suicidal ideation and behaviour: a Delphi study. BMC Psychiatry 2008, 8:17.

Development of mental health first aid guidelines for deliberate non-suicidal self-injury: a Delphi study. C M Kelly, A F Jorm, B A Kitchener, R L Langlands, BMC Psychiatry. 862Kelly CM, Jorm AF, Kitchener BA, Langlands RL: Development of mental health first aid guidelines for deliberate non-suicidal self-injury: a Delphi study. BMC Psychiatry 2008, 8:62.

Development of mental health first aid guidelines for panic attacks: a Delphi study. C M Kelly, A F Jorm, B A Kitchener, BMC Psychiatry. 949Kelly CM, Jorm AF, Kitchener BA: Development of mental health first aid guidelines for panic attacks: a Delphi study. BMC Psychiatry 2009, 9:49.

First aid guidelines for psychosis in Asian countries: A Delphi consensus study. A F Jorm, H Minas, R L Langlands, M L Kelly, Int J Ment Health Syst. 22Jorm AF, Minas H, Langlands RL, Kelly ML: First aid guidelines for psychosis in Asian countries: A Delphi consensus study. Int J Ment Health Syst 2008, 2:2.

First aid for depression: a Delphi consensus study with consumers, carers and clinicians. R L Langlands, A F Jorm, C M Kelly, B A Kitchener, J Affect Disord. 105Langlands RL, Jorm AF, Kelly CM, Kitchener BA: First aid for depression: a Delphi consensus study with consumers, carers and clinicians. J Affect Disord 2008, 105:157-165.

First Aid for Eating Disorders. L M Hart, A F Jorm, S J Paxton, C M Kelly, B A Kitchener, Eating Disorders. 17Hart LM, Jorm AF, Paxton SJ, Kelly CM, Kitchener BA: First Aid for Eating Disorders. Eating Disorders 2009, 17:357-384.

Helping someone with problem drinking: Mental health first aid guidelines -a Delphi expert consensus study. A H Kingston, A F Jorm, B A Kitchener, L Hides, C M Kelly, A J Morgan, L M Hart, D I Lubman, BMC Psychiatry. 979Kingston AH, Jorm AF, Kitchener BA, Hides L, Kelly CM, Morgan AJ, Hart LM, Lubman DI: Helping someone with problem drinking: Mental health first aid guidelines -a Delphi expert consensus study. BMC Psychiatry 2009, 9:79.

Helping someone with problem drug use: a Delphi consensus study of consumers, carers, and clinicians. A H Kingston, A Morgan, A Jorm, K Hall, L Hart, C Kelly, D Lubman, BMC Psychiatry. 113Kingston AH, Morgan A, Jorm A, Hall K, Hart L, Kelly C, Lubman D: Helping someone with problem drug use: a Delphi consensus study of consumers, carers, and clinicians. BMC Psychiatry 2011, 11:3.

Development of mental health first aid guidelines on how a member of the public can support a person affected by a traumatic event: a Delphi study. C M Kelly, A F Jorm, B A Kitchener, BMC Psychiatry. 1049Kelly CM, Jorm AF, Kitchener BA: Development of mental health first aid guidelines on how a member of the public can support a person affected by a traumatic event: a Delphi study. BMC Psychiatry 2010, 10:49.

The cultural facet of suicidal behaviour: Its importance and negligence. E Colucci, 5Colucci E: The cultural facet of suicidal behaviour: Its importance and negligence. AeJAMH 2006, 5.

Cultural issues in suicide risk assessment. E Colucci, Suicidal Behavior: Assessment of People-at-risk. Kumar U, Mandal MKNew Delhi: SAGEColucci E: Cultural issues in suicide risk assessment. In Suicidal Behavior: Assessment of People-at-risk. Edited by: Kumar U, Mandal MK. New Delhi: SAGE; 2009:.

Ethnocultural aspects of suicide in young people: a systematic literature review. Part 1: Rates and methods of youth suicide. E Colucci, G Martin, Suicide Life Threat Behav. 37Colucci E, Martin G: Ethnocultural aspects of suicide in young people: a systematic literature review. Part 1: Rates and methods of youth suicide. Suicide Life Threat Behav 2007, 37:197-221.

Ethnocultural aspects of suicide in young people: a systematic literature review. Part 2: Risk factors, precipitating agents, and attitudes toward suicide. E Colucci, G Martin, Suicide Life Threat Behav. 37Colucci E, Martin G: Ethnocultural aspects of suicide in young people: a systematic literature review. Part 2: Risk factors, precipitating agents, and attitudes toward suicide. Suicide Life Threat Behav 2007, 37:222-237.

Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in the Philippines. E Colucci, C Kelly, H Minas, A Jorm, D Nadera, Int J Ment Health Syst. 432Colucci E, Kelly C, Minas H, Jorm A, Nadera D: Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in the Philippines. Int J Ment Health Syst 2010, 4:32.

Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in India. E Colucci, C M Kelly, H Minas, A F Jorm, S Chatterjee, Int J Ment Health Syst. 44Colucci E, Kelly CM, Minas H, Jorm AF, Chatterjee S: Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in India. Int J Ment Health Syst 2010, 4:4.

Suicide Prevention Skills Training Manual: An Accredited Training Program. J Hawgood, D Deleo, Griffith UniversityBrisbane, AustraliaHawgood J, DeLeo D: Suicide Prevention Skills Training Manual: An Accredited Training Program. Brisbane, Australia: Griffith University; 2002.

Mental Health and Drug and Alcohol Office: Mental Health for Emergency Departments -A Reference Guide. Sydney: NSW Department of Health. Mental Health and Drug and Alcohol Office: Mental Health for Emergency Departments -A Reference Guide. Sydney: NSW Department of Health; 2002.

Recognizing spirituality in the assessment and prevention of suicidal behaviour. E Colucci, 3Colucci E: Recognizing spirituality in the assessment and prevention of suicidal behaviour. WCPRR 2008, 3:77-95.

Research guidelines for the Delphi survey technique. F Hasson, S Keeney, H Mckenna, J Adv Nurs. 32Hasson F, Keeney S, McKenna H: Research guidelines for the Delphi survey technique. J Adv Nurs 2000, 32:1008-1015.

The Delphi method for graduate research. G J Skulmoski, F T Hartman, J Krahn, Journal of Information Technology Education. 6Skulmoski GJ, Hartman FT, Krahn J: The Delphi method for graduate research. Journal of Information Technology Education 2007, 6.

. Colucci, International Journal of Mental Health Systems. 5Colucci et al. International Journal of Mental Health Systems 2011, 5:12 http://

The Delphi technique in health sciences education research. De Villiers, M R , De Villiers, Pjt Kent, A P , 5. 33. SurveyMonkeyMedical Teacher. 277De Villiers MR, De Villiers PJT, Kent AP: The Delphi technique in health sciences education research. Medical Teacher 2005, 27:639-643, 27(7), 5. 33. SurveyMonkey. [http://www.surveymonkey.com].

Mental health first aid for Indigenous Australians: using Delphi consensus studies to develop guidelines for culturally appropriate responses to mental health problems. L M Hart, A F Jorm, L G Kanowski, C M Kelly, R L Langlands, BMC Psychiatry. 947Hart LM, Jorm AF, Kanowski LG, Kelly CM, Langlands RL: Mental health first aid for Indigenous Australians: using Delphi consensus studies to develop guidelines for culturally appropriate responses to mental health problems. BMC Psychiatry 2009, 9:47.

Development of mental health first aid guidelines for Aboriginal and Torres Strait Islander people experiencing problems with substance use: a Delphi study. L Hart, S Bourchier, A Jorm, L Kanowski, A Kingston, D Stanley, D Lubman, BMC Psychiatry. 10Hart L, Bourchier S, Jorm A, Kanowski L, Kingston A, Stanley D, Lubman D: Development of mental health first aid guidelines for Aboriginal and Torres Strait Islander people experiencing problems with substance use: a Delphi study. BMC Psychiatry 2010, 10.

Evaluation of Mental Health First Aid training with members of the Vietnamese community in Melbourne. H Minas, E Colucci, A F Jorm, Int J Ment Health Syst. 319Minas H, Colucci E, Jorm AF: Evaluation of Mental Health First Aid training with members of the Vietnamese community in Melbourne. Int J Ment Health Syst 2009, 3:19.

Mental health first aid training for the Chinese community in Melbourne, Australia: effects on knowledge about and attitudes toward people with mental illness. A Lam, A Jorm, D Wong, Int J Ment Health Syst. 418Lam A, Jorm A, Wong D: Mental health first aid training for the Chinese community in Melbourne, Australia: effects on knowledge about and attitudes toward people with mental illness. Int J Ment Health Syst 2010, 4:18.

Evaluation of a brief anti-stigma campaign in Cambridge: do short-term campaigns work? BMC Public Health. Slj Evans-Lacko, K Little, C Henderson, G Thornicroft, 339Evans-Lacko SLJ, Little K, Henderson C, Thornicroft G: Evaluation of a brief anti-stigma campaign in Cambridge: do short-term campaigns work? BMC Public Health 2010, 339.

Experiences in applying skills learned in a Mental Health First Aid training course: a qualitative study of participants' stories. A F Jorm, B A Kitchener, S K Mugford, BMC Psychiatry. 543Jorm AF, Kitchener BA, Mugford SK: Experiences in applying skills learned in a Mental Health First Aid training course: a qualitative study of participants' stories. BMC Psychiatry 2005, 5:43.

Mental health first aid training: review of evaluation studies. B A Kitchener, A F Jorm, 10.1186/1752-4458-5-12Australian and New Zealand Journal of Psychiatry. 40Kitchener BA, Jorm AF: Mental health first aid training: review of evaluation studies. Australian and New Zealand Journal of Psychiatry 2006, 40:6-8. doi:10.1186/1752-4458-5-12

Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in Japan. Colucci, International Journal of Mental Health Systems. 512Cite this article asCite this article as: Colucci et al.: Mental Health First Aid guidelines for helping a suicidal person: a Delphi consensus study in Japan. International Journal of Mental Health Systems 2011 5:12.