The new age of play audiometry: prospective validation testing of an iPad-based play audiometer

Jeffrey Yeung
Hedyeh Javidnia
Sophie Heley
Yves Beauregard
Sandra Champagne
Matthew Bromwich
The new age of play audiometry: prospective validation testing of an iPad-based play audiometer
O R I G I N A L R E S E A R C H A R T I C L E Open AccessHearing lossPlay audiometryPediatric Background & rationale
Objective: The timely diagnosis of hearing loss in the pediatric population has significant implications for a child's development. However, audiological evaluation in this population poses unique challenges due to difficulties with patient cooperation. Though specialized adaptations exist (such as conditioned play audiometry), these methods can be time consuming and costly. The objective of this study was to validate an iPad-based play audiometer that addresses the shortcomings of existing audiometry. Methods: We designed a novel, interactive game for the Apple W iPad W that tests pure tone thresholds. In a prospective, randomized study, the efficacy of this tool was compared to standard play audiometry. 85 consecutive patients presenting to the Audiology Clinic at the Children's Hospital of Eastern Ontario (ages 3 and older) were recruited into this study. Their hearing was evaluated using both tablet and traditional play audiometry.Outcome measure: Warble-tone thresholds obtained by both tablet and traditional audiometry. Results: The majority of children in this age group were capable of completing an audiologic assessment using the tablet computer. The data demonstrate no statistically significant difference between warble-tone thresholds obtained by tablet and traditional audiometry (p=0.29). Moreover, the tablet audiometer demonstrates strong sensitivity (93.3%), specificity (94.5%) and negative predictive value (98.1%).Conclusion:The tablet audiometer is a valid and sensitive instrument for screening and assessment of warble-tone thresholds in children.

Background & rationale

The timely diagnosis of hearing loss in children has significant implications for a child's social and cognitive development. In young children, hearing loss is often inconspicuous resulting in delayed diagnosis and rehabilitation. As a result, children are at increased risk of delayed speech acquisition and the subsequent longterm sequelae. Speech delay and hearing loss also represents a significant cost to society due to the resources required for their treatment, such as special education, speech therapy, and social services [1,2]. However, it is clear that early detection is the key to restoring normal speech development and a favorable long-term outcome [3,4].

Definitive diagnosis of hearing loss is typically made through audiologic assessment of pure-tone air, bone and speech thresholds. Traditionally, pure-tone thresholds are documented by asking the subject if they can hear tones of varying frequency and intensity. While auditory brainstem reflex and otoacoustic emissions testing remain the gold standard of hearing assessment, these investigations pose some difficulty in children. Currently, standard clinical audiometry is performed using non-portable hardware and access is therefore limited in developing countries, where hearing loss is more prevalent [5].

Automation of pure-tone audiometry offers several potential benefits [6]. Particularly, a portable, automated audiometer improves accessibility, creating the possibility of routine pure-tone audiometry in the primary care setting or remote communities [6][7][8][9][10][11]. Such a device may eventually permit a parent or teacher access to screening audiometry, resulting in earlier detection of hearing impairment. However, there may be significant differences between mean hearing thresholds determined though automated audiometry and manual audiometry [12][13][14]. As such, clinical validation of any novel automated audiometer is a necessity.

Pure-tone audiometry, regardless of automation or lack thereof, is a mundane task. It is especially challenging to perform in the pediatric population where short attention span and the extent of cognitive development can be limiting factors. Children with hearing impairment may find audiometric testing even more difficult. Several adaptations of pure-tone audiometry are used to evaluate hearing in children. These techniques include behavioral observation, visual re-enforcement, and conditioned play audiometry. While more successful than conventional pure-tone audiometry, these adaptations are resource intensive and typically require two specially trained audiologists to administer. Naturally, an ideal solution would capitalize on the advantages of automation, while maintaining clinical validity and age appropriateness.

With the success of the Apple W iPad W (Apple Inc., Cupertino, CA), it has become evident that touch interface computing enables even the youngest of users to interact intuitively with complex systems. It is not surprising then that portable tablet computers have been widely embraced in healthcare and education [15][16][17][18][19]. Thus, the tablet operating system appears to the optimal environment in which to develop portable, automated diagnostic applications targeted at children. Our group has developed the first such pediatric tablet-based play audiometer [20].

The purpose of this study is to (1) determine the accuracy of audiometric thresholds obtained by tablet audiometry compared to the accepted standard, conditioned play audiometry in children; and (2) describe our experiences with administering an iPad-based hearing assessment in the pediatric population.

We expect children ages 3 and above to be capable of completing a hearing assessment using the tablet audiometer. Further, we hypothesize that the tablet audiometer will be a sensitive screening test and that the audiometric thresholds obtained will closely correlate with conventional audiometric evaluation.

Methods

Tablet audiometer

Interactive audiometry is a novel paradigm in the assessment of hearing thresholds, in which the patient controls the presentation and pace of sound stimuli, rather than the audiologist. The tablet play audiometer is engineered as a decision-tree interactive game (yes/no paradigm) or as a two alternative forced choice interaction [20]. The two-alternative forced choice test paradigm is a psychophysical method for eliciting responses from a person about his or her experiences of a stimulus whereby the user is only given two possible responses. No option for "I don't know" or "not sure" is given [21][22][23][24][25].

In this test paradigm, the user is sequentially presented with a series of objects (eggs, for instance) and is tasked to categorize the objects into 'sound-producing' or 'silent' by dragging the objects into one of two 'containers' (i.e. chicken coop or egg carton) ( Figure 1). In this "game", the user is essentially navigating his/her own test and responding in a yes/no fashion to each stimulus. In another simplified version of the game, the child is only given one container and is tasked to drag the object into this container when it produces a sound. Each soundproducing object plays a unique warble-tone at either 500, 1000, 2000 or 4000 Hz. The intensity of sound decreases with each presentation until the child is unable to reproducibly sort the objects. Subsequently, the intensity of the sounds is increased. Thresholds are determined by the Hughson-Westlake method [26] in an up/down fashion to bracket the true threshold. Several "silent eggs" are also presented randomly as a measure of internal consistency (henceforth referred to as reliability). Once all frequencies are tested, a standard audiogram is obtained ( Figure 2).

Study population

Study participants were recruited from patients presenting to the Audiology Clinic at the Children's Hospital of Eastern Ontario. The inclusion criteria for this study comprised children at least 3 years of age with normal or abnormal hearing. The exclusion criteria for the study included patients with previously diagnosed visual impairment, learning disability, or developmental delay. These patients were identified by a staff otolaryngologist (MB) through review of their patient records prior to enrollment. If the patient was eligible, informed consent was obtained from the parent/guardian. We aimed to enroll 80 patients into the study.

Study design

A prospective, randomized study design was employed. Participants completed two sequential audiometric evaluations in a double walled sound booth, one with the tablet audiometer and one with traditional play audiometry assessing warble-tone thresholds. The order in which participants performed the two assessments was determined randomly by the flip of a coin. For the sake of simplicity and speed, unmasked air conduction was the sole investigation in this study. Sound field testing was performed if the child was not amenable to wearing headphones and binaural testing was performed in all other children. Stimuli were presented using TDH-39 headphones in both the tablet and audiologist test scenarios. When sound field testing was employed, the sound booth speaker system or the tablet speaker were used. An audiologist accompanied participants in both groups to provide motivation during the assessment. This study was approved by the Children's Hospital of Eastern Ontario Research Ethics Board #11/10x.

Calibration

The tablet computers, headphones and speakers that were used in this study were professionally calibrated by Génie Audio Inc. (St-Laurent, QC) to ANSI S3.6-2004 standards [27].

Sample size

Given the novel nature of both the method and the testing hardware no prior studies were available from which to calculate a sample size. A sample size of 80 consecutive patients was selected based on related publications comparing automated hearing tests to standard audiometry [7,10,11]. Five additional children were recruited to allow for possible subject dropout.

Outcome measures

The primary outcome measure consisted of warble-tone thresholds obtained by both the tablet audiometer and by standard play audiometry. Normal hearing was defined as a threshold of less than or equal to 25 dB in each of the 4 test frequencies (500, 1000, 2000 or 4000 Hz). Hearing loss was defined as audiometric thresholds greater than 25 dB in any of 4 test frequencies. Secondary outcomes related to participant performance were also documented, including time to completion as well as feedback from the audiologists. In the tablet group, test reliability was

Statistical analysis

Participants were excluded from analysis for (1) technical/ gameplay issues (2) behavioural issues, or (3) questionable reliability, defined as incorrectly assigning greater than 50% of the silent objects. Descriptive statistics were used to summarize participant characteristics. Student's t-test with Bonferroni correction was used to compare secondary outcomes between the two groups. Preliminary evaluation of the tablet's performance was determined using a two-by-two table and calculations of sensitivity, specificity, positive and negative predictive value, and likelihood ratios positive were calculated.

A more intricate analysis of the concordance between the two hearing assessments was also performed. In order to test for overall differences between modalities (tablet versus play audiometry), a repeated measures model for the detection threshold in each ear at each frequency was fitted using linear mixed effects modeling. The model included fixed effects for testing modality (tablet versus play audiometry), frequency, and ear. In order to account for the correlation of thresholds within participants, random effects modeling was used for frequency nested within ear nested within participant.

Results

85 consecutive patients who met the inclusion criteria were recruited into the study. 15 patients were excluded after hearing assessment, resulting in 70 subjects available for analysis. Fourteen of the 70 patients were tested using a sound field. The remainder (56) had binaural assessments for both hearing evaluations. The mean age of study participants was 5.2 years (range 3-13). The demographics for these individuals are summarized in Table 1.

Fifteen subjects were excluded from the analysis. Four were excluded from analysis due to poor reliability (<50%). Four patients were excluded for behavioural issues that prevented successful completion of the one or both hearing assessments. Seven patients were excluded for technical issues related to test administration. Of note, 10 of 11 subjects in the latter two categories had some degree of hearing loss. These results are summarized in Table 2.

Overall, 55 patients were identified to have normal hearing by conditioned play audiometry, the traditionally accepted standard test. Of these, 52 were found to have normal hearing by tablet audiometry. The 3 remaining children scored slightly outside the defined parameters of normal hearing, namely a 30 dB threshold, in at least one frequency. This appeared to either be due to the child moving through the game too quickly or the presentation timing out before the child could make a decision.

The mean time to complete a binaural hearing test with the tablet was 152 s (SD±114 s) overall. In the group with normal hearing, the time was 109 s (SD±65 s). This increased to 317 s (SD±113 s) in children with abnormal hearing. The difference in test duration was statistically significant (p<0.0001). 53 patients in total had normal evaluation by tablet audiometry. One of these children was found to have a true mild hearing loss. This child appeared to understand the game but their results did not correlate well. The reliability score of this patient when using the table was only 75%. However he did not meet reliability criteria for exclusion (<50%). The results of the 2×2 table are summarized in Table 3.

For the repeated measures analysis, 56 participants with valid results and binaural testing were available. Patients who were tested with a sound field were excluded from this particular analysis, as ear specific information was not available, making analysis impossible.

The model showed no significant effect of testing modality (compared to audiologists, the mean tablet threshold was 0.21 dB lower (95% CI=0.18 to 0.6 dB, p=0.29). The modeling was repeated for the abnormal cases (n=15), and for the normal cases (n=41). In both cases, the model showed no significant effect of testing modality. Compared to play audiometry, the mean tablet threshold was 1.13 dB lower (95% CI=0.27 to 2.52 dB, p=0.12) and 0.12 dB higher (95% CI=0.02 to 0.27 dB, p=0.10), respectively.

Discussion

This paper describes the first trial of both a novel play algorithm using interactive audiometry and a new tablet audiometer. It is, to our knowledge, the first tablet-based, semi-automated, play audiometer to be used in a pediatric setting. The purpose of this study was two-fold, to validate the tablet audiometer as a child-friendly application for hearing assessment, as well as to compare tablet thresholds to the traditionally accepted standard play audiometry. The data reveal that the tablet audiometer produces warble-tone thresholds that are in agreement with the accepted standard (traditional play audiometry). This was achieved with narrow confidence intervals, suggesting sufficient statistical power. Audiometric data are acquired in an efficient manner, as demonstrated by a mean test duration of approximately 2.5 minutes. Moreover, the data reveals a high specificity 94.5% with a negative predictive value (NPV) of 98.1%, denoting that tablet audiometry is a robust screening tool. A positive likelihood ratio of 17.1 confirms the tablet audiometer's capacity to diagnose hearing loss ( Figure 3).

As the tablet audiometer is by definition an objective test, if calibrated adequately, it is not likely to be subject to issues with inter-rater reliability. However, it will be prudent in subsequent investigations to ensure strong inter-rater and test-retest reliability.

Conditioned play audiometry can often be employed with children as young as 2 years of age. The supervision and motivation given to the child by a second audiologist in the sound booth allows these very young children to be tested. In the current study, this second audiologist was present for both assessments (tablet and traditional) to maximize the successful completion of assessments,   although attaining the appropriate level of support from the audiologists required training and experience. Our data suggests that using the tablet interactive audiometry method, the majority (82%, 70/85) of children as young as 3 years of age are capable of understanding the concept of the game and completing the hearing assessment. In fact, 4 of the 15 patients were unable to complete the assessment due to a technical issue related to the audiologist. Therefore, 82% is a somewhat conservative assessment of the user-friendliness of the tablet audiometer. Despite the supervision of an audiologist however, some children had difficulties with the tablet audiometer. These difficulties resulted from technical deficiencies of the hardware/software (i.e. attempts to open other software, failure to understand 'drag and drop' , becoming distracted by visual re-enforcements) and behaviors of the patient (i.e. boredom, poor comprehension of the game). A number of these difficulties eventually lead to the subject being excluded from the statistical analysis. Failure to complete the assessment also appeared to be more prevalent in children with abnormal hearing.

Several technical/gameplay issues were documented during data collection. In particular, children showed signs of fatigue with either test method quite quickly. During standard play audiometry, audiologists often switched games several times during standard play audiometry to keep the child engaged. By contrast, only two games were available when using the tablet, with the current software version. This stresses the importance of maintaining attention in this particular age group.

Furthermore, due to the nature of interactive audiometry, whereby the test is user-directed, action is required at each point in the decision tree. This gives the appearance of more decisions as compared to standard audiometry, where users who did not hear a sound were not required to perform an action. This was exacerbated in children with hearing loss, who were required to sort more objects in order to determine exact thresholds. For example, when testing a normal hearing individual, the minimum number of objects to complete an entire assessment was 16. This number increased to a maximum of 113 when hearing loss was present or unreliable results were being obtained. The average number of objects presented in in hearing loss was 67.1 (SD±19.1) ( Table 1). For children with normal hearing, the average was 23.8 (SD±9.45) (p<0.0001). Despite these challenges, the vast majority of children were engaged enough to complete the tablet hearing assessment.

Some younger children were found to have difficulty understanding the concept of sorting. A simplified version of the game was also developed, where the child was only presented with one object and one container. The child placed the object in the container when it produced sound, and a timeout function advanced the game if the child did not. We found this allowed younger children to be accurately tested with the tablet audiometer.

Visual re-enforcements were originally included. Specifically, if the child sorted an object correctly, a pop-up star appeared. However, children with abnormal hearing tended to focus on these visual cues, ignoring the auditory cues ( Table 2). Subsequent versions of the tablet game will optimize visual cues to maintain the user's interest without distracting them.

The authors acknowledge several limitations to this study. Firstly, the majority of patients were normal hearing children. This is simply a reflection of the patient population when conducting sequential recruitment. However, a test population that is predominantly normal hearing will bias the study toward good correlation and successful completion of the relatively shorter hearing test. Second, the methods of our analysis excluded patients who were identified as having clear difficulty with the hearing assessment. This was done to ensure that the results reflect only the performance of the hardware. Although exclusion of 'difficult' patients limits the generalizability of our results, this analysis was deliberately used during this hardware validation phase. The proportion of patients excluded from analysis (18%, 15/85) insinuates a high degree of user-friendliness, especially given the potentially difficult patient population. Additionally, this emphasizes the importance of audiologist supervision, as the software is currently unable to determine if a child fully comprehends the game. Further gameplay refinement will likely increase the number of patients who are suitable candidates for tablet audiometry.

Conclusion

The goal of our research is to develop a portable, versatile clinical audiometer designed for children. This study is the first to validate an automated play audiometer designed specifically for the pediatric population. In this study, we demonstrated that air conduction thresholds obtained by tablet audiometer are not significantly different from those obtained by standard play audiometry. The device has a strong predictive value for normal hearing and is highly sensitive for hearing loss. Finally, our results demonstrated that the tablet audiometer is a viable platform for testing children as young as 3 years of age. Future directions will focus on gameplay refinement and further validation testing of air vs. bone conduction thresholds and test-retest reliability.

organization. MB is the senior author and involved in all levels of study design, data analysis and manuscript preparation. The manuscript was read and approved by all authors.

Figure 1
1Tablet audiometer gameplay screenshot.

Figure 2
2Tablet audiogram output.

Figure 3
3Receiver operating characteristic curve for tablet audiometer. Plot of the true positive rate (TPR) versus false positive rate (FPR) for varying definitions of normal hearing. The maximal TPR and minimal FPR are achieved at 25 dB. This demonstrates a high sensitivity and specificity for hearing loss.

Table 1
1Participant demographicsNormal hearing
Abnormal hearing

n
5 5
1 5

Mean age (years)
5.06 (range 3-9)
5.81 (range 3-13)

Duration of tablet
assessment (secs)*

109±65
317±113

Number of objects*
23.8±9.45
67.1±19.1

Reliability (%)
90.4±22.4
92.0±10.8

± denotes standard deviation.
* denotes statistically significant difference.
Duration and number of objects for binaural assessment only.

Table 2
2Excluded participants and rationaleStudy ID
Age (yrs)
Normal hearing?
Reason for exclusion

Poor reliability

2-007
4
Y
Reliability <50%

2-038
4
Y
Reliability <50%

2-043
4
Y
Reliability <50%

3-023
5
Y
Reliability <50%

Technical/Gameplay issues

2-042
3
N
Incorrect speaker setting

2-060
7
N
Relied on visual cues

2-063
4
N
Poor headphone placement

2-066
4
N
Performed masked conditioned play audiometry

3-019
12
N
Performed masked conditioned play audiometry

2-050
3
N
Did not understand drag and drop

2-044
4
N
Did not understand drag and drop

Behavioural issues

2-071
4
Y
Poor attention span

2-092
10
N
Poor attention span

2-108
4
N
Very slow. Poor attention span

2-011
3
N
Appeared to put all objects into one container

Table 3
3Comparison of tablet audiometer and
conventional play audiometry

Play audiometry

Abnormal
hearing

Normal
hearing
Tablet audiometry

Abnormal hearing
14
3

Normal hearing
1
5 2

Sensitivity: 93.3% (95% CI = 71.7-99.6%).
Specificity: 94.5% (95% CI = 88.6-96.3%).
Positive predictive value: 82.3% (95% CI = 63.3-87.9%).
Negative predictive value: 98.1% (95% CI = 92.0-99.9%).
Positive likelihood ratio: 17.1 (95% CI = 6.31-26.7).

http://www.journalotohns.com/content/42/1/21
Competing interestsAt the time of research the authors had no conflicts of interest to disclose, however the idea has since been patented.Authors' contributions JY was responsible for study design, statistical analysis and manuscript preparation. HJ was responsible for study design. SH, YB and SC were responsible for administration of audiometric testing and data collection/
Redefining the survival of the fittest: communication disorders in the 21st century. R J Ruben, Laryngoscope. 1102Pt 1Ruben RJ: Redefining the survival of the fittest: communication disorders in the 21st century. Laryngoscope 2000, 110(2 Pt 1):241-245.

The societal costs of severe to profound hearing loss in the United States. P E Mohr, J J Feldman, J L Dunbar, Int J Technol Assess Health Care. 164Mohr PE, Feldman JJ, Dunbar JL, et al: The societal costs of severe to profound hearing loss in the United States. Int J Technol Assess Health Care 2000, 16(4):1120-1135.

Language of early-and later-identified children with hearing loss. C Yoshinaga-Itano, A L Sedey, D K Coulter, A L Mehl, Pediatrics. 1025Yoshinaga-Itano C, Sedey AL, Coulter DK, Mehl AL: Language of early-and later-identified children with hearing loss. Pediatrics 1998, 102(5):1161-1171.

Early intervention and language development in children who are deaf and hard of hearing. M P Moeller, Pediatrics. 106343Moeller MP: Early intervention and language development in children who are deaf and hard of hearing. Pediatrics 2000, 106(3):E43.

A summary of the literature on global hearing impairment: current status and priorities for action. D Tucci, M H Merson, B S Wilson, Otology & neurotology: official publication of the American Otological Society, American Neurotology Society. 31Tucci D, Merson MH, Wilson BS: A summary of the literature on global hearing impairment: current status and priorities for action. Otology & neurotology: official publication of the American Otological Society, American Neurotology Society [and] European Academy of Otology and Neurotology 2010, 31(1):31-41.

Automated pure-tone audiometry: an analysis of capacity, need, and benefit. R H Margolis, D E Morgan, Am J Audiol. 172Margolis RH, Morgan DE: Automated pure-tone audiometry: an analysis of capacity, need, and benefit. Am J Audiol 2008, 17(2):109-113.

AMTAS: automated method for testing auditory sensitivity: validation studies. R H Margolis, B R Glasberg, S Creeke, Bcj Moore, Int J Audiol. 493Margolis RH, Glasberg BR, Creeke S, Moore BCJ: AMTAS: automated method for testing auditory sensitivity: validation studies. Int J Audiol 2010, 49(3):185-194.

AMTAS( W ): automated method for testing auditory sensitivity: III. sensorineural hearing loss and air-bone gaps. R H Margolis, Bcj Moore, Int J Audiol. 507Margolis RH, Moore BCJ: AMTAS( W ): automated method for testing auditory sensitivity: III. sensorineural hearing loss and air-bone gaps. Int J Audiol 2011, 50(7):440-447.

AMTAS W : Automated method for testing auditory sensitivity: II. Air conduction audiograms in children and adults. R H Margolis, R Frisina, J P Walton, Int J Audiol. 507Margolis RH, Frisina R, Walton JP: AMTAS W : Automated method for testing auditory sensitivity: II. Air conduction audiograms in children and adults. Int J Audiol 2011, 50(7):434-439.

Hearing screening for school children: comparison of low-cost, computer-based and conventional audiometry. Child Care Health Dev. B Mcpherson, Mms Law, Msm Wong, 36McPherson B, Law MMS, Wong MSM: Hearing screening for school children: comparison of low-cost, computer-based and conventional audiometry. Child Care Health Dev 2010, 36(3):323-331.

An audiometer to monitor progressive hearing change in school-aged children. W H Liao, S T Young, C F Lien, S J Wang, J Med Screen. 181Liao WH, Young ST, Lien CF, Wang SJ: An audiometer to monitor progressive hearing change in school-aged children. J Med Screen 2011, 18(1):8-11.

Reliability of computerautomated hearing thresholds in cochlear-impaired listeners using ER-4B Canal Phone earphones. J A Henry, C L Flick, A Gilbert, R M Ellingson, S A Fausti, J Rehabil Res Dev. 403Henry JA, Flick CL, Gilbert A, Ellingson RM, Fausti SA: Reliability of computer- automated hearing thresholds in cochlear-impaired listeners using ER-4B Canal Phone earphones. J Rehabil Res Dev 2003, 40(3):253-264.

Reliability of hearing thresholds: computer-automated testing with ER-4B Canal Phone earphones. J A Henry, C L Flick, A Gilbert, R M Ellingson, S A Fausti, J Rehabil Res Dev. 385Henry JA, Flick CL, Gilbert A, Ellingson RM, Fausti SA: Reliability of hearing thresholds: computer-automated testing with ER-4B Canal Phone earphones. J Rehabil Res Dev 2001, 38(5):567-581.

Evaluation of a maximum likelihood procedure for measuring pure-tone thresholds under computer control. C Formby, L P Sherlock, D M Green, J Am Acad Audiol. 72Formby C, Sherlock LP, Green DM: Evaluation of a maximum likelihood procedure for measuring pure-tone thresholds under computer control. J Am Acad Audiol 1996, 7(2):125-129.

Scalpel, stethoscope, iPad: the future is now in the intensive care unit. R H Savel, C L Munro, Am J Crit Care. 204Savel RH, Munro CL: Scalpel, stethoscope, iPad: the future is now in the intensive care unit. Am J Crit Care 2011, 20(4):275-277.

Emergency CT brain: preliminary interpretation with a tablet device: image quality and diagnostic performance of the Apple iPad. P Mc Laughlin, S O Neill, N Fanning, Emerg Radiol. 192Mc Laughlin P, Neill SO, Fanning N, et al: Emergency CT brain: preliminary interpretation with a tablet device: image quality and diagnostic performance of the Apple iPad. Emerg Radiol 2012, 19(2):127-133.

A novel method for sterile intra-operative iPad use. A D Murphy, Hjcr Belcher, J Plastic, Reconstructive & Aesthetic Surgery. 20123JPRASMurphy AD, Belcher HJCR: A novel method for sterile intra-operative iPad use. J Plastic, Reconstructive & Aesthetic Surgery: JPRAS 2012, 65(3):403-404.

Math that moves: schools embrace the iPad. W Hu, New York Times. 16Hu W: Math that moves: schools embrace the iPad. New York City: New York Times; 2011:A16.

10.1080/07380561003801590Banister S: Integrating the iPod Touch in K-12 Education: Visions and Vices. Banister S: Integrating the iPod Touch in K-12 Education: Visions and Vices. http://dx.doi.org.proxy2.lib.uwo.ca:2048/10.1080/07380561003801590.

A New iPad application for Hearing screening in children. N Ellaham, Y Yilma, Jourdan G Bromwich, M , In pressEllaham N, Yilma Y, Jourdan G, Bromwich M: A New iPad application for Hearing screening in children. In press.

Further studies of a maximum-likelihood yes-no procedure. X Gu, D M Green, J Acoust Soc Am. 961Gu X, Green DM: Further studies of a maximum-likelihood yes-no procedure. J Acoust Soc Am 1994, 96(1):93-101.

A comparison of psychophysical procedures for level-discrimination thresholds. P Marvit, M Florentine, S Buus, J Acoust Soc Am. 1136Marvit P, Florentine M, Buus S: A comparison of psychophysical procedures for level-discrimination thresholds. J Acoust Soc Am 2003, 113(6):3348-3361.

Two-alternative versus three-alternative procedures for threshold estimation. B R Shelton, I Scarrow, Percept Psychophys. 354Shelton BR, Scarrow I: Two-alternative versus three-alternative procedures for threshold estimation. Percept Psychophys 1984, 35(4):385-392.

Nonparametric relationships between single-interval and two-interval forced-choice tasks in the Theory of Signal Detectability. J Miller, B Scurfield, V Drga, S Galvin, J Whitmore, J Math Psychol. 464Miller J, Scurfield B, Drga V, Galvin S, Whitmore J: Nonparametric relationships between single-interval and two-interval forced-choice tasks in the Theory of Signal Detectability. J Math Psychol 2002, 46(4):383-417.

Hearing: An introduction to psychological and physiological acoustics. S Gelfand, New York: Marcel Dekker3rd editionGelfand S: Hearing: An introduction to psychological and physiological acoustics. 3rd edition. New York: Marcel Dekker; 1998.

Manual for program outline for rehabilitation of aural casualties both military and civilian. W Westlake, Hd H, Trans Am Acad Ophthalmol Otolaryngology. 48SuppWestlake W, HD H: Manual for program outline for rehabilitation of aural casualties both military and civilian. Trans Am Acad Ophthalmol Otolaryngology 1944, 48(Supp):1-15.

American National Standards Institute: Specification for audiometers (ANSI S3.6-2004). 10.1186/1916-0216-42-21New York: American National Standards InstituteAmerican National Standards Institute: Specification for audiometers (ANSI S3.6-2004). New York: American National Standards Institute; 2004. doi:10.1186/1916-0216-42-21

Submit your next manuscript to BioMed Central and take full advantage of: • Convenient online submission • Thorough peer review • No space constraints or color figure charges • Immediate publication on acceptance • Inclusion in PubMed, CAS, Scopus and Google Scholar • Research which is freely available for redistribution. Yeung, Journal of Otolaryngology -Head and Neck Surgery. 4221The new age of play audiometry: prospective validation testing of an iPad-based play audiometer. Submit your manuscript at www.biomedcentral.com/submitCite this article as: Yeung et al.: The new age of play audiometry: prospective validation testing of an iPad-based play audiometer. Journal of Otolaryngology -Head and Neck Surgery 2013 42:21. Submit your next manuscript to BioMed Central and take full advantage of: • Convenient online submission • Thorough peer review • No space constraints or color figure charges • Immediate publication on acceptance • Inclusion in PubMed, CAS, Scopus and Google Scholar • Research which is freely available for redistribution Submit your manuscript at www.biomedcentral.com/submit


Economic Evaluation of Severe Anaemia: Review-Based Recommendations and a Conceptual Framework
September 2020 • EMJ

Dimitrios Tomaras
Purple Squirrel Economics
MontrealCanada

William Lemay
Purple Squirrel Economics
MontrealCanada

Gabriel Tremblay
Purple Squirrel Economics
MontrealCanada

Economic Evaluation of Severe Anaemia: Review-Based Recommendations and a Conceptual Framework
September 2020 • EMJReceived: 16.04.20 Accepted: 08.06.20Creative Commons Attribution-Non Commercial 4.0 45 Authors: Disclosure: The authors have declared no conflicts of interest.Anaemiaconceptual frameworkred blood cell (RBC) disorders Citation: EMJ 2020;5[3]:45-54
Objective: Novel anaemia treatments have greatly improved patient outcomes in the last decade and have also undergone economic evaluations in various settings using heterogenous model structures, costs, and inputs. The objectives were to review published economic evaluation studies in major red blood cell disorders, identify limitations in the applied methodology, provide a set of recommendations, and produce a conceptual framework for future economic research in this disease area.Methods: A targeted search was conducted for economic literature evaluating treatments in major red blood cell disorders related to anaemia. Disorders included autoimmune haemolytic anaemia, β-thalassaemia, chemotherapy-induced anaemia, anaemia in chronic kidney disease, and severe aplastic anaemia. Budget impact models and cost-effectiveness and cost-utility analyses were considered. Modelling assumptions regarding the model structure, time horizon, perspective, and type of costs were reviewed and recommendations and a conceptual framework for future economic analyses were created.Results: A total of four budget impact models, nine cost-utility analyses, and four cost-effectiveness analyses were investigated. A major limitation was that the included costs varied significantly across studies. Costs which were rarely included, and generally should be considered, were related to adverse events, mortality, and productivity. Additionally, relationships between levels of serum ferritin, hepatic or total body iron, and haemoglobin with long-term complications and mortality were rarely included.Conclusion: Published economic analyses evaluating treatments for major red blood cell disorders frequently exclude vital costs. A set of recommendations and a conceptual framework will aid researchers in applying a more comprehensive approach for economic evaluations in major red blood cell disorders.

INTRODUCTION

Anaemia is characterised by a decrease in the total amount of red blood cells (RBC) and haemoglobin (Hb) levels, reducing the volume of oxygen transported through the blood. 1 Several varieties of anaemia exist, and the disease pathology can be explained either as a disease-related complication (e.g., anaemia of inflammation) or as a general hereditary condition. 2,3 Anaemia classifications and conditions leading to severe anaemia include autoimmune haemolytic anaemia (AIHA), β-thalassaemia, chemotherapy-induced anaemia, anaemia in chronic kidney disease (CKD), and severe aplastic anaemia (SAA); many of which are associated with early mortality.

RBC transfusions were frequently used as a standard of care in the treatment of anaemia; however, regular RBC transfusions are accompanied with serious health risks. 4 Iron overloading, related to regular RBC transfusions, can lead to fatal complications including organ failure. One class of alternative therapies, erythropoiesis-stimulating agents (ESA), received U.S. Food and Drug Administration (FDA) approval in 1989 for anaemia related to chronic renal failure. 5 Subsequently, a combination of ESA and RBC transfusions are generally used to treat severe anaemias. Recently, new treatment strategies have been undergoing clinical development with several novel therapies gaining regulatory approval. Novel therapies are also accompanied with high costs, which can act as potential barriers to patient access. Economic evaluations aid decision makers to assess whether the reimbursement of novel therapies would be an efficient use of limited healthcare resources and if they can be afforded with current healthcare budgets.

The purpose of this review is to identify relevant literature and explore the budgetary impacts of novel treatments for severe anaemia and whether they were found to be cost-effective. The benefit of reviewing economic literature is two-fold. Firstly, standardised cost-effectiveness analyses (CAE) in severe anaemias will allow clinicians to consider lifetime clinical outcomes, which may not be adequately captured in clinical trials, the length of which is generally limited to 1-2 years. Reviewing standardised economic evaluations would help clinicians consider long-term health implications in their decision-making process. Secondly, as novel therapies are generally very costly, it is important for clinicians to consider whether the added clinical benefit of a therapy justifies the incremental cost in settings with limited healthcare resources. Standardised economic evaluations in this disease area will allow clinicians to compare various treatments across analyses and aid decision-making while considering budgetary implications.

To the authors' knowledge, no manuscript investigating the evolution of severe anaemia treatment from an economic perspective has been published. As severe anaemia is generally a chronic disease, careful economic modelling should be employed to appropriately capture long-term disease implications.

METHODS

A targeted literature review was conducted via PubMed Central (PMC), Google Scholar, and OVID. The disease area of interest included any type of severe anaemia evaluated in a published comprehensive economic analysis. Although stem cell transplant treatments are important in this disease area, they are significantly different from ongoing, continuous therapies, which are the focus of this analysis. Differences such as cost structures and cure potential result in different designs and, for this reason, stem cell transplants are excluded from this analysis. No other limits on the type of treatments were employed. Search terms included a combination of a disease term with a study design term. Disease terms included "anemia", "beta-thalassemia", "autoimmune hemolytic anemia", "chemotherapy-induced anemia", "sickle cell disease", "chronic kidney disease anemia", and "severe aplastic anemia". Study design terms included "cost-effectiveness analysis", "cost-utility analysis", and "budget impact model". Any economic outcome was included. Additional limits included studies published in English and studies published between 2010 and 2020. Subsequently, the methodology of identified studies was critically assessed. Recommendations for future economic evaluations were developed based on a critical assessment. Additionally, a conceptual framework was created to aid future economic analyses, building on past studies' strengths, and identifying gaps to be avoided in future research.

RESULTS

Overview

A total of 18 studies were identified which met the inclusion/exclusion criteria. The following elements were explored for each study: disease area, study design and perspective, treatments, time horizon, costs, measures of uncertainty, as well as the model structure and health states. A summary of studies is presented in Table 1. [6][7][8][9][10][11][12][13][14][15][16][17][18][19][20][21][22][23] Diseases Seven studies evaluated therapy for β-thalassaemia, one for AIHA, one for chemotherapy-induced anaemia, one for chronic heart failure-related anaemia, two for SAA, and six for CKD-related anaemia.

Study Design and Perspective

Four studies were budget impact models (BIM), four were CEA, and 10 were cost-utility analyses (CUA). From the 18 studies identified, two were Canadian, three were from the USA, two from Australia, two from Italy, and one from the UK, Thailand, China, Iran, Poland, Germany, South Korea, and Morocco, respectively. A total of 16 studies used a healthcare system perspective while two others used a societal perspective.   were related to the usage of ESA for patients on or without dialysis. For the chronic heart failure-related iron-deficient anaemia, iron supplementation was evaluated against placebo.

Time Horizon

Time horizons identified across studies ranged from 15 weeks to a lifetime. Shorter time horizons (15 weeks to 5 years) were employed in ten studies (56%) while two used 50 year-horizons (11%), and a lifetime horizon was used in six (33%). As chemotherapy-induced anaemia is a non-chronic condition, a 15-week horizon was applied. Excluding chemotherapy-induced anaemia, and chronic heart failure-related anaemia, time horizons ranged from 1 year to a lifetime.

Costs

All studies included drug costs (100%), 12 studies included administration costs (67%), and seven studies included healthcare resource utilisation costs (39%). Indirect costs were frequently missing from analyses, specifically, adverse events, mortality, and productivity costs. They were included in five (28%), two , (11%), and one (6%) studies, respectively. Administration costs were excluded in two biosimilar studies and biosimilar studies also excluded indirect costs, which were assumed to be similar for both treatment arms.

Uncertainty

A total of 15 studies (83%) included sensitivity analyses, which assessed stochastic and deterministic uncertainty by varying inputs and testing model assumptions. Two BIM and one CUA did not report any assessment of uncertainty.

Model Structure and Health States

The model structure varied greatly across different studies. For CEA and CUA, the model structure was reported as Markov models for nine studies (50%), microsimulations for two studies (11%), and unreported for three studies (17%). Four additional studies were BIM (24%). The number of health states varied from three to seven across diseases.

DISCUSSION

Overview and Conceptual Framework

Economic evaluations in severe anaemia were reviewed with the objective of creating recommendations for individual components of future economic analyses. The lack of economic-focussed guidelines for severe anaemia demonstrates a gap in the literature. Conceptual frameworks have been developed in multiple other therapeutic areas including immune thrombocytopenia 24 and acute coronary syndrome. 25 As anaemia is generally a chronic condition with long-term implications, it is vital to consider if short-term trial endpoints can accurately be extrapolated to long-term horizons and outcomes. Moreover, structural heterogeneity across economic evaluations can hinder cross-study comparisons. A validated conceptual economic framework used in future research could facilitate easier comparisons. As long-term complications are associated with severe anaemia and standard of care treatments, relevant biomarkers such as levels of Hb and serum ferritin should be included as proxies to quantify the long-term risk of complications.

A general conceptual framework was developed for CEA and CUA in severe anaemia. It is possible that different disease-specific considerations and economic questions would require a tailoring of the conceptual framework. However, the one presented in Figure 1 has been designed to accurately capture the implications of chronic anaemia and related RBC transfusions, including long-term complications related to iron overloading (serum ferritin proxy) and reduced or elevated Hb, treatment-related adverse events, key costs, key efficacy measures, and important endpoints. The conceptual framework may help clinicians and formulary committees evaluate the robustness of economic analyses. This, in turn, could help formulary committees consider which treatments obtain formulary coverage and subsequently affect the clinician's ability to prescribe these treatments. Additionally, standardised economic study designs could help clinicians evaluate the long-term effectiveness of therapies across different analyses and aid clinical decision-making. Additional components requiring further consideration are summarised in the following subsections.

Treatment Comparisons

As economic evaluations are comparative by nature, it is important to specify which techniques or trial designs were used to compare treatments and to report the source of data. Typically, trial-based data should be utilised as a source of clinical inputs for novel therapies. In cases where multiple trials are used as data sources, an indirect treatment comparison (ITC) should be considered. Naïve, unadjusted ITC could lead to bias because of differing baseline patient characteristics and trial heterogeneity. If the baseline characteristics between treatment arms are imbalanced, an ITC should be conducted. Appropriate ITC require individual patient data which is matched for baseline characteristics to another trial population using statistical techniques. Two ITC methods are recommended, including the simulated treatment comparison and the matching-adjusted indirect comparison, whose objectives are to produce a covariateadjusted treatment effect estimate. Propensity scoring and regression form the basis of their methods and both are recommended by the National Institute for Health and Care Excellence (NICE). 26 One important consideration is to assess whether trial data can accurately be extrapolated to long-term outcomes. As this can vary with different types of severe anaemia and trial design, it is recommended to consult a clinical expert. 27

Health States

Transfusion-dependent patients have a higher likelihood of experiencing complications than their non-dependent counterparts, potentially necessitating a greater number of hospitalisations and outpatient visits. 28 Iron overload related to RBC transfusions is associated with long-term cardiac and hepatic infection and endocrine complications. 29 Multiple studies found an association between quality of life and Hb levels in transfusion-dependent patients. [30][31][32] Additionally, numerous studies found that complications related to iron overloading, estimated using serum ferritin, were also associated with reductions in quality of life and utility. [33][34][35] Therefore, levels of serum ferritin and Hb should be accounted for throughout the horizon of the model with elevated levels of serum ferritin and reduced/elevated levels of Hb leading to complications and quality of life decrements. One study showed that patients with lower (9.0-10.9 g/dL) and higher (>12.0 g/dL) Hb levels have increased frequencies of adverse events and early mortality. 20 These differences could be reflected in different health states. If the data permits, multiple health states with various levels of transfusion burden, serum ferritin, and hepatic or total body iron count should be used to account for higher frequency of RBC transfusions and related long-term complications. The evaluated diseases are mostly chronic by nature and long-term horizons should be prioritised. Models should also incorporate the dynamic nature of complications and patient burden, which increase with time. 36

Costs

As anaemia is a chronic condition, costs which arise over an extended period should be considered. Both direct and indirect costs should be included when comparing two different treatments. Several key costs were absent from the literature. Adverse events, long-term complications, mortality, and productivity costs should be included to create a comprehensive analysis. The proposed conceptual framework enables the inclusion of numerous costs including the cost to treat the underlying cause of anaemia (e.g., β-thalassaemia), which can vary between transfusion burden, health states, and anaemia severity. A granular approach between transfusion burden health states (e.g., high versus low) is preferable if data is available, because the annual costs of RBC transfusions can vary with frequency of administration. Anaemia severity may be evaluated through iron concentration or Hb levels, but is economically manifested through the cost of RBC transfusions and other supporting therapies (e.g., iron chelation therapy), adverse events, long-term complications, routine care and testing, early mortality, and indirect costs.

Complications and Adverse Events

Serum ferritin is used to assess iron levels and oxygen levels are used to estimate Hb. Depending on anaemia severity, patients may require frequent RBC transfusions, resulting in long-term iron overloading, long-term complications, and treatment-related adverse events. 37 Examples of anaemia-related long-term complications and treatment-related adverse events include cardiovascular disease, cerebrovascular disease, and infection. 38 Complications and adverse events could lead to elevated costs through more frequent hospitalisations, outpatient visits, and potentially early mortality. Levels of Hb were only directly considered in five economic evaluations (29%). A modelling approach which includes dynamic rates for the risk of long-term complications based on transfusion burden and Hb and serum ferritin levels is recommended. A gap in the published analyses was identified as no economic studies considered the relationship between complications and serum ferritin levels.

In four identified studies, adverse events affected quality of life; however, their costs were not considered. It is recommended that economic evaluations include both cost and quality of life effects for long-term complications and adverse events.

Sensitivity Analysis

All economic evaluations should contain an uncertainty assessment. It is essential to test assumptions and quantify the stochastic effect/ probabilistic and deterministic uncertainty on the results of the analyses. Evaluations without uncertainty assessments are not comprehensive. For BIM, it is recommended to perform one-way deterministic sensitivity analyses which varies inputs individually in various scenarios. The magnitude of variance should be obtained from published literature or from consultation with disease-specific clinical experts. 39 For CEA and CUA, one-way deterministic and probabilistic sensitivity analyses are also recommended. In probabilistic sensitivity analyses, all inputs should be varied simultaneously for 1,000-10,000 iterations to identify a range of incremental cost-effectiveness ratios and to estimate the probability that an incremental cost-effectiveness ratio falls under a willingness-to-pay threshold. The stochasticity of variables is assessed using their standard error and a randomly generated value across specific distributions. Appropriate distributions should be selected based on variable characteristics. Generally, the gamma distribution should be considered for costs, beta or gamma distributions for utility, and log-normal distributions if a skew to the right is needed. Uncertainty analysis is vital to assess robustness. NICE guidelines also recommend exploring the impact of uncertainty on the results of the economic analyses. 40

Limitations

The proposed framework and set of recommendations were specifically designed for economic evaluations related to severe anaemia. It should be noted that disease-specific anaemias, e.g., CKD-related anaemia, could require modellers to adapt and tailor the framework by adding dialysis or kidney transplant pathways or health states, for example. The disease-specific framework may also require further adjustments based on the availability of data for the disease, for example, the availability of an association between health states and mortality. Additionally, although ITC is recommended using individual patient data, it is plausible that future modellers may have limited access to this type of data, potentially impeding the implementation of this recommendation. No economic evaluations were identified for several common anaemias including sickle cell anaemia. Notably, the Institute for Clinical and Economic Review (ICER) also released an evidence report on sickle cell disease in March 2020. The report states that sickle cell disease has been an underfunded area for research; therefore, there is limited literature on the subject. 41 Additionally, it should be noted that, while the analysis is focussed on severe anaemia or conditions related to severe anaemias, anaemia may not be the sole determinant of costs and quality of life. The underlying disease would also have a significant effect. Several treatments targeting anaemia, such as iron chelation therapy, would however, require economic evaluations.

An additional limitation is regarding the validity of serum ferritin as a proxy for iron overload. Other methods, such as T2*-weighted MRI of the liver or other major organs, may capture iron overload more accurately. 42,43 However, as clinical trials would frequently be the source of inputs for economic models, their relatively short durations may not allow variations of iron concentration from T2*-weighted MRI to correlate with patient quality of life, an essential consideration in cost-effectiveness analysis. Serum ferritin levels fluctuate more rapidly relative to T2*-weighted MRI. Another reason serum ferritin is frequently used to assess the level of iron overload is its relatively low cost and ease of implementation compared with other techniques. 44 Weekly or monthly T2*-weighted MRI is unlikely to be recommended, as fluctuations require longer periods of time. MRI is recommended every 6 to 24 months depending on the condition severity; this testing interval can be longer than typical clinical trials. 45

CONCLUSION

The targeted literature review demonstrated numerous gaps in published economic evaluations for severe anaemia-related diseases. By identifying these gaps, a conceptual framework and set of recommendations were created for future economic evaluations in severe anaemia-related diseases. Key considerations for future analyses consist of the inclusion of Hb and iron concentration levels in the model, multiple transfusion-dependent health states (if possible), long-term complications, adverse events, early mortality, and productivity costs. Future economic analyses which use a consistent anaemia-specific framework will allow decision makers to appropriately compare various published studies, critique the robustness of their analyses and ultimately aid them in their reimbursement decision-making process to optimise the use of scarce healthcare resources.

UIFigure 1 :
1Un nd de er rl ly yi in ng g c ca au us se e o of f a an na ae em mi ia a T Tr re ea at tm me en nt t f fo or r u un nd de er rl ly yi in ng g c ca au us se e T Tr ra an ns sf fu us si io on n d de ep pe en nd de en nc ce e* * D Di is sc co on nt ti in nu ua at ti io on n M Mo or rt ta al li it ty y C Co os st ts s • Mortality E Ef ff fe ec ct ti iv ve en ne es ss s In nc cr re em me en nt ta al l c co os st t p pe er r L LY Y I In nc cr re em me en nt ta al l c co os st t p pe er r r re es sp po on nd de ra an ns sf fu us si io on n i in nd de ep pe en nd de en nc ce om mp pl li ic ca at ti io on ns s (costs, QALY/LY) I In nc cr re em me en nt ta al l c co os st t p pe er r Q QA AL LY Y P Pa at ti ie en nt t p po op pu ul la at ti io on n H He ea al lt th h s st ta at te es s a an nd d p pa at th hw wa ay y E En nd dp po oi in nt ts s C Co om mp pl li ic ca at ti io on ns Conceptual framework for cost-effectiveness analysis and cost-utility analysis evaluations of severe anaemia disorders. *Transfusion-dependent health states should be separated by level of transfusion burden when possible (if data permits). This will facilitate a more granular, accurate approach as different levels of transfusion burden area are associated with different costs and quality of life. Colour legend: purple: patient pathway and various health states; blue: costs (and effectiveness as well for complications); grey: measures of effectiveness; yellow: biomarkers which should be included in the model; green: economic evaluation endpoints. AE: adverse event; Hb: haemoglobin; LY: life year; QALY: quality-adjusted life year; SF: serum ferritin.

Table 1 :
1Identified publications. BIM: budget impact model; CEA: cost-effectiveness analysis; CUA: cost-utility analysis; DFX: deferasirox; DFP: deferiprone; DFO: desferrioxiamine; EPAG: eltrombopag; ESA: erythropoiesis-stimulating agent; Hb: haemoglobin; IST: immunosuppressive therapy (ATGAM plus cyclosporine); IV: intravenous; NHS: National Health Service; RBC: red blood cell; SC: subcutaneous.Treatments

For β-thalassaemia, the study treatments
assessed were various chelation therapies
(deferoxamine, deferiprone, deferasirox, and
combination of deferoxamine plus deferiprone).

The AIHA study compared rituximab against a
biosimilar, also comparing subcutaneous versus
intravenous administration. For chemotherapy-
induced anaemia, ESA originators (Aranesp®
[Amgen, Thousand Oaks, California, USA]

[darbepoetin alpha], NeoRecormon [Roche,
Basel, Switzerland] [epoetin beta], and Eprex
[Janssen, Beerse, Belgium] [epoetin alpha]) were
compared to biosimilars (Abseamed [MEDICE,
Iserlohn, Germany] [epoetin alpha], Binocrit®
[Sandoz, Holzkirchen, Germany] [epoetin alpha],
and Retacrit® [Pfizer, New York City, New York,
USA] [epoetin zeta]). For SAA, two identified
studies assessed eltrombopag and ATGAM®
(Pfizer), respectively, against immunosuppressive
therapy. For CKD-related anaemia, treatments

Table 1 continued.
1

Your guide to anemia national hearth lung and blood institute. 31Blood Institute (NHLBI)1. National Heart, Lung, and Blood Institute (NHLBI). Your guide to anemia national hearth lung and blood institute. 2011. Available at: https://www.nhlbi.nih.gov/ health-topics/all-publications-and- resources/your-guide-anemia. Last accessed: 31 March 2020.

The hereditary anemias. R Watson, Bull N Y Acad Med. 302Watson R. The hereditary anemias. Bull N Y Acad Med. 1954;30(2):106-21.

Anemia of inflammation. E Nemeth, T Ganz, Hematol Oncol Clin North Am. 284Nemeth E, Ganz T. Anemia of inflammation. Hematol Oncol Clin North Am. 2014;28(4):671-81.

Anemia and transfusion of red blood cells. Cortés Buelvas, A , Colomb Med (Cali). 444Cortés Buelvas A. Anemia and transfusion of red blood cells. Colomb Med (Cali). 2013;44(4):236-42.

Food and Drug Administration (FDA). U S , Information for epogen/ procritU.S. Food and Drug Administration (FDA). Information for epogen/ procrit. 2018. Available at: https:// www.fda.gov/drugs/postmarket- drug-safety-information-patients- and-providers/information- epogenprocrit-epoetin-alfa. Last accessed: 7 April 2020.

Cost-utility analysis of deferiprone for the treatment of β-thalassaemia patients with chronic iron overload: a UK perspective. A Bentley, Pharmacoeconomics. 319Bentley A et al. Cost-utility analysis of deferiprone for the treatment of β-thalassaemia patients with chronic iron overload: a UK perspective. Pharmacoeconomics. 2013;31(9):807-22.

Iron-chelating therapies in a transfusion-dependent thalassaemia population in Thailand. N Luangasanatip, Clin Drug Investig. 317Luangasanatip N et al. Iron-chelating therapies in a transfusion-dependent thalassaemia population in Thailand. Clin Drug Investig. 2011;31(7):493-505.

Lifetime cost-utility analyses of deferasirox in betathalassaemia patients with chronic iron overload. J Karnon, Clin Drug Investig. 3212Karnon J et al. Lifetime cost-utility analyses of deferasirox in beta- thalassaemia patients with chronic iron overload. Clin Drug Investig. 2012;32(12):805-15.

A pharmaco-economic evaluation of deferasirox for treating patients with iron overload caused by transfusion-dependent thalassemia in Taiwan. W L Ho, J Formos Med Assoc. 1124Ho WL et al. A pharmaco-economic evaluation of deferasirox for treating patients with iron overload caused by transfusion-dependent thalassemia in Taiwan. J Formos Med Assoc. 2013;112(4):221-9.

Cost-utility analysis of oral deferasirox versus infusional deferoxamine in transfusion-dependent β-thalassemia patients. A Keshtkaran, Transfusion. 538Keshtkaran A et al. Cost-utility analysis of oral deferasirox versus infusional deferoxamine in transfusion-dependent β-thalassemia patients. Transfusion. 2013;53(8):1722-9.

The budget impact analysis of deferasirox for the treatment of iron overload due to frequent blood transfusions in children and adolescents (age ≤18 years). Value Health. J Walczak, 16379Walczak J et al. The budget impact analysis of deferasirox for the treatment of iron overload due to frequent blood transfusions in children and adolescents (age ≤18 years). Value Health. 2013;16(7):A379.

Cost-utility analysis of three iron chelators used in monotherapy for the treatment of chronic iron overload in β-thalassaemia major patients: an Italian perspective. A Pepe, Clin Drug Investig. 375Pepe A et al. Cost-utility analysis of three iron chelators used in monotherapy for the treatment of chronic iron overload in β-thalassaemia major patients: an Italian perspective. Clin Drug Investig. 2017;37(5):453-64.

Budget impact analysis of rituximab biosimilar in Italy from the hospital and payer perspectives. Global & Regional Health Technology Assessment. C Rognoni, 10.1177/2284240318784289Rognoni C et al. Budget impact analysis of rituximab biosimilar in Italy from the hospital and payer perspectives. Global & Regional Health Technology Assessment. 2018; DOI:10.1177/2284240318784289.

Budget impact analysis on erythropoiesis-stimulating agents use for the management of chemotherapy-induced anaemia in Greece. E Nikolaidi, Cost Eff Resour Alloc. 11116Nikolaidi E et al. Budget impact analysis on erythropoiesis-stimulating agents use for the management of chemotherapy-induced anaemia in Greece. Cost Eff Resour Alloc. 2013;11(1):16.

Recurrent acute decompensated heart failure owing to severe iron deficiency anemia caused by inappropriate habitual bloodletting. W H Lim, J Cardiovasc Ultrasound. 234Lim WH et al. Recurrent acute decompensated heart failure owing to severe iron deficiency anemia caused by inappropriate habitual bloodletting. J Cardiovasc Ultrasound. 2015;23(4):253-6.

Budget impact of eltrombopag as first-line treatment for severe aplastic anemia in the United States. G Tremblay, Clinicoecon Outcomes Res. 11Tremblay G et al. Budget impact of eltrombopag as first-line treatment for severe aplastic anemia in the United States. Clinicoecon Outcomes Res. 2019;11:673-81.

Modelling cost effectiveness of horse antithymocyte globulin for treating severe aplastic anaemia in Germany. S Heublein, Ann of Hematol. 926Heublein S et al. Modelling cost effectiveness of horse antithymocyte globulin for treating severe aplastic anaemia in Germany. Ann of Hematol. 2013;92(6):825-30.

The cost-effectiveness of anemia treatment for persons with chronic kidney disease. B Yarnoff, PLoS One. 117157323Yarnoff B et al. The cost-effectiveness of anemia treatment for persons with chronic kidney disease. PLoS One. 2016;11(7):e0157323.

An economic evaluation of intravenous versus oral iron supplementation in people on haemodialysis. G Wong, Nephrol Dial Transplant. 282Wong G et al. An economic evaluation of intravenous versus oral iron supplementation in people on haemodialysis. Nephrol Dial Transplant. 2013;28(2):413-20.

An economic evaluation of erythropoiesisstimulating agents in CKD. F Clement, Am J Kidney Dis. 566Clement F et al. An economic evaluation of erythropoiesis- stimulating agents in CKD. Am J Kidney Dis. 2010;56(6):1050-61.

A budget impact analysis of the introduction of erythropoiesis stimulating agent subsequent entry biologics for the treatment of anemia of chronic kidney disease in Canada. N Tsao, Can J Kidney Health Dis. 128Tsao N et al. A budget impact analysis of the introduction of erythropoiesis stimulating agent subsequent entry biologics for the treatment of anemia of chronic kidney disease in Canada. Can J Kidney Health Dis. 2014;1:28.

The cost-utility of treating anemia with continuous erythropoietin receptor activator or Epoetin versus routine blood transfusions among chronic hemodialysis patients. O Maoujoud, Int J Nephrol Renovasc Dis. 9Maoujoud O et al. The cost-utility of treating anemia with continuous erythropoietin receptor activator or Epoetin versus routine blood transfusions among chronic hemodialysis patients. Int J Nephrol Renovasc Dis. 2016;9:35-43.

Cost-effectiveness of treating chronic anemia with epoetin alfa among hemodialysis patients in the United States. P Quon, Health Outcomes Res Med. 32Quon P et al. Cost-effectiveness of treating chronic anemia with epoetin alfa among hemodialysis patients in the United States. Health Outcomes Res Med. 2012;3(2):e79-89.

A decision framework for treating chronic immune thrombocytopenia with thrombopoietin receptor agonists. M Dolph, J Comp Eff Res. 78Dolph M et al. A decision framework for treating chronic immune thrombocytopenia with thrombopoietin receptor agonists. J Comp Eff Res. 2018;7(8):775-84.

The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application. M Espinoza, Med Decis Making. 348Espinoza M et al. The value of heterogeneity for cost-effectiveness subgroup analysis: conceptual framework and application. Med Decis Making. 2014;34(8):951-64.

Methods for population-adjusted indirect comparisons in health technology appraisal. D Phillippo, Med Decis Making. 382Phillippo D et al. Methods for population-adjusted indirect comparisons in health technology appraisal. Med Decis Making. 2018;38(2):200-11.

PNS297 clinical and key opinion leaders validation in health economics model and comparative effectiveness: a guide for non-expert audience. K Azimpour, Value in Health. 223813Azimpour K et al. PNS297 clinical and key opinion leaders validation in health economics model and comparative effectiveness: a guide for non-expert audience. Value in Health. 2019;22(3):S813.

Adverse events related to blood transfusion. S Sahu, Indian J Anaesth. 585Sahu S et al. Adverse events related to blood transfusion. Indian J Anaesth. 2014;58(5):543-51.

Transfusiondependent low-risk myelodysplastic patients receiving deferasirox: long-term follow-up. S Improta, Oncol Lett. 66Improta S et al. Transfusion- dependent low-risk myelodysplastic patients receiving deferasirox: long-term follow-up. Oncol Lett. 2013;6(6):1774-8.

Health related quality of life and the CKD patient: challenges for the nephrology community. F Finkelstein, Kidney Int. 769Finkelstein F et al. Health related quality of life and the CKD patient: challenges for the nephrology community. Kidney Int. 2009;76(9):946-52.

The impact of selecting a high hemoglobin target level on health-related quality of life for patients with chronic kidney disease: a systematic review and meta-analysis. F Clement, Arch Intern Med. 16912Clement F et al. The impact of selecting a high hemoglobin target level on health-related quality of life for patients with chronic kidney disease: a systematic review and meta-analysis. Arch Intern Med. 2009;169(12):1104-12.

Erythropoietin therapy, hemoglobin targets, and quality of life in healthy hemodialysis patients: a randomized trial. R Foley, Clin J Am Soc Nephrol. 44Foley R et al. Erythropoietin therapy, hemoglobin targets, and quality of life in healthy hemodialysis patients: a randomized trial. Clin J Am Soc Nephrol. 2009;4(4):726-33.