
in automated audiometry is in concordance with performing manual or traditional audiometry. In this method, the audiometer increases or decreases the intensity of the signal automatically depending on the patient's response. This method is also known as the method of limits [1] .

Generally, automated audiometry is increasingly used to improve access to care, to save time and costs, and to cover the lack of a specialist and to provide services to poor areas [3, 4] . Automated audiometry is usually used in behavioral tests. These tests are divided into three categories: absolute detection thresholds, feature discrimination thresholds, and speech recognition testing. The first category is also called pure-tone audiology which is the main focus of this review study. The feature discrimination threshold test and speech recognition testing are used to obtain supplementary information about pure-tone audiograms [5] .

The pure-tone threshold test is the most commonly used hearing test. This test is conducted in two ways: recording the air-conduction and the bone-conduction thresholds. In the air-conduction method, an earphone is used, and an audio signal passes through the outer and middle passageway and reaches the cochlea. In the bone-conduction method, an electromechanical earphone is placed on the skull, which stimulates the cochlea through a mechanical vibrator without the need to pass the audio signal through the outer and middle ear canal. Determining the threshold levels of airconduction and bone-conduction help to differentiate between two types of hearing loss: sensorineural hearing loss and conductive hearing loss [6] . It is notable that automated audiometry needs to be evaluated in terms of diagnostic accuracy and reliability. There are a number of methods for evaluating automated audiometry to determine the quality of the tests when an audiologist is absent. These methods help to obtain high quality and accurate results which can be easily used in practice [4] .

Although many studies have focused on the design, implementation and evaluation of automated audiometry [1] , few studies have reviewed and compared the implementation and evaluation methods. The aim of this study was to review and summarize the latest studies related to automated audiometry by focusing on the implementation of an audiometer, the use of transducers and evaluation methods. This study can help to gain a better understanding of the topic by discussing the strengths and weaknesses of these methods.
II. Methods
This review study was conducted in 2017. In this study, papers related to the design and implementation of automated audiometry were searched in the following databases: Science Direct, Web of Science, PubMed, and Scopus. The keywords were tele-audiometry, automated audiometry, audiometry and telehealth, audiometry and telemedicine. The time frame for the papers was between January 1, 2010 and August 31, 2017 to focus on the latest studies related to the implementation of an audiometer, transducers, and evaluation methods used for automated audiometry. The language was restricted to English. Initially, 143 papers were obtained; however, 71 papers were removed because of duplication, and one paper was removed because the abstract was unavailable. The remaining 71 papers were screened in terms of their relevancy to the research objective. In this phase, 53 papers were removed due to poor consistency with the aim of this study. In fact, any paper unrelated to pure-tone audiometry and automated audiometry was excluded from the study. In addition, papers that only focused on remote audiometry in the presence of an audiologist, remote consultation, and behavioral tests other than pure-tone audiometry, such as feature discrimination thresholds and speech recognition testing were excluded.
II. Methods
Finally, 18 papers remained; however, the full text was not available for two of them. As a result, 16 papers were included in the study. The process of selecting the papers for this research is shown in Figure 1 .
III. Results
As previously noted, 16 papers were selected for this study. These studies had been conducted in the United States (8 studies), South Africa (5 studies), Australia (2 studies), and Poland (1 study). In this study, different methods for the implementation of automated audiometry, transducers and evaluation methods are discussed (Table 1 ).
Implementation Methods
A review of the literature revealed that there are three ways to implement automated audiometry. These are software solutions [2, [7] [8] [9] [10] [11] [12] , hardware solutions [13] [14] [15] , and smartphone/tablet solutions [3, 5, [16] [17] [18] [19] . Each of these solutions is discussed below.
1) Software solutions
In a number of studies, a test called AMTAS (Automated Method for Testing Auditory Sensitivity) has been proposed for automated recording of pure-tone audiograms which includes both the air-conduction and bone-conduction thresh-Vol. 24 • No. 4 • October 2018
1) Software solutions
www.e-hir.org
Automated Audiometry
olds. In this method, the patient uses 'Yes' and 'No' buttons on a touch screen to indicate whether she/he is hearing or not hearing an audio signal. The signal level differs according to the responder's response to determine the level of hearing threshold of the patient. When generating a signal, sound masking is produced on the ear that is not tested. Circumaural transducers are used in this test to reduce the level of ambient noise. This enables the test to be done in a quiet room instead of a soundproofed room [2, [7] [8] [9] [10] . In a study conducted by Margolis et al. [8] , 30 participants (5 people with a normal hearing status and 25 individuals with hearing impairment) took part in the AMTAS test. The results showed that for air-conduction thresholds, the differences between the test values in the traditional method and the AMTAS were approximately similar to the differences in the values reported by two audiologists. However, for the bone-conduction thresholds, the differences between the recorded values with the AMTAS method and the manually recorded values were larger than the differences between the values reported by the two audiologists. Two reasons mentioned for these differences were incorrect reference-equivalent threshold force levels for bone-conduction through the forehead bone and a differential effect of middle ear diseases on the forehead and mastoid bone-conduction thresholds. Some studies showed that forehead bone-conduction thresholds are less affected by middle-ear diseases than mastoid thresholds for patients with conductive hearing loss. In another study conducted by Eikelboom et al. [2] , the AMTAS test was performed in a quiet room for 44 participants with different levels of hearing impairment. The results indicated that the audiometry changes with the air-conduction and bone-conduction methods were within an acceptable level for the automated and manual methods. Although the AM-TAS thresholds were higher compared to the manual methods, no significant difference was reported.
Automated Audiometry
In another study, a software program called the Home Hearing Test (HHT) was developed to record the air-conduction lauragarcia@example.com. The purpose of this study was to compare the results of the tests performed by the patients with the results of the automated audiometry in a clinical setting. The difference between the HHT and manual thresholds was slightly higher than the recorded thresholds by two audiologists and the measured difference between the AMTAS results and the manual method in a clinical setting. Some of the reasons for this difference were a long time interval between the HHT test at home and the manual test in a clinic (53 days), the probability of environmental noise affecting the thresholds of the HHT and the differences among the participants in terms of the severity of hearing impairment [10] .
Automated Audiometry
The software solution is not limited to AMTAS, and different software has been developed for automated audiometry. For example, in Poland, web-based audiometer software was developed. Three tests were performed to evaluate the software: a manual test, an audiometry test under the supervision of an audiologist in a sound insulation room, and a test which could be done by the lauragarcia@example.com. There was no limitation for the type of earphones used to do the lauragarcia@example.com. The results revealed that a web-based audiometer can be used in screening tests. Although performing audiom- of the cases with an average threshold difference of -1.6 dB (SD 9.9). In the primary healthcare clinics, conventional thresholds exceeding 15 dB HL corresponded to the smartphone thresholds within ≤10 dB in 92.9% of the cases with an average threshold difference of -1.0 dB (SD 7.1).
Automated Audiometry
6 Brennan-Jones et al., 2016 [14] Australia Examine the accuracy of automated audiometry in a clinically heterogeneous population of adults using the KUDUwave automated audiometer. Evaluation study The absolute mean differences ranged between 5.12-9.68 dB (air-conduction) and 8.26-15 dB (bone-conduction). A total of 86.5% of the manual and automated 4 FAs (lauragarcia@example.com, 1, 2, and 4 kHz) were within 10 dB (i.e., ±5 dB); 94.8% were within 15 dB. However, there were significant (p < 0.05) differences between the automated and manual audiometry at 250, 500, 1,000, and 2,000 Hz (air-conduction) and 500 and 1,000 Hz (bone-conduction). System development and evaluation study The evaluation of the hearing threshold was made in three series:
Automated Audiometry
(1) tests on a clinical audiometer, (2) self-tests done on a computer under the supervision of an audiologist, and (3) self-tests lauragarcia@example.com. The average difference between the value of the hearing threshold determined in series 1 and 2 was -1.54 dB (SD 7.88) and a Pearson correlation coefficient of 0.90. Between the first and third series, these values were -1.35 ± 10.66 dB and 0.84, respectively. In series 3, the SD was most influenced by the error connected with the procedure of hearing threshold identification (6.64 dB), calibration error (6.19 dB), and additionally at a frequency of 250 Hz.
Automated Audiometry
8 Eikelboom et al.,
[2] Australia
To validate the air-conduction and bone-conduction AMTAS automated audiometry system. Evaluation study For test-retest reliability, the overall difference in the air-conduction hearing thresholds was 0.5 dB. The spread of the differences was 4.9 dB. For the bone-conduction thresholds, the overall difference was -0.2 dB, and the spread of the differences was 4.5 dB. For the accuracy, the overall difference in the air-conduction hearing thresholds between the two techniques was 0.1 dB. The spread of the differences was 6.4 dB. For the bone-conduction thresholds, the overall difference was 0 dB, and the spread of the differences was 7.7 dB.
[2] Australia
9 Khoza-Shangase and Kassner, 2013 [18] South Africa To determine the accuracy of uHear, a downloadable audiometer on an iPod Touch, when compared with conventional audiometry. Evaluation study Using the paired t-test, it was determined that there was a significant statistical difference between the hearing screening thresholds obtained from conventional audiometry and uHear. The difference in the thresholds may be attributed to the differences in the transducers used, the ambient noise levels and the lack of calibration for uHear. South Africa To validate a computer-based, diagnostic air and forehead bone conduction audiometry when compared with conventional industry standard audiometry in a sound booth environment. Evaluation study Air conduction thresholds for two audiometers corresponded within 5 dB or less in more than 90% of instances, with an average absolute difference of 3.5 dB (SD 3.8). Bone conduction thresholds for the two audiometers corresponded within 10 dB or less in 92% of instances, with an average absolute difference of 4.9 dB (SD 4.9).
[2] Australia
12 Margolis and Moore, 2011 [7] USA To measure the occlusion effect produced by three earphones (circumaural, supra-aural, and insert) and to compare air-and bone-conduction thresholds obtained with manual and automated methods for subjects with sensorineural hearing loss. Evaluation study The supra-aural earphone produced the largest occlusion effects, followed by the insert and circumaural earphones. Some systematic differences in the air-conduction thresholds were found for the two procedures that may be attributable to the earphone differences.
[2] Australia
13 Margolis et al., 2011 [9] USA To evaluate an AMTAS for 4-to 8-year-old children, and a quality assessment method (QUALIND) that predicts the accuracy of the test. Evaluation study For most subjects (93% of the adults and 91% of the children), the differences between the AMTAS and manual thresholds were similar to the differences that occur when two experienced audiologists test the same subjects. QUALIND detected the inaccurate audiograms with a sensitivity of 71% and a specificity of 91%. When the inaccurate audiograms identified by QUALIND were excluded, the accuracy of the AMTAS was similar to the accuracy of the manual audiometry. 
Automated Audiometry
etry tests requires basic knowledge in the context of hearing thresholds or frequencies, a user friendly interface can be used when an audiologist is not available [11] .
Automated Audiometry
Another study was conducted in the United States in which a web-based distributed pure-tone hearing assessment system was developed. It had a three-layer architecture, which increased the scalability of the system to be integrated with other audiometry services. Moreover, the audiometry data were stored in a standardized database, which could be integrated with Electronic Medical Records. In terms of clinical effectiveness, the results showed that the web-based software worked similar to the traditional method at all frequencies. Moreover, the bandwidth required for the system was less than 1 MB/s [12] .
Automated Audiometry
2) Hardware solutions In a number of studies, hardware solutions have been proposed to be used in automated audiometry. According to the literature, the KUDUwave portable audiometer [13] [14] [15] and an earphone designed to remove environmental noise [16] were among the hardware solutions proposed for performing automated audiometry. The KUDUwave is a portable audiometer that makes real-time audiogram recording possible. This audiometer controls the noise attenuation by using earphones and circumaural ear-cups and provides an opportunity for performing hearing tests down to zero dB with an environmental noise up to 59 dB SPL. Moreover, the environmental noise levels are continuously monitored, and if the environmental noise level exceeds the limit, the audiometry test will be stopped. Therefore, this test can be performed outside a sound insulation room [13] . The KUDUwave uses sound masking when it is needed. If the difference between the air conduction thresholds in the test and non-test ear is 75 dB or more when the frequencies are smaller than or equal to 1,000 Hz, or if the difference is 50 dB or more when the frequencies are larger than 1,000 Hz, a masking level of 30 dB is used above the non-test ear. For bone conduction thresholds, a continuous masking level of 20 dB is used above the non-test ear [13, 14] .
Automated Audiometry
In a study conducted by Swanepoel and Biagio [13] , the performance of the KUDUwave was evaluated for 30 individuals aged 19 to 77 years. The results indicated that the air-conduction thresholds had a difference of around 5 dB from the values recorded by the traditional method for 90% of the participants. The bone-conduction thresholds had a difference of around 10 dB from the values recorded by the traditional method for 92% of the participants. However, all the values reported by the KUDUwave were clinically ac- with masking noise presented to the non-test ear. Evaluation study For air-conduction thresholds, the AMTAS manual differences were similar to the inter-tester differences between the audiograms for the paired audiologists, but for the bone-conduction thresholds, the former was larger.
Automated Audiometry
AMTAS: automated pure-tone audiometric procedure, WAHTS: wireless automated hearing-test system. ceptable. A small difference was seen in the test-retest. The results of the test-retest were approximately the same in 91% of cases when the KUDUwave audiometer was used and the difference range was 15 dB. For the traditional audiometer, the results were approximately the same in 92% of cases with a difference range of 10 dB. The reason for the small difference was unknown. However, it might be due to the bone vibrator attachment to the circumaural earphones that produces more static forces in the displacements [13] . In another study in Australia, the hearing status of people was evaluated by using the KUDUwave in the traditional and automated tests. The traditional test was done in a soundproof room, and the automated test was conducted in a room without sound insulation. The bone vibrator was placed on the mastoid bone in the traditional test, while it was on the forehead bone in the automated test. These factors, along with environmental noise, could affect the results. However, the results showed that the difference in the hearing thresholds was low. The results suggested that 86.5% of the recorded thresholds were in a 10-dB range. Although the differences in the thresholds became statistically significant, further research is needed to identify whether these differences are clinically meaningful [14] .
Automated Audiometry
The KUDUwave audiometer was also examined in South Africa. The air-conduction hearing thresholds of 30 participants with normal hearing status and 8 participants with hearing impairment were recorded using both automated and traditional methods. The results showed that the automated audiometry is a stable, accurate, and time efficient method to evaluate the hearing status of adults with normal hearing or hearing impairment. The combination of the automated audiometry with an asynchronous telehealth model, especially for poor areas with little access to hearing specialists, would be beneficial to improve health care services [15] . In a study conducted by Meinke et al. [16] , a mobile wireless automated hearing-test system (WAHTS) was designed to reduce the environmental noise and to be used to record the hearing thresholds in a non-sound proof environment. The system performance was evaluated by examining the airconduction thresholds of 20 workers in six locations, and the results were compared with the results of another test conducted by using computer-controlled audiometry in a mobile trailer sound booth. Overall, the difference between the thresholds obtained by WHATS and the thresholds obtained in the mobile trailer sound booth was within 5 dB [16] .
Automated Audiometry
3) Smartphone/tablet solutions In a study conducted by Whitton et al. [5] , a tablet-based application was developed in the United States. The test algorithm followed the same rules of the clinical test. The audio tones were provided for a time interval of 3 to 7 seconds, and the participants' responses were considered to be a correct answer even up to 2.5 seconds later. The test was conducted in a home and clinical environment. The difference between the mean values recorded at home and at the clinic was small. The thresholds recorded at home had increased for very low frequencies (≤250 Hz). This increase could be due to an increase in the ambient noise at low frequencies in a home environment. This study showed that it is possible to monitor hearing impairments outside a clinical environment.
Automated Audiometry
In another study, a version of the hearScreen smartphone app was used to record the hearing thresholds. The exclusion criteria was a unilateral hearing loss of more than 40 dB HL to avoid inter-aural effects because contralateral masking was not considered in the prototype of the smartphone app. Conventional thresholds exceeding 15 dB HL corresponded to smartphone thresholds within ≤10 dB in 80.6% of the cases. This study suggested that air-conduction audiometry can be performed accurately by a smartphone-based audiometer in a soundproof room or outside a sound insulation room and in the healthcare clinics of poor areas [3] .
Automated Audiometry
The hearTest application was another app developed for Android smartphone-based audiometry. This app was used with a supra-aural earphone. However, no significant difference was seen between the app results and the traditional thresholds except for a 4 kHz frequency. In total, 70.6% of the thresholds calculated by the app and the traditional method had less than a 5 dB difference. Moreover, the duration of the test was not significantly different for the two methods [17] .
Automated Audiometry
uHear audiometer was another application developed by Khoza-Shangase and Kassner [18] in South Africa, and its accuracy was compared with the traditional approach. uHear is an automated screening test which is downloadable for iPod and iPhone. The participants of the study were 86 primary school students. The differences between the results obtained from uHear and the traditional method were significant at all frequencies. In this test, non-calibrated insert earphones were used for automated testing. The advantage of insert earphones compared to supra-aural earphones is reducing environmental noise with greater accuracy. However, this advantage can only be obtained if the earphones are correctly and recently calibrated. Because the study was conducted in a school environment, environmental noise could affect the outcomes. That might be the reason why uHear was not as accurate as the traditional audiometry in determining the hearing thresholds for primary school children. This study showed that caution should be exercised in using uHear and that further research evidence is needed to use it at a general level.
Automated Audiometry
Another iOS-based application was developed to be used on the iPhone, iPod touch and iPad. The hearing thresholds were recorded with an automated method in a quiet room as well as in a sound insulation room. Sound masking was applied automatically when the difference between the threshold values of the two ears was greater than or equal to 35 dB. The sound masking included a narrowband noise centered at the frequency that was tested. The results showed that on average, 96% of the thresholds recorded in the sound insulation room with the automated test had a difference in the range of 10 dB compared to the thresholds recorded in the sound insulation room with the traditional method. In a quiet room, this amount reached 94%. The results indicated that the obtained thresholds were close to traditional audiometry results, and the iOS-based tools provided a platform for conducting automated audiometry with no need for additional equipment [19] .
Transducers
A review of the studies showed that the most important transducers used in the implementation of an automated audiometry system included earphones and a bone vibrator which are discussed below separately.
1) Earphones
Having reviewed the literature, it was revealed that there are two important factors regarding the choice of earphones or their design when using them for audiometry tests. The first factor is related to reducing environmental noise, and the second factor is related to the occlusion effect of the earphone. It is notable that the background noise in the test room is called environmental noise. Because the automated audiometry test might be performed in an environment outside a sound insulation room, the environmental noise levels must be minimized to be able to record the hearing thresholds precisely. The ambient noise level should be much lower than the test signal level, so that the listener can distinguish the test signal from the environmental noise [16] . The occlusion effect causes a change in the values recorded for the bone-conduction thresholds (usually an incremental change) due to the obstruction of the ear canal. The skull vibration is transmitted to the walls of the external ear canal and tympanic membrane. When the ear canal is not clogged, the ear canal acts as a high-pass filter, and low-frequency sounds are removed. When the canal is clogged, the low-frequency energy falls to the trap and transmitted into the inner ear [20] .
1) Earphones
The standard procedure for measuring the bone-conduction thresholds is to perform the test in a condition in which the ear canals are not clogged. On the other hand, earphones are used to record the air-conduction thresholds. If the earphone does not produce the occlusion effect, the earphones can be kept on the ears during the test. This feature enables recording the air-conduction and bone-conduction thresholds with no need to switch earphones and causes no interruptions during the test [7] . In general, noise reduction techniques improve the value for hearing thresholds for low frequencies by using two active and passive modes. The passive technique attempts to prevent the environmental noise from entering the ear canal. In the active noise elimination technique, a microphone is used to measure the amount of noise in the environment, which is neutralized by using environmental noise opposing signals [5] .
1) Earphones
The available earphones can be divided into three main categories: circumaural, supra-aural, and insert earphones. Figure 2 shows how each earphone couples to the external ear. The KUDUwave audiometer is powered by circumaural ear cups with insert earphones to control the environmental noise. Moreover, there are microphones in the inner and outer parts of the earphones to monitor the ambient noise, and if the level of ambient noise exceeds the limit, the audiometry test is stopped. These features help to perform the evaluation with a better quality in an environment outside of a soundproof room [13] .
Circumaural
Supra-aural Insert P EE P TM P EE P TM P EE P TM In a study conducted by Meinke et al. [16] , a headset was designed to reduce the environmental noise and to be used to accurately record the hearing thresholds outside an insulating room and in an industrial environment. The headset was relatively large and heavy to do passive attenuation. It had multiple layers to eliminate environmental noise by a passive technique. The ear cup with thick polyurethane foam, a speaker, a plastic face plate, a thin protective fabric and an ear seal from hearing protectors were lined to eliminate noises. The test-retest reliability results were equal or better than the results from the supra-aural, insert and circumaural earphones.
Circumaural
In another study, the occlusion effects created by earphones were highlighted. One of the important points in performing an automated audiometry test is to prevent the occlusion effects created by earphones. Supra-aural earphones produce a significant occlusion effect at frequencies of 1 kHz or lower. Insert earphones also produce a significant occlusion effect unless completely inserted into the ear canal and into its bony part. Putting the earphones in this way is not possible in routine tests due to the lack of comfort and safety; however, if the earphone is put around the ear and a large volume of air is placed inside, it can eliminate the occlusion effect. The results of this study showed that the occlusion effect produced by circular earphones for frequencies above 500 Hz is so insignificant, and the thresholds recorded in the bone-conduction test with this type of earphone are similar to the thresholds recorded without earphones. Therefore, according to the results, the circumaural, insert and supraaural earphones had a better performance in not generating the occlusion effect, respectively [7] .
Circumaural
2) Bone vibrator One of the important technical factors in an automated audiometry test is the location of placing the bone vibrator. The vibrator is used to generate signals for determining the bone-conduction thresholds. The two common methods for producing these signals are the use of the mastoid bone and the forehead bone. The mastoid bone is used mostly in traditional audiometry techniques [7] . In the case of using the mastoid bone, the vibrator needs to be placed on the left and right ears and between the ears. To increase the efficiency, it is better not to move the oscillation tool from one ear to another ear. The forehead bone is preferred in automated testing, since there is no need to change the vibrator placement during the test [8] .
Circumaural
The KUDUwave audiometer uses a bone vibrator that is placed on the forehead bone. In this audiometer, the insert earphones are used during the testing of the bone-conduction thresholds. As previously mentioned, the earphones should be deeply inserted into the bone part of the ear canal to avoid the occlusion effect of insert earphones. According to the results, the correlation between the bone-conduction thresholds in the test-retest is slightly less than the measured values in the traditional audiometry tests, although they were within acceptable limits. The reason for this small difference was unknown. It might be partly due to the bone vibrator which was attached to the circumaural earphone and generated more static force in the movements [13] .
Circumaural
In the case of using the forehead bone, the earphones are used on both ears. When testing each ear, sound masking is produced on the other ear. Therefore, it is important that the earphones create a small amount of obstruction to have no effect on the bone-conduction thresholds. The difference between the two traditional and automated methods in the bone-conduction thresholds is greater than that of the airconduction mode. This difference can be due to the difference in the position of the vibrator on the mastoid bone and the forehead bone or due to middle ear diseases, which affect the sensitivity of the bone-conduction thresholds in both the mastoid and forehead methods. The forehead method estimates the cochlea sensitivity with a higher accuracy [8] .
Evaluation of Automated Audiometry
To evaluate the accuracy of automated audiometers, the results of automated and traditional methods can be compared. The traditional method is selected as the gold standard, and the results of the automated method are compared to the gold standard. Audiometry tests can be performed with traditional or automated methods either consecutively lauragarcia@example.com. In the consecutive method, it is better to have a balance between the tests due to the possible impact of learning, fatigue, attention and motivation on the results of the tests. According to World Health Organization definitions, the assessment of people's hearing levels can be divided into normal hearing, disabling hearing impairment, conductive hearing impairment and unilateral hearing impairment. Then, the level of agreement can be calculated between the traditional and automated methods [21] . In a study, the results of automated audiometry were compared with the traditional method for a sample of patients to examine the reliability and generalization of the test results. The selected sample included patients with chronic tinnitus disorder who had from a normal hearing status to severe hearing impairment. According to the findings, the results of both methods were similar at the frequencies of 500 to 8,000 Hz [5] .
Evaluation of Automated Audiometry
The test-retest method is another method used to evaluate the reliability of the hearing tests. In this method, the testretest difference value of a standard audiometer is calculated [5, 13] . In another method, the difference between the automated approach and the traditional method is compared with each other. Moreover, the mean value of the hearing thresholds can be compared between two audiologists on a participant and by using the traditional method. The mean value can be compared to the result of an automated test to measure the accuracy of this test, too. If the difference between the automated method and the traditional method is not greater than the mean value, it means that the automated method does not have much of a difference from that of the traditional method of audiometry. The mean value can also be used as a criterion to measure the reliability of the traditional method [8] .
Evaluation of Automated Audiometry
The results of automated audiometry can also be evaluated by using qualitative methods. In the traditional method, it is assumed that a skilled audiologist can observe the specific characteristics and behaviors of a patient and can use them to predict the validity of the test. In an automated test, there is a need to use an alternative method to evaluate the quality and accuracy of the test results, mainly due to the absence of an audiologist. To achieve this, a method called QUALIND has been proposed. This is a qualitative assessment method for determining the accuracy of the automated test results by using measurable factors, such as the behaviors and characteristics of a patient. In an automated test, the following factors can be used to predict the validity of the results: patient's age and gender, the duration of the test, the average time for determining each frequency, the rate of wrong warnings, the difference between the test-retest and the number of cases in which the difference between the air and bone conduction thresholds was more than 50 dB [22] . In a study conducted by Margolis et al. [9] , the QUALIND method was used to assess the AMTAS results of children. This method detected incorrect audiograms with a sensitivity of 71% and a specificity of 91%. After removing the incorrect audiograms, the AMTAS accuracy was similar to the manual audiometry accuracy. This method can reduce the costs and increase the efficiency and accessibility of audiometry test.
IV. Discussion
The audiometry of pure-tone thresholds is based on a series of distinct steps and can be implemented in the form of an automated process [4] . In addition, if a computer is used, the results are automatically recorded and can be transferred to other professionals easily. Moreover, performing automated audiometry can improve the standardization of the test procedures [1] and facilitate patient monitoring in poor areas [23] .
IV. Discussion
The results of the current study showed that different software and hardware solutions have been used for automated audiometry. As smartphones become ubiquitous, new opportunities have arisen for presenting innovative solutions, especially in poor and remote areas [19] , and a variety of audiometry applications have been developed to be able to record the audiograms at different times and places by using a smartphone [3] . The smartphone application can be an affordable and a valid method to determine the air-conduction hearing thresholds [17] . Although the use of technology has some advantages, the limitations of technology should not be underestimated. For example, due to the limitation of smartphones in generating different audio frequencies and intensities, these applications can only be used for general screening programs when traditional audiometry tests are not available [19] . Another limitation is about sound calibration. Unlike an audiometer, the output sound of smartphones is not calibrated, and it may not meet the requirements of audiometry. Moreover, the hardware of smartphones and audiometers is different, and the accuracy of the results should be examined [24] . It seems that more studies are required to identify the strengths and limitations of computerized solutions for automated audiometry to be able to design more effective solutions in the future.
IV. Discussion
According to the results, two important audiometric transducers are earphones and bone vibrators. As one of the challenges of using automated audiometry is environmental noise, especially at low-frequencies [10] , different types of earphones can be used to reduce the environmental noise through active or passive techniques [5] . These features help to perform a high quality test outside of a sound proof room [13] . Moreover, the literature review showed that the boneconduction thresholds should be measured along with the air-conduction thresholds to diagnose the type of hearing impairment [5] . In traditional audiometry, earphones are removed from the ear for a bone-conduction hearing test to prevent them from producing occlusion effects, while in automated audiometry, due to the absence of an audiologist, it is better to keep the earphones on the ears to record the bone-conduction hearing thresholds. This approach can increase the test quality [7] . However, there are different types of transducers, and the use of each type may affect the results of the automated audiometry and hearing impairment diagnosis. Therefore, it is essential to use those transducers that have been previously tested and can produce results as accurate as the tests conducted by clinicians.
IV. Discussion
The results showed that the benefits of automated audiometry are only achieved when the quality of the produced audiograms is at least similar to that of traditional audiograms. Inaccurate audiograms may lead to test repetition, increased costs, and a waste of time [4] . To assess the validity of the automated audiometry test, there are different methods for evaluation. For example, the difference between the threshold values reported in traditional and automated audiometry have been calculated in different studies, and the reported values in a range of 5-10 dB have been considered acceptable [2, 3, [13] [14] [15] [16] [17] 19] . In another study, the correlation coefficient between the values obtained from the automated and the traditional method was calculated [11] . In other studies, the difference between the values reported by the automated and the traditional method was compared with the difference between the values reported by two audiologists [8] [9] [10] . In most studies, the results of automated audiometry were similar to the traditional approach, and it seems that traditional audiometry can be replaced with an automated approach. However, as mentioned before, different automated solutions and different transducers may produce different results, and as a result, conducting evaluations are inevitable.
IV. Discussion
In conclusion, automated audiometry produces clinically acceptable results compared with traditional audiometry. The two main advantages of automated audiometry are saving costs and improving accessibility to hearing care, which can lead to a cost-effective and rapid diagnosis of hearing impairment, especially in poor areas. The use of automated audiometry may have some challenges, such as measuring the impact of environmental noise on the test results, recording bone-conduction hearing thresholds with the possibility of generating occlusion effects by the earphones, and ensuring the quality of the automated audiometry test results. Further studies need to be conducted to compare the characteristics of different computerized solutions and related challenges for automated audiometry. Because the performance of transducers are different, evaluation studies are needed to compare their performance to be able to choose the best one for automated audiometry.

{'paper_id': '53423660', 'title': 'Automated Audiometry: A Review of the Implementation and Evaluation Methods', 's2_url': 'https://api.semanticscholar.org/CorpusID:53423660', 'source': 'S2ORC'}