Mohammad Ali Esmaeili,Mohammad Reza Ghotbi Ravandi,Sajad Zare
Assessing the impact of COVID-19 pandemic on the performance indicators of safety management using the analytic hierarchy process (AHP) in an electricity industry
Analytic hierarchy process,Performance indicator,Safety,COVID-19
Introduction The importance of evaluating the performance of different management systems in industries necessitates examining the performance of the Health, Safety, and Environment (HSE) management system along with other management systems. Particularly during the COVID-19 pandemic, there is a greater need than ever to assess the impact of the COVID-19 spread on the performance of the HSE management system compared to before this pandemic. This research aims to investigate safety performance indicators (SPIs) on the performance of the HSE management system and the impact of the COVID-19 spread on these indicators. Methods This cross-sectional, descriptive-analytical study was conducted to evaluate the influence of the COVID-19 outbreak on the safety performance to revise the industry safety index in an electricity distribution company using the multi-criteria decision-making method before and after the disease epidemic in three stages. In the first part, the safety indicators were identified according to the comprehensive safety indicators available in the industries and experts' opinions. In the second part, safety indicators were ranked, weighted, and prioritized using the Analytic Hierarchy Process (AHP). In the third part, these indicators were calculated and compared in the periods before and after the outbreak of COVID-19. Results Two main criteria, namely the “effectiveness criterion” and the “applicability/calculation criterion”, were identified for the evaluation and pairwise comparisons of performance indicators. Among these two criteria, the applicability/calculation criterion had higher priority and importance for the evaluation of indicators. Pairwise comparisons of the indicators indicated that the “accident frequency rate” and “safe T-factor” indicators (with weights of 0.238 and 0.023 respectively) had the highest and lowest priorities, respectively, for the assessment of organizational safety performance among the SPIs. Conclusion Based on the calculations of the indicators and their analyses before and after the outbreak of COVID-19, the current status of the safety performance of the HSE unit was not significantly affected by this pandemic. However, the investigations carried out while collecting the data needed to calculate the indicators and evaluate the performance of this unit demonstrated that some indicators were not considered sufficiently in the studied electricity industry. Since conducting regular performance evaluations greatly impacts the achievement of continuous improvement, more attention should be paid to compiled indicators, which should be periodically assessed in the organization to achieve continuous improvement.
Assessing the impact of COVID-19 pandemic on the performance indicators of safety management using the analytic hierarchy process (AHP) in an electricity industry

The importance of evaluating the performance of different management systems in industries necessitates examining the performance of the Health, Safety, and Environment (HSE) management system along with other management systems. Particularly during the COVID-19 pandemic, there is a greater need than ever to assess the impact of the COVID-19 spread on the performance of the HSE management system compared to before this pandemic. This research aims to investigate safety performance indicators (SPIs) on the performance of the HSE management system and the impact of the COVID-19 spread on these indicators.

This cross-sectional, descriptive-analytical study was conducted to evaluate the influence of the COVID-19 outbreak on the safety performance to revise the industry safety index in an electricity distribution company using the multi-criteria decision-making method before and after the disease epidemic in three stages. In the first part, the safety indicators were identified according to the comprehensive safety indicators available in the industries and experts' opinions. In the second part, safety indicators were ranked, weighted, and prioritized using the Analytic Hierarchy Process (AHP). In the third part, these indicators were calculated and compared in the periods before and after the outbreak of COVID-19.

Two main criteria, namely the “effectiveness criterion” and the “applicability/calculation criterion”, were identified for the evaluation and pairwise comparisons of performance indicators. Among these two criteria, the applicability/calculation criterion had higher priority and importance for the evaluation of indicators. Pairwise comparisons of the indicators indicated that the “accident frequency rate” and “safe T-factor” indicators (with weights of 0.238 and 0.023 respectively) had the highest and lowest priorities, respectively, for the assessment of organizational safety performance among the SPIs.

Based on the calculations of the indicators and their analyses before and after the outbreak of COVID-19, the current status of the safety performance of the HSE unit was not significantly affected by this pandemic. However, the investigations carried out while collecting the data needed to calculate the indicators and evaluate the performance of this unit demonstrated that some indicators were not considered sufficiently in the studied electricity industry. Since conducting regular performance evaluations greatly impacts the achievement of continuous improvement, more attention should be paid to compiled indicators, which should be periodically assessed in the organization to achieve continuous improvement.

In the current highly competitive world, many organizations also consider Health, Safety, and Environment (HSE) management as an indispensable and necessary part of the organization and value it in parallel with other management systems [1]. The effectiveness of the HSE management system is an important factor to achieve the goals of the organization in the field of safety, health, and environment. Obviously, accidents, occupational diseases, and environmental damages cause lots of direct and indirect damage to organizations [2]. By monitoring the organization's safety and minimizing injuries and occupational diseases, HSE management can advance industrial activities toward sustainable development. Factors affecting the sustainable development, productivity, and profitability of an organization include the health of human resources, the protection and improvement of the efficiency of equipment and facilities, and environmental protection, and all kinds of protection from these factors will ultimately increase the productivity of an organization [3]. In all organizations, performance evaluation of management systems is an important factor influencing sustainable development. Organizations measure the performance of an HSE management system by planning and determining performance indicators and examining the effectiveness of their performance. The identification of HSE performance indicators to measure and analyze management performance reduces safety and environmental risks and ultimately paves the ground to achieve sustainable development in relevant operations. The HSE unit is a service or independent business unit that needs a performance evaluation system [4]. The methods used in the performance evaluation of HSE have so far mostly relied on the indicators that draw the past performance; hence these methods are not sufficiently efficient in drawing the performance results in the past. However, one of the effective methods of evaluating the performance of the HSE management system is to examine the status of the safety, health and environment performance indicators which are generally known as leading indicators in industries [5]. In the industrial context, the introduction of indicators has proven to be a useful tool for obtaining information about the internal consequences of activities and processes so that potential risks can be monitored [6]. Using and evaluating safety performance indicators as one of the leading indicators helps the organization in evaluating the safety situation as well as predicting and preventing incidents [7]. During the outbreak of COVID-19, the evaluation of safety performance became very important so that de Oliveira Neto et al. stated that the spread of this pandemic can have a significant negative effect on the safety and health performance of an industry [8]. In another study, Nnaji et al. stated that, this pandemic could exacerbate an already fire safety and health situation in the industry, adversely affecting construction workers and employers [9]. Decision-making is defined as the process of determining the best option among all available options. Nonetheless, in most cases, it is difficult to achieve the optimum result as decision-makers often face various problems when it comes to making certain decisions. Multi-criteria decision-making (MCDM) is one of the most commonly used methods in the decision-making process and is used to identify the best of all available choices [10]. There are numerous techniques to perform MCDM, including AHP, superiority and inferiority ranking (SIR), Simos ’ranking method, multi-attribute utility theory (MAUT), and elimination and choice corresponding to reality (ELECTRE) [11]. The analytic hierarchy process (AHP) is used in making decisions based on quantitative and qualitative criteria. In this analysis, different options are assessed together based on different criteria such that the elements of each level are compared with their respective elements at a higher level to select the best option or suitable options [12]. The AHP is very compatible with the thinking style and cognitive processes of humans, and its algorithm is based on mathematical logic, making it a highly efficient method that has been used to solve many decision-making problems [13]. This technique examines a problem or a factor hierarchically, from goal to criteria, sub-criteria, and alternatives at successive levels [14]. A hierarchical assessment provides experts with an overview of intrinsic complex relationships in a certain field and allows them to compare various factors based on their weight using a 9-level scale [15]. Various studies have been conducted in order to make decisions in the field of safety management using the AHP technique. For example, In the study of Bragatto et al. A questionnaire was developed to assess the safety resilience of organizations. In their study, a two-level AHP model was developed to define the weight assigned to each point of questionnaire. The AHP panel included practitioners from industry, regulators, authorities and researchers. The results are based on their experience with COVID-19, and as a result the model developed is tailored to address health emergencies. They also stated the approach easily can translates to other external crises [16]. Hadidi and Khater in a study, developed an AHP model that is applied to rank Turnaround maintenance contractors' selection. In this study, they defined main needed safety attributes to prequalify contractors [17]. In another study Jabbari et al. applied fuzzy analytical hierarchy process to weigh and priorate fire, explosion and release of toxic gas risk in the context of safety management of gas pipelines [18]. At the end of December 2019, the outbreak of an unknown type of pneumonia was reported in Wuhan, China, which was very similar to other types of viral pneumonia. This virus, which belongs to the Beta coronavirus family with many natural hosts like other viruses, was named COVID-19 by the World Health Organization (WHO) [19]. This virus is known as the most dangerous virus in the branch of pneumonia-causing viruses [20]. After the global spread of this virus and the increased cases of infection with this virus, the spread of the coronavirus was introduced as the sixth reason for declaring a global health emergency and a threat to the whole world by the WHO on January 30, 2020 [21]. The spread of this virus has influenced the performance of various industries and has even led to the widespread national and global cessation of many industrial units, production units, markets, and other supply chain activities [22]. According to one of the existing theories, organizations will be propelled to change production systems and supply chains more sustainably in the post-COVID-19 era. Accordingly, there is a need to develop sufficient organizational and operational policies to overcome the existing losses, which will further flourish the industries [23]. Further consideration of the crisis and its effects on different industries may encourage management systems to better prepare for different types of crises and the resulting shocks in the future. Therefore, the present study aims to investigate the impact of the COVID-19 outbreak on the safety performance indicators (SPIs) to evaluate the safety performance of the industry during this pandemic and help to increase the understanding of industries about the impact of the COVID-19 pandemic on the industry and their performance.

This cross-sectional, descriptive-analytical study was designed to identify and evaluate the impact of the COVID-19 outbreak on safety performance and compare it with the situation before this pandemic to revise the industry safety index in an electricity distribution company using a multi-criteria decision-making method. The periods before and after the COVID-19 pandemic were determined based on consultation with the company. Fig. 1 shows study flow diagram for applying AHP method [24]. Accordingly, the present study was conducted based on the following two main parts.

According to previous studies, the number of experts is not very important in decision-making using the AHP method, but the competence of the experts participating in the decision-making process is more important [25]. In the studied industry, we only had access to 8 qualified experts, including university professors, health and safety experts, and supervisors with more than 1 years’ experience, were selected for this study. The experts were asked to complete two questionnaires, namely a comparative questionnaire to evaluate the criteria and a questionnaire to assess the solutions based on the criteria. In this study, two main criteria and 11 alternatives (safety indicators) were identified (Table 1) according to field studies and experts' opinions.

The main steps of the AHP methodology were summarized as follows.

In this study, criteria and alternatives should first be determined to prioritize and weight SPIs by means of AHP. The study aims to prioritize and weight the control measures of sound engineering.

To draw the AHP network, a connectivity matrix is first specified in which the rows and columns both contain influencing factors. Then, the final structure of the problem is obtained in the form of a tree. In general terms, it can be stated that the method for making a hierarchy depends on the type of decision to be taken. The goal is placed at the top of the AHP network, and the criteria are displayed afterward [26]. The alternatives lie at the lowest level of this network. Fig. 2 depicts the relationship between the goal of the study (prioritization and weighting of safety indicators), study criteria, and identified indicators.

As input to the AHP, a questionnaire should be designed to collect the opinions of the target community and perform paired comparisons based on these questionnaires. To do this, a questionnaire containing criteria and alternatives is designed based on Saaty's nine-point scale and is offered to technical and health occupational experts in several industries who are active in the specialized field of sound control and management, after which the criteria and alternatives are determined based on the experts' score. The selected cases are used based on the AHP method to determine the weighting of sound engineering control measures [27]. Experts should score each question between 1 and 9, which respectively indicate the same importance of two elements and very high importance of one element versus the other one. Following guide is useful for filling out the questionnaire according to the 1–9 Saaty scale: Experts must give each question a score from 1 to 9. A score of 1 indicates the equal importance of the two elements, and a score of 9 indicates the great significance of one element in comparison to another element. The degree of importance of the scores (1–9) in the pairwise comparisons has been presented in the Table 2 [27].

The AHP analyses difficult and complex problems to convert them into a simple form and solves such issues. This method is widely used in economic and social issues and has also been used in management issues in recent years. To solve a problem or decision, two steps (i) calculation of weights and (ii) system compatibility should be taken after making the structure of the problem [28].

The relative weight is obtained from a pairwise comparison matrix (PCM), while the absolute weight is the final rank of each option, which is calculated by combining the relative weights.

In the AHP, the elements are first compared in pairs to form a PCM, and then the relative weights of the elements are calculated using this matrix. In general, a PCM is shown as below, where aij is the preference of the i-th element to the j-th element, and the weights of elements, i.e. the wis, are obtained according to the values of aijs (Equation (1)) [28].Where (positive reciprocal) and . Each PCM may be consistent or inconsistent; in the former case, the weight (wi) is calculated simply by normalizing the elements of each column. In an inconsistent matrix, the weight calculation is not simple and is obtained using four methods (least squares method, logarithmic least squares method, eigenvector method, and approximation methods) [28].

In an AHP, the final weight of each option is obtained from the sum of the product of the importance of the criteria by the weight of the options as follow (Equations (2), (3))):By multiplying by we have: By solving equation (2.3), the eigenvalue of the problem is obtained, then the comparative weights can be obtained by finding the eigenvector with the corresponding λmax.

A matrix may be consistent or inconsistent; in the former case, the weight is calculated simply by normalizing the elements of individual columns. In an inconsistent matrix, however, the weight is obtained using several methods. In addition to weight calculation in inconsistent matrices, calculating the inconsistency ratio (IR) is of paramount importance. It can generally be stated that the acceptable IR of a matrix or a system depends on the decision-maker, but Saaty introduces 0.1 as an acceptable limit and believes that the judgment should be reconsidered if the IR is more than 0.1 [28]. The inconsistency ratio is calculated as follows (Equation (4)):where is the largest eigenvalue, and denotes the numbers of the attributes. The consistency ratio is obtained by dividing the inconsistency ratio by the random inconsistency index according to the Equation (5): The values of the inconsistency index (I.I.) have been calculated for the matrices whose numbers are chosen completely randomly, named the random inconsistency index (R.I.I.), whose values are for n-dimensional matrices according to the Table 3. For each matrix, the result of dividing the I.I. by its same-dimensional R.I.I. is then a suitable criterion to judge inconsistency, called the inconsistency ratio (I.R.). If this number is < 0.1, the consistency of the system is acceptable, otherwise, it is necessary to reconsider the judgments. After ensuring the acceptability of obtained priorities, the average of the submitted opinions will be calculated using the geometric mean method with the coefficients of each PCM. Before introducing different people's opinions to the software, they must be integrated and consolidated in the form of a questionnaire [22], for which the geometric mean formula will be used according to Equation (6):

The SPIs of the organization were evaluated and prioritized using Expert Choice software.

The present study was approved by the ethics committee (with the code IR. KMU.REC.1401.030) at Kerman University of Medical Sciences.

Table 4 represents the pairwise comparisons of the criteria based on the goal of the problem. According to the results, the criterion of applicability and calculation had a higher priority than the effectiveness criterion.

Pairwise comparisons of SPIs presented based on the performance criteria of each indicator are shown in Table 5. Based on the effectiveness criterion, the “accident severity rate (ASR)” indicator had a higher priority for evaluating the organization's safety performance than the other indicators. As shown in this table, the next priorities belong to the “accident frequency severity (AFS)” and “accident frequency rate (AFR)” indicators. Additionally, the “safety training” and “number of distributed personal protective equipment” indicators had the lowest priorities for evaluating the organization's safety performance.

Table 6 shows the pairwise comparisons of SPIs based on the criterion of applicability and calculation for each indicator. According to this criterion, the “AFR” and “safety training index” indicators had higher priorities for evaluating the organization's safety performance than the other indicators. The mentioned criterion also indicated that the “AFS” and “ASR” indicators had lower priorities for evaluating the organization's safety performance than the other indicators. As shown in all above tables (Table 2, Table 3, Table 4), all the IRs of the above PCM are <0.1, which is a desirable result.

According to the results of the normalized weights and ranks of individual criteria presented in Table 7, the criterion of “applicability and calculation” has a higher weight and priority for evaluating the safety performance indicators of the organization.

According to Table 8, the normalized weights of individual indicators based on the efficiency criterion revealed that “ASR” and “AFR” indicators regarding the criterion of applicability and calculation had the highest priorities for evaluating the safety performance of the relevant organization.

According Table 9, the final normalized weights and ranks of the indicators showed that the “AFR” and “safety training” indicators, with weights of 0.238747 and 0.152497 respectively, had the utmost effectiveness and priority for evaluating the organization's safety performance. Moreover, the “safe T-score” and “AFS” indicators, with weights of 0.023129 and 0.032971 respectively, possessed the lowest priorities for evaluating the organization's safety performance.

Table 10 indicates the decrease in the majority of safety indicators in 2020 due to the increased training hours compared to those in 2019, suggesting the positive effect of training on the safety of the organization.

The present study aimed to investigate the impact of the COVID-19 pandemic on SPIs in the electricity industry. HSE requirements are an essential part of the management process of organizations and firms. The statistics of occupational accidents and injuries are still high despite the implementation of HSE programs [29]. Evaluating the performance of management systems is a major concern of managers, which is not exempt from the HSE system [30]. Managers want to know whether or not all the elements of an implemented management system are working properly and whether the effect of the implemented system is as expected [31]. The need to create a precise performance evaluation system in the area of HSE has been emphasized in many studies [[32], [33], [34], [35]]. A study considers the necessary existence of an HSE system as a management tool to monitor and confirm the effective establishment of the HSE policy in every organization and factory, along with an emphasis on the performance evaluation of this sector periodically [35]. The present study was conducted on an Electricity Company, a leading industry in the field of HSE and a company with a comprehensive management system throughout the organization. The performance of this industry was evaluated in terms of HSE in the COVID-19 pandemic and the extent to which the performance of this organization was influenced by the spread of this disease. Based on the abovementioned issues, SPIs were first identified and then evaluated in this research based on some criteria. Two main criteria, namely applicability and effectiveness, were used in prioritization. Applicability refers to the extent to which the intended index can be calculated in terms of cost, facilities, and currently available data. This criterion is opposed to the effectiveness criterion, meaning the extent to which the performance of the field covered by that index can be accurately evaluated by calculating this index. The results of the current study, which is based on experts' opinions, indicates that the applicability criterion has a higher priority than the effectiveness criterion. Therefore, the applicability and calculation indicator should first be considered in the calculation of indicators, followed by examining the effectiveness criterion. This result is defensible from a logical point of view because if an index has the best effectiveness but it is not applicable and calculable, it will have no role in performance evaluation., The applicability and calculation of performance indicators, as well as the rationality of the process of safety management systems, were also emphasized in a review study on the literature on safety management systems in 2018 [36], the results of which are similar to ours regarding the applicability and calculation of indicators. Based on the pairwise comparisons of the SPIs presented based on the criterion of applicability and calculation of each indicator, the “AFR” and “safety training” indicators had a higher priority for evaluating the organization's safety performance than the other indicators. According to the effectiveness criterion, a higher priority was recorded for the “ASR” indicator in evaluating the safety performance of the organization than the other indicators. In a study conducted in 2020, all the above three indicators were used as performance indicators to investigate the impact of an occupational health and safety management system on the organizational HSE performance in a casting industry over 5 years [37]. A comparison of the results of the “AFR” indicator revealed no significant differences between before and after the spread of COVID-19. The results of the study disclosed that the outbreak of COVID-19 did not influence the year-long number of accidents in the studied electricity industry. Nevertheless, the comparison of the “safety training” indicator indicated that safety training increased significantly after the COVID-19 pandemic. One of the main reasons for this result may be because society and consequently the company managers focused more on the health of human resources after the COVID-19 pandemic than before, providing an opportunity to allocate more time and expense of the organization to educational topics. A similar study emphasizes education during the COVID-19 pandemic by reporting a considerable increase in education during the outbreak of the disease, along with an emphasis on the leadership role of managers in education [38]. A significant difference was observed in the comparison of the “percentage of performed corrective measures” in the safety area before and after the spread of COVID-19. As such, no corrective measures were taken before the disease, but all the corrective measures were taken after this pandemic. It seems that the activities of the electricity industry are followed up more seriously due to the COVID-19 pandemic than before. A major reason for this assumption is that the importance of people's health has been highlighted at all social levels during the COVID-19 period. This importance has been transferred from the world community to countries and from politicians of countries to managers of organizations and the general public. On the other hand, the increased awareness of the personnel has increased their demands regarding the provision of a safe work environment, and this demand has driven more serious follow-up of corrective measures and improvement of workplace conditions. In 2021, a study on nurses' demands and views in the care environments of COVID-19-infected patients reported an increase in nurses' demands to create a safer environment than before [39]. In another study conducted in 2021, the perceived risk of infection with COVID-19 by personnel was reported to cause emotional exhaustion, and the existence of safety systems, communication, decision-making, and participation in decision-making would reduce the perceived risk. Consequently, organizations were recommended to reduce the risk of infection with the virus in work environments by taking intervention measures [40]. In the study conducted by Bragatto et al. [16], the AHP method was also used for weighting and prioritizing safety performance indicators during COVID-19 pandemic. The results of their study showed that risk assessment, operating control, and resources indicators with weights of 24.3%, 22.1%, and 15.5%, respectively, are in the first to third priorities of the effective indicators in evaluating safety performance [16]. Based on this, the results of our study also showed that the indicators of accident frequency rate, safety training and risk assessments have weights of 23.8% and 15.2%, respectively. 14% were in the first to third priorities. Besides this, it can be said that the results of both studies are consistent. Also, in another study conducted by Kusrini and Maswadi [41] in order to prioritize indicators for improving the performance of supply chain management in a palm oil industry using F-AHP technique during COVID-19, among the safety and health indicators, the “Increasing the health and safety of employees” indicator with a weight of 0.502 had the most influential weight. In addition, the “Significant reduction in work-related deaths” indicator with a weight of 0.382 was ranked second. Also, in the current study, the indicators that led to the improvement of workers' safety and health and the reduction of their deaths, such as the indicators of safety training and the indicators of corrective actions carried out in the industry, have a high impact weight compared to others. In addition, in the study of Guo and Wu [42], which was conducted in order to evaluate the performance of the supply chain of manufacturing companies using the fuzzy-AHP-DEMATEL-VIKOR technique in the period of COVID-19, it was stated that the indicators of the frequency of product safety incidents and the frequency Export control safety incidents are two valuable and effective indicators on the social sustainability of the supply chain in the organization that should be taken into account. The findings were completely in line with the results of the present study, which showed that the accident frequency rate indicator has the greatest impact on the organization's safety performance. Altogether, the status of safety indicators in the studied electricity industry has relatively improved in 2020 compared to before the spread of COVID-19 in 2018. This relative improvement can be justified for two reasons, one of which is the improved “safety training” indicator that could effectively improve overall safety as indicated by the results. Another reason is a 100% improvement in the indicator of “percentage of performed corrective measures”. Obviously, the elimination of shortcomings and the follow-up and complete implementation of corrective measures will play a significant role in improving the overall status of safety. One of the limitations of the existing study was the organization's lack of access to the appropriate system for recording information related to indicators after the outbreak of COVID-19. For this reason, comparing the status of safety performance indicators in the time period before and after the outbreak of COVID-19 was completely dependent on the opinion of experts working in the studied industry. Another limitation of this study was the lack of proper cooperation of experts working in the studied industry, and to overcome this problem, a decision-making technique such as AHP was used, which was simple to use for them. And another limitation of this study was the evaluation and collection of experts' opinions. To overcome this problem, we used the Expert choice software, which is a specialized software for evaluating the opinions of experts in the field of MCDM.

According to the calculations of the indicators and their analyses in the period before and after the COVID-19 pandemic, the current condition of the safety performance of the HSE unit was not considerably affected by the outbreak of this disease. However, the investigations performed during the collection of data required to calculate the indicators and evaluate the performance of this unit revealed that some indicators were not sufficiently paid attention to in the studied electricity company. Despite this issue, these indicators play an effective role in HSE performance, workforce health, and reduction of occupational accidents. Some of the mentioned indicators include the percentage of risk assessment, the percentage of corrective measures, and the prevalence of occupational diseases. In these indicators, risk assessment has not been done regularly and annually, and corrective measures have only been taken in the area of safety. Annual risk assessment allows regular access to the pros and cons of an organization's performance. Consequently, corrective measures in all areas of HSE will considerably reduce the number of accidents, improve the health of the workplace and workforce, and ultimately increase the work efficiency of the organization.

Mohammad Ali Esmaeili: Performed the experiments; Wrote the paper. Sajad Zare: Conceived and designed the experiments; Contributed reagents, materials, analysis tools or data. Mohammad Reza Ghotbi ravandi: Analyzed and interpreted the data.

No data was used for the research described in the article.

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.