Qiang Zhi,Li Gong,Jianguo Ren,Meiyu Liu,Zhengshu Zhou,Shuichiro Yamamoto
Element quality indicator: A quality assessment and defect detection method for software requirement specification
Software engineering,SRS Review,Software requirements,Software quality
A software requirements specification (SRS) provides a detailed description of the requirements of a software system that is to be developed. The Element Quality Indicator (EQI) is a novel method to detect defects and assess the quality of an SRS. It does not hinge on review guidelines and is based on the SRS element questions extraction method (EQEM). In this study, we optimized the EQI and carried out a systematic and comprehensive experiment to validate and evaluate its effectiveness. The controlled experiment, which included 60 software engineering students, found that 100% of the subjects identified defects in the SRS using EQI. Moreover, the results indicated that the average number of defects detected with EQI was greater than that of the classical review technique - perspective-based reading. Furthermore, the controlled experiment demonstrated that EQI provides a comparatively objective and accurate evaluation of the quality of the SRS and markedly diminishes the bias in understanding software requirements due to the ambiguity of natural language.
Element quality indicator: A quality assessment and defect detection method for software requirement specification

A software requirements specification (SRS) provides a detailed description of the requirements of a software system that is to be developed. The Element Quality Indicator (EQI) is a novel method to detect defects and assess the quality of an SRS. It does not hinge on review guidelines and is based on the SRS element questions extraction method (EQEM). In this study, we optimized the EQI and carried out a systematic and comprehensive experiment to validate and evaluate its effectiveness. The controlled experiment, which included 60 software engineering students, found that 100% of the subjects identified defects in the SRS using EQI. Moreover, the results indicated that the average number of defects detected with EQI was greater than that of the classical review technique - perspective-based reading. Furthermore, the controlled experiment demonstrated that EQI provides a comparatively objective and accurate evaluation of the quality of the SRS and markedly diminishes the bias in understanding software requirements due to the ambiguity of natural language.

With the rapid advancement of information technology, the demand for electronics and digitalization is increasing day by day, leading to more detailed and comprehensive functional requirements. The quality of a software requirements specification (SRS) has a significant impact on the development and commercialization of a software system, making it an important basis for software development, testing, post-maintenance, and commercial contracts. A high-quality SRS has become the critical first step in software development. In most cases, an SRS is compiled by an analyst, requirements engineer, product owner, or general business employee after performing feasibility and investment benefit analysis based on an understanding of user requirements. An SRS typically encompasses functional and non-functional requirements, design requirements and constraints, testing guidelines, and quality assurance requirements [1]. It serves as a requirement agreement between users and software developers, providing the foundation for starting the design work, and forming the basis for product acceptance after the system development is complete. Non-formal natural language, as the most common way of describing SRS, helps system developers and users communicate more efficiently. However, because of the semantic ambiguity of natural language, it may lead to deviations in reading comprehension and bring about developers’ misunderstanding of certain requirements which diverge from the original intent, resulting in the development of the systems which are flawed or can’t satisfy the actual necessity [2]. A recent survey found that inconsistent, unspecified and incomplete requirements were the cause of project failure [3]. Therefore, the quality of SRS needs to be guaranteed, and it is necessary to review the SRS to detect defects as early as possible in the development stage, which can largely reduce the risk of software development. At the present stage, SRS review methods based on informal natural language can be divided into manual-based and automation-based methods. The most classic manual-based methods are ad hoc reading, perspective-based reading (PBR) [[4], [5], [6]], checklist-based reading (CBR) [7], use case-based reading (UBR) [8,9], and defect-based reading (DBR) [10]. A typical way of automation is natural language processing (NLP) technology [11]. These manual-based methods follow industry standards [12,13] and have their own advantages, and they are still the mainstream methods in the industry. We propose a new concept for SRS review, known as the SRS Element Quality Indicator (EQI) [14]. The main purpose of this concept is to detect defects and assess risks at the early stage of software system development, reducing both the risk and the number of man-hours required for development. Building upon this concept, we optimize the EQI and conduct comprehensive and rigorous experiments to evaluate the effectiveness of the method. This study has the following contributions: 1. We define EQI more precisely and introduce the concept of weight for EQI. This will make the value of EQI more accurate and meaningful. 2. We conduct an empirical study on EQI to verify its effectiveness. Firstly, we carry out a controlled experiment to compare EQI with PBR to confirm its effectiveness in SRS defect detection. Secondly, we conduct a controlled experiment to confirm whether EQI can objectively and accurately assess the quality of the SRS. The rest of the paper is structured as follows: Section 2 briefly introduces the research background of the SRS review methodology. Section 3 defines the EQI methodology. Section 4 describes the setup of controlled experiments to evaluate EQI. Section 5 presents the experimental results. Section 6 discusses the results. Finally, Section 7 provides a summary of this paper.

The semantic ambiguity of natural language leads to the misunderstanding of software requirements, which is a frequent problem in SRS, which will always result in rework as the consequence [15]. Therefore, the review for SRS is the first and very important step in software development. Although research on the application of NLP technology in the field of software requirements has made great progress [[16], [17], [18]], the mainstream method of software requirements review in the industry is still manual review [[19], [20], [21], [22], [23]]. Manual SRS review methods mainly include random review based on reviewer experience (ad hoc Reading), CBR, and PBR. The accuracy of ad hoc-based review is entirely dependent on the experience and competence of the reviewer. In CBR, reviewers use a checklist as a guide, which provides the content to check but little guidance on how to perform the review. In PBR, stakeholders such as software users, software designers and software developers will focus on the content of SRS based on review guidelines from different perspectives, but in order to use PBR and CBR, researchers must prepare review guidelines in advance. PBR-based review guidelines should be tailored according to the positioning of software stakeholders’ own roles, and different scenarios need to be prepared for different roles, which is also known as scenario-based reading [24]. Use case-based reading [8,9] sets concrete use cases for reviewers to check and prompt them to focus on the parts of the software which are most important. A similar review method is defects-based reading [10], which aims to classify defects in SRS into different types, including data type consistency scenario, incorrect functionality scenario, ambiguities or missing functionality scenario. The methods above have their own advantages, some studies have conducted control experiments based on them. Maya Daneva [25] et al. conducted reliability assessments of SRS review guidelines, concluding that the contextualization and operationalization of PBR are more useful to practitioners. Anu et al. [26] studied the effect of SRS reviewers’ ability to extract human error on the effectiveness of SRS defect detection. In the experiments of Laitenberger et al. [27], the PBR scenarios of UML documents were defined in four different perspectives of testers, designers, maintainers and requirements engineers, and each perspective focused on different parts. In an advanced study by Laitenberger et al. [28], they designed and conducted a controlled experiment with UML documents to compare and evaluate the accuracy and effectiveness of PBR and CBR. In addition, there are other review methods that focus on a specific attribute of an SRS: Zhi et al. [29] proposed a value-oriented SRS defect detection approach. Wakimoto [30] proposed a metric for questions and discussions identifying concerns in SRS reviews. Singh et al. [31] proposed a machine learning-based method for automatically validating SRS requirements. Mustaffa et al. [32] proposed a semi-automatic requirements generation tool based on SRSs. Dev Dave et al. [33] proposed an experimental method based on a supervised machine learning model and comparative analysis of fundamental language processing concepts that can be used to separate functional and non-functional requirements from massive data. Effective requirement definition is essential, as the system's ability to meet its requirements--whether explicit or implicit--is one of the most critical factors that determine its success. As big data has increased, it is no longer practical to rely on human analysts to manually examine SRS files to find any implicit requirements (IMRS) connected to the requirements clearly recorded in SRS files, leading to a large body of study on implicit requirements. Emebo Onyeka et al. are committed to the study of implicit requirements. They propose a novel IMRS identification and management method based on the three core technologies of common sense, text mining, and ontology [34], and develop a common sense knowledge (CSK) method to automatically identify implicit requirements (IMRS) from SRS text [35]. For the early identification of implicit requirements, COTIR software tools that integrate common sense knowledge, ontology, and text mining systems are developed [36]. Because some unhandled IMRS can cause software system failure during software development, there are numerous methods for identifying and managing IMRS, many of which emphasize that common sense knowledge in the field of AI knowledge representation will aid in the automatic identification and management of IMRS. Simon Razniewski et al. [37] discuss text-extraction based, multi-model, and converter-based techniques to compile and reinforce such common sense. To address recurring issues in software development, researchers proposed a tried-and-true solution: safe mode. However, secure mode has not yet permeated all aspects of software development practices. Sayem Mohammad Imtiaz et al. [38] proposed a secure software development model based on association rule mining to construct a secure learner-friendly intelligent tutoring system (ITS) to help students understand vulnerable code and guide new developers to secure coding. Although the accuracy of NLP technology has been greatly improved, and it has been applied in the industry, due to some constraints [11], there is still a long way to go before NLP can completely replace manual labor [19]. To sum up, manual review is still the mainstream method of SRS review at present, and related review methods all require a certain amount of development man-hours. The EQI in this study does not require review guidelines, but instead utilizes an SRS element question extraction method (EQEM) to review SRS. EQI can help identify defects in an SRS and serve as a reference indicator to assess its quality.

During software development, stakeholders may ask a lot of questions when discussing the content of SRS. Typically, these questions are not technical in nature, but focus on the requirements themselves. Consequently, the SRS often contains ambiguous elements or requirements that are not clearly expressed, thereby affecting the software design and development. The rework and functional redoing are also common, resulting in a huge waste of time, effort and money. In conclusion, the EQI improves the clarity of SRS elements, thereby enhancing the overall quality of the SRS.

From the perspective of an article, the elements typically include material, theme, structure and language. Regarding the perspective of web page development, structural elements, block-level elements, in-line semantic elements, and interactive elements are all considered necessary. From the document perspective of Web service description language, the most important elements are Type, Import, Message, Port Type, Operation, Binding, and Service. In general, variable reasonable elements can be identified in a document from different perspectives. This paper discusses the various elements that constitute the requirements of an SRS. In the context of designing a library management system, the system can be divided into five parts, namely, users (including login, registration, data management, borrowing rights, etc.), books (including all books, classification, lending situation, borrowing permissions, etc.). This study refers to these parts as elements, which are essentially "requirements analysis objects" that can be further divided based on content attributes. Any doubts about these elements can change the value of EQI. For instance, during the development of the library management system, if the requirements are unclear, the developers might find themselves confused and may need to ask questions, such as: • What permissions do users have? • Can anyone borrow any books? • How long can books be borrowed? • What if the books are not returned on time? • What should the system do if someone wants to borrow a book that is already borrowed? Obviously, a clearer description should be included in the SRS to avoid such questions arising.

The EQI-based SRS evaluation procedure is structured as follows: 1. Understand the background and purpose of system development (software development) and take note of the following fundamental characteristics: • Conduct a feasibility analysis to determine whether a requirement can be tested and verified accurately. • Determine which information requires attention as not all SRS content needs to be verified. • Establish a distinction between functional and non-functional requirements (NFR) during the review process. For the review of NFR, please refer to the experimental data (See Data Availability Statements). 2. Know what an SRS element is and what questions to ask about it. The content of an SRS is comprised of various elements, such as users and borrowing, that can be considered as analysis subjects. Analysis methods can then be chosen based on the attributes of elements. As an illustration, ‘user’ is a noun and ‘borrow’ is a verb. It should go without saying that elements with different attributes should have varied questioning methods. • Questions related to nouns should focus on their states and attributes, for example: 'What permissions do users have?' This question refers to the attributes of the noun ‘user’. • Questions related to verbs should focus on the occasion and consequence of the action. For example: ‘How long can I borrow?’ This question is concerned with the result of the verb ‘borrow’. 3 Element-Based Question Extraction method In this study, elements are extracted by reviewers from sentences or paragraphs of SRSs, upon which questions are posed. However, it is acknowledged that some questions may lack substance. However, some of the questions might not be informative. The EQEM is suggested to standardize the element extraction and questioning procedure of EQI in order to improve the quality of these questions presented by the reviewers. Key elements, decomposed elements, goal items, expected items, and exceptions are the five parts of EQEM. Key components can be extracted from them based on software tasks or software modules. Here is an example of EQEM: Suppose an SRS document reads "Security monitoring equipment must be regularly adjusted at least once during typical time intervals to guarantee its sensitivity and dependability. ", in this case, there are the following four elements that can be extracted. • Security monitoring • Typical time intervals • At least once • Must be regularly adjusted Based on the aforementioned elements, the process of confirming the clarity of the SRS requirements can be structured as follows: according to each decomposed element, check "target items" first, and then make sure whether "expected items" and "exceptions" are clear. For the confirmation method, refer to the "Expectations" and "Exceptions" questioning methods in Table 1. From the above, EQEM can be summarized as follows: [Input] Key elements provided in an SRS. [Procedure]. 1. Combine the sentences or paragraphs in the SRS to break them down into local elements using the provided key elements. 2. After determining the target item from the element, create a form that confirms the requirements (RCF). 3. Based on the RCF, determine whether the ‘expected items’ and ‘exceptions’ for the extracted ‘target items’ are obvious. 4. In the RCF, highlight any unclear ‘expectations’ and ‘exceptions’ (questions raised). [Output] RCFs.

The EQI defines the ambiguity of the SRS in terms of the ratio of the number of EQEM-based questions. A low ratio implies that the SRS content is highly defined and of good quality, whereas a high ratio indicates low definition and poor quality. The clarity of the SRS can be defined as follows: Let SRS (clarity) represent the sum of the key elements of the SRS, let represent the sum of the key elements of the SRS, represent the number of unclear questions in ‘Expectations’ and ‘Exceptions’ in all EQEM-based RCFs, and let represent the number of items that have been identified as all key elements. Now, suppose there are I reviewers in all participate in the review of the SRS based on EQI. In this instance, SRS (clarity) can be described as: In this study, we designed six types of defects in SRS, as shown in Table 2. Furthermore, we divide the EQEM-based questions into four types, as shown in Table 3. In SRS, different types of questions should be assigned different degrees of importance, and different weight values can be considered for each question type, the weights for the questions can be set as (). Then the EQI can also be defined as follows: Let represent the sum of the valid questions from the RCFs, represent the number of each type of valid questions from the RCFs and represent the weight of the corresponding question type, then can be described as: According to (1), (2), as SRS (clarity) and rise, the clarity and quality of the related SRS decline. If there are no issues that requiring clarification in the RCFs, the value of should be 0, indicating that the SRS has a high degree of clarity and quality. The correspondence between the EQI elements and the SRS attributes in the IEEE 830 standard is schematically represented in Fig. 1. As shown in Fig. 1, the properties of the SRS in the outer area can be verified by the EQI elements presented in the inner rectangular area.

The purpose of this experiment is to verify the effectiveness of EQI in assessing the quality of SRS, and to confirm whether EQI can objectively reflect the quality of SRS. Another purpose is to verify whether EQI can effectively detect defects in SRS, and to evaluate the effect of EQI in detecting various defects through comparison with PBR. These purposes have led to following research questions: RQ1: Can subjects raise valid questions based on the proposed EQI review process? RQ2: Do the questions raised by the subjects contain SRS defects? RQ3: How about the ability of EQI to detect defects in SRS compared to PBR? RQ4: Which type of defect is EQI better at detecting? RQ5: Can EQI be used as an effective indicator to represent the quality of an SRS? Based on this experiment, two hypotheses are proposed as follows:Hypothesis 1PBR has multi-perspective review guidelines, but EQI was speculated that it also has some effect in defect detection.Hypothesis 2Since EQI assesses the quality of SRS in terms of asking questions, it was speculated that EQI may detect more representational defects (description problems).The subjects of the experiment are software engineering students who are undergoing a one-year software development and design training. The experimental professionalism should obey the following rules: 1) In the research activities, the experimental data can be obtained from subjects with different review methods, and the effectiveness of the corresponding review methods can be analyzed. 2) Statistical analysis requires both the quantity and quality of experimental subjects. In the designed experiments, the number of subjects and the basic expertise of the subjects should meet the experimental conditions. 3) The subjects were trained before the experiment to ensure that they could master the corresponding experimental knowledge and skills.

Subjects were asked to review the SRS in the experiment and record RCFs (EQI) or defects (PBR). In consideration of the time constraints required for the entire training, the amount of the training handout and the scale of experimental SRS should be moderate. Besides, considering that the students do not have extensive software project development experience, the experiment prepared two SRSs related to the actual needs of the students - Student Performance Management System (A) and Educational Affairs Information System (B). The scales of the two SRSs are close. Students have practical experience in using such systems, and can conduct more objective and detailed evaluations. At the suggestion of software engineers with extensive SRS review experience, similar defect sizes, numbers, and types for SRSs A and B were designed. Moreover, high-quality versions of SRSs C and D which were corresponded to A and B were also prepared. Table 4 presents the experimental SRSs and the corresponding review methods. In the controlled experiment, a total of 60 students majoring in software engineering participated in the experiment. According to the subjects’ academic performance during the four-year university period and their performance during the one-year training, they were divided into six groups of similar levels, with 10 students in each.

To reduce the influence of learning effect on the experimental results, the experiments were designed as shown in Table 5.

1. The organizer of the experiment distributed a questionnaire to all subjects about their software development training experience and coursework during their undergraduate studies. 2. According to the subjects' academic performance during the undergraduate period and the one-year training, the subjects were divided into six groups of equal levels. 

The lecturer introduced and explained the basic concepts and experimental methods of PBR, EQI (EQEM).

The experimental procedure is shown in Fig. 2. • In the first round, four groups of subjects reviewed SRSs A and B, based on ad hoc. • Groups 1 and 2 reviewed SRSs B and A respectively after receiving PBR-related knowledge training. The organizer needed to distribute the relevant SRSs and the designed PBR scenarios to the two groups. • Simultaneously, Groups 3 and 4 reviewed SRSs B and A respectively after receiving the training of EQI-related knowledge. • Groups 5 and 6 reviewed SRSs C and D respectively based on EQI. Notably, both SRSs C and D were of higher quality and more complete expressions than SRSs A and B. • The review time was controlled within 120 min.

Subjects submitted SRS review results and the questionnaire.

How should EQI and PBR be compared? In a PBR review, reviewers should form a team. However, due to the limitation of the number of subjects and the unequal distribution of PBR roles, the individual averages (RQ3) of the two methods will be compared. In the EQI review, not only whether subjects can propose RCFs effectively based on the EQI (RQ1), but also whether SRS defects are pointed out in these RCFs (RQ2) will be investigated. Additionally, which defect categories are more easily detected by EQI (RQ4) will be investigated. Finally, the experimental results of Groups 3, 4 with the ones of Groups 5, 6 to confirm whether the value of EQI can reflect the quality of SRSs (RQ5) will be compared.

Table 6 shows the average number of valid defects found by each method in each round of experiments.

Categorical statistics were performed on each group of subjects. Groups 1 and 2 adopted the review methods of ad hoc and PBR. According to Table 2, the defect types were divided into omission, description problem, ambiguity, feature suggestions, and incorrect. The experimental results of the number of detected defects are shown in Fig. 3, in which the statistics of invalid detection were excluded. Fig. 4 (left) shows the proportion of defect types. Notably, the statistics based on PBR perspectives from designers, testers, and users were also conducted (right in Fig. 4).

Fig. 5 shows the data and statistics for Groups 3 and 4. The left side of Fig. 5 shows the number of each defect type, and the right side shows the corresponding proportion. Notably, the statistics of invalid detection were excluded.

The weights are set based on how the questions impact the SRS. The weight can be set freely according to different needs, in this experiment, the weights were set in this manner: let represent the weight of questions about description, represent the weight of questions about functionality, represent the weight of questions about logic (Questioning the method, procedure, logic in the software development), represent the weight of invalid questions. This study assigns a high weight to questions that have a significant effect on the SRS, and a low weight to minor questions. In this experiment, , , , , were set as 0.2, 0.45, 0.3 and 0.05.

Table 7 presents the data of EQI, including the elements extracted from the RCFs prepared by the subjects based on EQEM and the corresponding questions (expected items and exceptions) for the elements. The experimental results are based on the mean values for each member of the corresponding group. 13 and 14 key elements were set for SRSs A and B respectively, and similarly for SRSs C and D. Subjects need to decompose the key elements based on SRSs, extract the questionable elements, and then mark these elements by asking questions. In Table 7, “questions” refers to the average number of all questions asked by the subjects, and “valid questions” refers to the average number of the remained questions after excluding some invalid questions or merging similar questions asked from the subjects. Additionally, if there are both expected items and exceptions questions to the same element, the number of questions or defects will only be counted once.

The experiment time for all subjects was recorded. However, the time used to prepare review guidelines for PBR was not included in the statistics. The experiment time statistics are shown in Fig. 6.

Table 7 gives the total number of questions including valid questions raised by the members of Groups 3 and 4. Among them, 100% of the subjects raised valid questions, which accounts for 65.7% of the total number of questions. The number of RCF questions based on SRS A (Group 4) is 37.5 on average, and each subject raised 24.2 valid questions on average. The average number of RCF questions based on SRS B (Group 3) is 41.2, and the number of valid questions is 27.5 on average. The experimental results shows that the EQI review based on EQEM can raise RCF questions, and the number and proportion of valid questions are high.

Table 6 presents the number of SRS defects included in the RCFs raised by the members of Groups 3 and 4, and Fig. 5 reveals the types of SRS defects included in the RCFs raised by members of Groups 3 and 4. Besides, each member of both groups pointed out defects in SRSs A and B, and these identified defects correspond to the pre-defined defects or a subset of pre-defined defects. The total number of defects in SRS A detected by Group 4 with EQI is 13.9 on average. Among them, Ⅱ defect (description problem) accounts for the highest proportion, reaching 37.4%. The total number of defects in SRS A detected by Group 3 with EQI is 14.5 on average. Among them, Ⅱ defect (description problem) also accounts for the highest proportion, reaching 42.8%. The experimental results shows that EQI can effectively detect SRS defects.

Table 6 shows the number of defects found by the members of Groups 1 and 2 based on PBR, Fig. 3, Fig. 4 show the corresponding defect types. According to the experimental results, the number of defects detected with PBR is smaller than that with EQI, and the data is statistically significant (P < 0.05). According to Table 6, both EQI and PBR are superior to ad hoc in their defects detecting ability.

According to Table 2 and Fig. 3, Fig. 4, Fig. 5, the defect number and defect type of the two SRSs with EQI and PBR can be obtained respectively. The number of defects detected with EQI is larger than that with PBR. Specifically, EQI shows significant advantages in detecting description problems with significant differences (P < 0.05). However, EQI does not show much advantage in other aspects. Compared with omission and ambiguity, description problems are often considered minor issues.

According to Table 7, the EQI values of SRSs A, B, C, and D can be got respectively. Both SRSs C and D are improved based on SRSs A and B. The experimental result indicates that the EQI values based on SRSs C and D are much smaller than those based on SRSs A and B. This indicates that EQI can be used as an effective indicator to represent the quality of SRS.

In the absence of available SRS review guidelines such as PBR or CBR, EQI supports a detailed and cost-effective SRS review, which can detect SRS defects and give a clear numerical indicator to assess the quality of SRS. Furthermore, combining EQI with other traditional SRS review methods, such as EQI-based PBR or CBR, may improve the review quality of SRS.

In this study, EQI was evaluated by seniors majoring in software engineering. Verifying the validity of an experiment should consider the questions as follows: how can human-centered experiments affect the validity, whether the components of the experiment can be guaranteed, and whether the experimental results can be generalized to the actual working environment. In the experiments in Section 5, the experiments of groups 1–4 were conducted in one day, the second experiment of each group may be affected by the learning effect, but the review method used in the first round is ad hoc, followed by the learning of PBR and EQI. Therefore, it can be inferred that the impact of the learning effect on this experiment is very limited. Furthermore, because the subjects were seniors majoring in software engineering and they were receiving training in software systems development, the learning effect of them was much smaller than that of subjects without basic knowledge of software engineering. Additionally, the SRS used in the experiment is based on the actual development project, and the experimental conclusion is universal.

The experiments in this study exist certain limitations, which may affect the generalization of the conclusions. The subjects of the experiment are students in school, and they do not have rich IT engineering experience, the experimental results may not reflect the actual situation well. In addition, the use of guidelines in PBR experiments may have a huge impact on the experimental results, thus the evaluation of PBR guidelines is also important. Besides, the EQI value will be affected by the number of freely defined elements, and it can only roughly reflect the quality of the SRS. As a result, it is difficult to compare the pros and cons of different SRSs through the EQI value. Additionally, EQI takes approximately 60% more time than PBR. All experimental data in this study are accessible, please refer to the Data Availability Statement section. In future work, EQI will be considered in practical projects, with engineers as subjects. EQI will also be compared with enterprise-level CBR to further verify the usability and effectiveness of EQI (CBR is still the prevailing method of enterprise-level software requirements and defect reviews [39]).

This study optimized the EQI and carried out a systematic and comprehensive experiment to validate and evaluate its effectiveness. In EQI, reviewers need to review the SRS based on Element-Based Question Extraction method (EQEM). To verify the effectiveness of the improved EQI, a controlled experiment involving 60 students majoring in software engineering was conducted. The experimental results show that 100% of the subjects raised valid questions in experimental SRSs. These valid questions can significantly reduce the deviation of understanding the software requirements and point out the defects in the SRS. In addition, the number of defects detected by EQI is greater than that detected by PBR, and the experimental results are statistically significant (P < 0.05). Unfortunately, EQI doesn’t show much advantage beyond description problems in detecting SRS defects. The experimental results also show that EQI can be used as an effective indicator to represent the quality of SRS. Finally, EQI can be considered combined with other general review methods (such as CBR) to improve the review quality of SRS. Future work mainly includes the following points: 1) Verify the impact of software engineers' experience on EQI; 2) Verify the effectiveness of the two methods of CBR and EQI; 3) Study the practicability and effectiveness comparison of EQI and NLP tools for SRS; 4) Evaluate the quality of developed software based on SRSs with different EQI values, the proportion of rework in development man-hours, and the degree of satisfaction, etc.; 5) Develop the SRS review system based on EQI.

Qiang Zhi: Conceived and designed the experiments; Performed the experiments; Analyzed and interpreted the data; Contributed reagents, materials, analysis tools or data; Wrote the paper. Li Gong: Analyzed and interpreted the data; Wrote the paper. Jianguo Ren: Shuichiro Yamamoto: Conceived and designed the experiments. Meiyu Liu: Zhengshu Zhou: Performed the experiments.

Data associated with this study has been deposited at https://github.com/Zhi-JSNU/EQI.

This work was partially supported by the Scientific Research Support Project of 10.13039/501100004030Jiangsu Normal University under Grant No. 21XSRX006, and partially supported by the 10.13039/501100004608Jiangsu Province Natural Science Foundation under Grant No. BK20201462, and partially supported by the 10.13039/100016800Natural Science Foundation of Xuzhou City under Grant No. KC21018.

All experiments in this study were conducted in accordance with the established ethical guidelines, and the informed consent of the participants was obtained.

The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Qiang Zhi reports financial support was provided by Jiangsu Normal University. Jianguo Ren reports financial support was provided by Jiangsu Province. Jianguo Ren reports financial support was provided by Xuzhou City. Zhengshu Zhou is employed by Hitachi Ltd.