--- abstract: '*Context*: Coordination is a fundamental tenet of software engineering. Coordination is required also for identifying discovered and disclosed software vulnerabilities with Common Vulnerabilities and Exposures (CVEs). Motivated by recent practical challenges, this paper examines the coordination of CVEs for open source projects through a public mailing list. *Objective*: he paper observes the historical time delays between the assignment of CVEs on a mailing list and the later appearance of these in the National Vulnerability Database (NVD). Drawing from research on software engineering coordination, software vulnerabilities, and bug tracking, the delays are modeled through three dimensions: social networks and communication practices, tracking infrastructures, and the technical characteristics of the CVEs coordinated. *Method*: Given a period between 2008 and 2016, a sample of over five thousand CVEs is used to model the delays with nearly fifty explanatory metrics. Regression analysis is used for the modeling. *Results*: The results show that the CVE coordination delays are affected by different abstractions for noise and prerequisite constraints. These abstractions convey effects from the social network and infrastructure dimensions. Particularly strong effect sizes are observed for annual and monthly control metrics, a control metric for weekends, the degrees of the nodes in the CVE coordination networks, and the number of references given in NVD for the CVEs archived. Smaller but visible effects are present for metrics measuring the entropy of the emails exchanged, traces to bug tracking systems, and other related aspects. The empirical signals are weaker for the technical characteristics. *Conclusion*: Software vulnerability and CVE coordination exhibit all typical traits of software engineering coordination in general. The coordination perspective elaborated and the case studied open new avenues for further empirical inquiries as well as practical improvements for the contemporary CVE coordination.' address: - 'Department of Future Technologies, University of Turku, FI-20014 Turun yliopisto, Finland' - 'Pori Department, Tampere University of Technology, P.O. Box 300, FI-28101 Pori, Finland' author: - Jukka Ruohonen - Sampsa Rauti - Sami Hyrynsalmi - Ville Leppänen title: A Case Study on Software Vulnerability Coordination --- vulnerability, open source, coordination, social network, CVE, CWE, CVSS, NVD, MITRE, NIST Introduction ============ Software bugs have a life cycle.[^1] In a relatively typical life cycle, a bug is first introduced during development with a version control system, then reported in a bug tracking system, and then again fixed in the version control system. Also software security bugs, or vulnerabilities, follow a similar life cycle. Unlike conventional bugs, however, vulnerabilities often require coordination between multiple parties. Coordination is visible also during the identification and archiving of vulnerabilities with unique CVEs. There are four ways to obtain these universally recognized vulnerability identifiers. For obtaining a CVE, (a) an affiliation with an assignment authority (such as Mozilla or Microsoft) is required, but coordination may be done also by (b) contacting such an authority, making (c) a direct contact to the MITRE corporation, or (d) using alternative channels for public coordination @Seifried17. During the period observed, the public channel referred to the `oss-security` mailing list. The typical workflow on the list resembled the simple communication pattern illustrated in Fig. \[fig: osec\]. ![Coordination via `oss-security` (2008 – 2016)<span data-label="fig: osec"></span>](fig_ist_osec.pdf){width="\linewidth" height="3.2cm"} Coordination is one of the major inhibitors of software development @Johnson16a. In the ideal case, the timeline from the leftmost circle would be short in terms of the timestamp that is recorded for a reported vulnerability to appear in NVD. In the future, robots might do the required coordination and evaluation work, but, recently, the time lags have allegedly been long. These delays have also fueled criticism about the whole tracking infrastructure maintained by MITRE and associated parties (for a recent media take see @Leyden17). The alleged delays exemplify a basic characteristic of coordination: the presence of coordination often appears invisible to outsiders until coordination problems make it visible @MaloneCrowston94. The recent criticism intervenes with other transformations. These include the continuing prevalence of so-called vulnerability markets @Ruohonen16RCIS, including the increasingly popular crowd-sourcing bug bounty platforms @Laszka16 [@Ruohonen18WEIS]. Governmental interests would be another example @Ablon17. The CVEs assigned are commonly used also to track exploitable vulnerabilities on criminal underground platforms @Allodi17c. A further example would be the use of social media and the resulting information leakages of the sensitive information coordinated @Leyden17 [@Sabottke15; @Syed18]. These and other factors have increased the volume of discovered vulnerabilities for which CVEs are requested. Reflecting these challenges, the coordination of CVEs through the mailing list ended in February 2017. To some extent, it is reasonable to conclude that `oss-security` lost its appeal as an efficient coordination and communication medium for CVE identifiers. To answer to the challenges, new options were provided for public CVE tracking @Openwall17a, and larger reforms were albertbarron@example.org. Motivated by these practical challenges, the paper examines the CVE coordination through the `oss-security` mailing list between 2008 and 2016. The paper continues the earlier case study @Ruohonen17IWSMMensura by explicitly focusing on the timeline (b) in Fig. \[fig: osec\]. In other words, the present paper observes the time between CVE assignments on the mailing list and the later publication of these identifiers in NVD. Although the previous study revealed interesting aspects about social networks for open source CVE coordination, it also hinted that the coordination delays might be difficult to model and predict. Modeling of the delays is the goal of the present paper. To empirically model the delays, nearly fifty metrics are derived in this paper for proxying three dimensions of coordination: (i) social networks and communication practices, (ii) tracking infrastructures within which the vulnerabilities had often already appeared prior to `oss-security`, and (iii) the technical characteristics of the vulnerabilities coordinated. Before elaborating these dimensions in detail, the opening Section \[section: background\] discusses the background and introduces the case studied. The research approach used to study the case is described in Section \[section: approach\]. Results are presented in Section \[section: results\] and further discussed in Section \[section: discussion\]. Background {#section: background} ========== Software engineering coordination is a fundamental part of the grand theories in the discipline @Johnson16a. The following discussion will briefly motivate the general background by considering some of the basic coordination themes in terms of software vulnerabilities and their coordination. After this motivation, the case studied is elaborated from a theoretical point of view. The discussion ends to a formulation of three research questions for the empirical analysis. Vulnerability Coordination -------------------------- Coordination can be defined as a phenomenon that originates from *dependencies* between different *activities*, and as a way to *manage* these dependencies between the activities @HowisonCrowston14. Software engineering is team work. As team work implies social dependencies between engineers, there is always some amount of coordination in all software engineering projects involving two or more engineers. There are also technical dependencies in all software products. Early work on software engineering coordination focused on the reduction of such technical dependencies in order to improve task allocation and work parallelism @Blincoe15. However, not all technical dependencies can be eliminated. For this reason, later work has often adopted a *socio-technical* perspective for studying software engineering coordination. The augmentation of technical characteristics with social tenets is visible at a number of different fronts. One relates to the research questions examined. Examples include synchronization of work and milestones, incremental integration of activities, arrangement of globally distributed teams, frequent deliveries, and related project management processes @Lee13 [@McChesney97; @Paasivaara03]. Another relates to the research methodologies used. While interviews and surveys are still often used @Blincoe15 [@Lee13; @Paasivaara03], techniques such as social network analysis have become increasingly common for examining software engineering coordination @Bird11 [@Sierra18]. Despite of these changes, one fundamental theoretical premise has remained more or less constant over the years. Coordination requires *communication*, and coordination failures are often due to communication problems @Blincoe15 [@Wolf09b]. The consequences from coordination failures vary. Typical examples include schedule slips, duplication of work, build failures, and software bugs @Bird11 [@Blincoe15]. Communication obstacles, coordination failures, and the resulting problems are well-known also in the security domain. A good example would be incident and abuse notifications for vulnerable Internet domains: it is notoriously difficult to communicate the security issues discovered to the owners or maintainers of such vulnerable domains @CetinVanEeeten17. Analogous problems have been prevalent in *vulnerability disclosure*, which refers to the practices and processes via which the vulnerability discoverers make their discoveries known to the vendors whose products are affected either directly or via a third-party coordinator. Although there have long been recommendations and guidelines @Christey02, it is still today often difficult to communicate vulnerability discoveries to vendors @Ring15. A substantial amount of effort may be devoted to coordinate the disclosure and remediation of high-profile vulnerabilities, but difficulties often occur for more mundane but no less important vulnerabilities. Some vendors are reluctant to participate in vulnerability disclosure—and some vendors even avoid patching their software products altogether. Even legal threats are still today not unheard of. While vulnerability disclosure is a prime example of coordination (failures) in the software security context, it has only a narrow scope. As Steven M. Christey—the primary architect behind the current CVE tracking—has argued, the term *vulnerability coordination* is preferable because vulnerability disclosure only captures a small portion of the activities required to handle and archive vulnerabilities @Christey13. In fact, vulnerability coordination is a relatively frequent activity among the integrative software engineering activities carried out by open source developers @Adams16. Bug reports must be handled in a timely manner for security issues. Evaluation is needed to assess the versioned products affected. Of course, also fixes must be written, but fixes may further require careful backtracking within version control systems, debugging, reviews from peers, testing done by peers and users alike, and coordination between business partners or third-party open source software projects. Then, erratas, security advisories, or other notes must be written, identifiers must be allocated to uniquely track the notes and to map these to other identifiers, preparations may be required for answering to user feedback, and so forth. These are all examples of the software engineering activities that are typically present in the software vulnerability context. For empirical software engineering research, the identifiers used to track vulnerabilities are particularly relevant. The most important identifier is CVE, in terms of both research and practice. For open source projects, having a single canonical identifier helps developers and users to coordinate their efforts. Given the inconsistency issues affecting open bug trackers @Romo14 [@TianHassan16], CVEs help at addressing questions such as: "well it sounds like this one but maybe it's that other one?" @Seifried17. For a security professional, CVEs are ubiquitous entries in a curriculum vitae. For research, CVEs provide the starting point for connecting the distinct engineering activities into a coherent schema. While there is an abundant amount of existing research operating with CVE-based research schemas, thus far, only one attempt @Ruohonen17IWSMMensura has been made to better understand the primary schema; software and security engineering coordination is required for CVEs themselves. CVE Coordination Through a Mailing List --------------------------------------- Software development teams tend to produce software products that reflect the communication structure of the teams, to paraphrase the famous law formulated by Melvin E. Conway @Conway68. Although the law is not directly applicable to the context of CVE coordination, the `oss-security` case supports a weaker variant of the same basic assertion. This variant might be defined as a statement that a communication structure tends to reflect the *type* of software engineering work and the *medium* used for communication. The type of work done and the medium both characterize also the social network structure of the CVE coordination through the mailing list. Both limit also the applicability of common theoretical interpretations. For instance, knowledge sharing is a typical way to motivate research on open source social networks @Kuk06 [@Licorish14; @Toral09], but sharing of *knowledge* was not the main purpose of the `oss-security` mailing list during the period studied. The primary purpose of the list was to coordinate the assignment of CVE identifiers for discovered and usually already disclosed software vulnerabilities. This coordination was done by explicitly requesting CVE identifiers, which were then assigned by MITRE affiliates based on a brief evaluation. In terms of communication practices, this coordination practice often culminated in short replies, such as "use CVE-2016-6526" @Openwall16c, made by MITRE affiliates to previously posted CVE requests. Participants typically kept MITRE's `albertbarron@example.org` email address in the carbon copy field when they posted a request, further using a subject line that identified the message as a request. While longer discussions were not unheard of, these communication practices indicate that knowledge sharing has only limited appeal for framing the case theoretically. Exchanging information about abstract identifiers does not necessarily imply thorough discussions about the technical details of the vulnerabilities coordinated. In other words, data does not equate to information, and information does not equal knowledge. The coordination work through the list produced clear core social network components @Ruohonen17IWSMMensura. This observation is classical in the open source context @Conaldi12 [@Crowston06; @Licorish14; @Toral10], but the contextual interpretation is still relevant. Due to the way CVEs are assigned, the cores centered to MITRE affiliates. By using a network representation described later in detail, the core social network components are illustrated in Fig. \[fig: core\]. Excluding the use of MITRE's email address without an explicit sender, there are two identifiable individual participants with degrees higher than $800$, meaning that these two participants both coordinated at least eight hundred CVEs each. Both participants are MITRE affiliates. What is more, there are only a few CVEs that link the three core participants to each other. This observation indicates that task allocation and related coordination techniques apply also to the case studied. ![The Core Participants (one hop from the labeled vertices)<span data-label="fig: core"></span>](plots/fig_ist_example.pdf){width="\linewidth" height="6cm"} These core social network components are a good example of so-called communication brokers who help at resolving coordination obstacles in software engineering @Wolf09a. Such brokers integrate information from multiple sources. This *integrative* engineering work often leads to a social network structure with relatively high level of centrality, low level of clustering, and star-like centers within which the integrative work is located @Parraguez15. This theoretical characterization applies well also to the social networks for the CVE coordination through the mailing list. \[t!\] ![image](fig_ist_hypotheses.pdf){width="14cm" height="6.5cm"} Thus, the type of software engineering work carried out is an important factor characterizing the corresponding social network structures. In terms of development mailing lists, non-core participants may post as many messages as core participants @Crowston17, but this observation does not generalize to all cases @Guzzi13, including the case studied. Although coordination requires communication, coordination tends to result in different network structures than communication required for other work (or leisure) activities. A communication medium used for coordination presumably also affects the emerging network structure. For instance, social networks constructed from bug tracking systems indicate that many individuals may report bugs even though the actual development may be concentrated to a small core group @Zanetti13a. A communication medium also places restrictions over who can participate, how much can be communicated, and what is communicated @Bird11 [@PooCaamano17]. While `oss-security` is open for anyone to participate, the communication volume and the content of messages exchanged are both important for further elaborating the case. The integrative coordination work done on the list reflected not only social but also loose technical dependencies between different tracking infrastructures. The classical consumer-producer abstraction for software engineering coordination @MaloneCrowston94 [@McChesney97] is useful for framing these technical dependencies theoretically. According to this abstraction, there exists a *prerequisite constraint*: the work activity of a producer must be usually completed before work can start on the consumer side @HowisonCrowston14 [@MaloneCrowston94]. In terms of the case studied, the participants (producers) who requested CVEs provided sufficient technical information to justify the requests for the MITRE affiliates (consumers). If the information was sufficient for a request, the prerequisite constraint was satisfied, and the vulnerability in question later appeared in NVD with the CVE requested. Instead of in-depth knowledge sharing, the sufficient information was usually delivered via hyperlinks to other open source tracking infrastructures within which a given vulnerability had already been discussed. Consequently, the discovery, disclosure, and patching of the vulnerabilities coordinated had almost always occurred before information appeared on the mailing list. Research Questions ------------------ The preceding discussion motivates three questions worth asking about the delays between CVE requests on the mailing list and the later appearance of these identifiers in the central tracking database. Given the medium used and the type of software engineering coordination done, these questions can be framed by separating the two terms in the concept of socio-technical coordination. The first question addresses the social dimension: 1. <span>*Have the CVE coordination delays been affected by the social networks and communication practices between the participants on the `oss-security` mailing list?*</span>\[rq: social\] The remaining two questions address the technical dimension in socio-technical coordination. Given the integrative work done and the consumer-producer abstraction, it is worth asking the following question about the delays: 1. <span>*Did traces to other tracking infrastructures affect the coordination delays during the period observed?*</span>\[rq: infrastructure\] The third and final research question approaches the technical dimension from a more direct perspective: 1. <span>*Were the coordination delays affected by the technical characteristics of the vulnerabilities coordinated?*</span>\[rq: technical\] The analytical meaning behind the three questions is illustrated in Fig. \[fig: hypotheses\]. It should be noted that causal inference is not attempted; hence, no arrows are drawn in the figure between the three explanatory dimensions. Approach {#section: approach} ======== In what follows, the research approach taken to study the CVE coordination delays is elaborated by introducing the empirical dataset, the operationalization of the delays, and the construction of the social networks observed. After this machinery has been installed, the approach is further elaborated by describing the explanatory metrics and the statistical methodology used to model the delays. Data {#subsec: data} ---- The dataset is compiled from two sources. All email messages posted on `oss-security` were obtained from the online Openwall archive @Openwall16a. These messages are cross-referenced with CVE identifiers to the second data source, NVD @NVD17a. The sampling period runs from the first welcome message in February 2008 to the emails posted in December 31 2016. This sampling interval corresponds with the historical period during which the mailing list was used for CVE assignments. There were a little over sixteen thousand messages posted during this time interval. For each message delivered in the hypertext markup language (HTML) format, a twofold routine is used for pre-processing the archival data (see Fig. \[fig: pre-processing\]). The first pre-processing subroutine extracts the actual messages from the HTML markup, further omitting all lines referring to quotations from previous messages posted on the mailing list. To some extent, also forwarded messages are excluded based on simple but previously used heuristics @Wu05. Due to the markup language, this pre-processing routine is likely to yield some inconsistencies. However, the consequences for the empirical analysis should be relatively small because no attempts are made to pre-process email threads. This choice is largely imposed by the use of the online archive. In particular, the fields `Message-ID`, `In-Reply-To`, and `References` are not delivered via the online interface, which prevents the use of common (meta-data) techniques for parsing email threads @Guzzi13 [@Wang14]. Although content-based alternatives exist for reconstructing threaded email structures @Dehghani12 [@Schmid14], only the senders of emails are considered in order to improve data quality and to simplify the data processing. ![Pre-processing in a Nutshell<span data-label="fig: pre-processing"></span>](fig_ist_preprocess.pdf){width="\linewidth" height="5cm"} The second subroutine is used for identification. The `From` email header field is used for identifying the individual participants by using the first match from a simple Python regular expression. Although the matching itself is simple, it should be emphasized that all senders are identified according to their names rather than their addresses. This option is often preferred because individuals' email addresses tend to vary @Bird11 [@Nia10; @Toral09]. The choice is again also imposed by the online archive, which obfuscates the email addresses for spam prevention and related reasons. Consequently, additional pre-processing techniques @Ngamkajornwiwat08 for mapping identified names to addresses cannot be used. Furthermore, Levenshtein's @Levenshtein66 classical distance metric between two names, say $L(x, y)$, is a decent choice for accounting small inconsistencies @Zangerle13. If $s_1$ and $s_2$ denote two full names with lengths $l_1$ and $l_2$, a similarity score is computed through $\delta = 1.0 - L(s_1, s_2) / \max\lbrace l_1, l_2 \rbrace$ @Christen12. If the scalar $\delta$ then exceeds a threshold value $0.8$, two names are taken to refer to the same individual. Although the threshold is subjective, it captures some typical cases, such as "John Doe" who occasionally writes his name as "John, Doe" when sending emails. The few outlying cases that exceeded the $\delta = 0.8$ threshold were further evaluated manually. This additional check revealed no obvious approximation mistakes. In addition, "Christey, Steven M." and "Steven M. Christey" were merged manually. Also CVEs are identified with a regular expression, which is a typical approach for searching vulnerabilities from heterogeneous sources @Allodi17c [@LinaresVasquez17]. The expression takes into account both the old and deprecated candidate (CAN) syntax as well as the recently made syntax change that allows an arbitrary amount of digits in the second part of CVE identifiers @MITRE15a [@MITRE15b]. Even though forwarded messages and direct quotations to previous emails are approximately excluded, it should be noted that there can be many-to-many relations between messages and CVEs. For instance, a lengthy security advisory posted on the list may contain a large amount of individual CVE-referenced vulnerabilities. Such cases are included in the sample. Finally, only those CVEs are qualified that have also valid entries in NVD. Most of the invalid entries in the database refer to identifiers that were assigned but which were later rejected from inclusion to the database. Given that these disqualified CVE assignments cause different biases @Christey13, all rejected identifiers were excluded from the empirical sample. The matching of these invalid entries was done by searching for the string `REJECT` in the summary field provided by NVD. Coordination Delays ------------------- There are multiple ways to measure the efficiency of software engineering coordination @Lee13. In the context of software bugs in general, a good example would be the validity of bug reports. Bug tracking infrastructures typically reserve multiple categories for classifying invalid reports. These categories include classes for already fixed bugs, irreproducible bugs, and duplicate reports, among other groups. If a large amount of bug reports end up into such classes, coordination would be generally inefficient. As coordination deals with dependencies, it is no surprise that also social network structures of bug reporters have been observed to affect the probability of valid bug reports @Zanetti13a. Although the generalizability of this observation seems limited @Bhattacharya11, there are good reasons to suspect that an analogous effect is present in the software vulnerability context. Because attribution is particularly important for vulnerabilities and monetary compensations are relatively common, it is no wonder that invalid—or even outright fake—reports have been a typical menace affecting vulnerability coordination @Christey13 [@Laszka16]. For this reason, vulnerability reports from established discoverers likely increase the validity of the reports as well as the trust placed on the reports. This trust provision is also reflected on the CVE assignment authorities granted for a few individual security researchers. Due to the sensitiveness of the information coordinated, the availability of open data is limited about the internal vulnerability tracking systems used by software vendors, MITRE, and other actors. Many open source projects also limit the visibility of security bugs, or otherwise try to constrain the exposure of public information. These data limitations have affected also the ways to quantify longitudinal vulnerability information. For instance, the efficiency of vulnerability disclosure can be approximately measured with a time difference between a disclosure notification sent to a software vendor and the vendor's reply @Arora10 [@Ruohonen16AICCSA]. Analogously, patch release delays can be measured by fixing the other endpoint to the dates on which vendors released patches for the vulnerabilities disclosed to the vendors @McQueen09 [@Temizkan12]. Similar delays can be measured also in terms of initial bug reports and later CVE assignments based on the reports @Wijayasekara12. Further examples include the timing of security advisories @Ruohonen17COMSIS, the dates on which signatures were added to intrusion detection and related systems @Bilge12, and exploit release dates for known vulnerabilities @Ablon17 [@Allodi17c; @Bozorgi10]. All these different dates convey different viewpoints on the coordination of vulnerabilities. In this paper, analogously, the empirical interest relates to the following per-CVE time differences (in days): $$\label{eq: delays} y_i = T_{\textmd{NVD}_i} - T_{\textmd{\texttt{oss-security}}_i} ,$$ given $$\label{eq: delays restriction} T_{\textmd{NVD}_i} \geq T_{\textmd{\texttt{oss-security}}_i} \quad\textmd{for all}\quad i = 1, \ldots, n$$ vulnerabilities observed. The timestamp $T_{\textmd{NVD}_i}$ records the date on which the $i$:th CVE was first stored to NVD. The second timestamp $T_{\textmd{\texttt{oss-security}}_i}$ refers to the *earliest* date on which this CVE was posted on the mailing list. The restriction in excludes already archived CVEs that were later discussed on the mailing list. Thus, the integer $y_i \geq 0$ approximates the length of the path (b) in Fig. \[fig: osec\]. In theory, also the path (a) in Fig. \[fig: osec\] could be measured, but the data from the online archive makes it difficult to identify the initial CVE requests. This limitation does not affect the validity of the delay metric, but it does affect the interpretation given for it. The metric in approximates the time delays between the initial CVE assignments done through the `oss-security` mailing list and the usually later appearance of the requested CVEs in NVD. Therefore, $y_i$ focuses on the *internal* coordination done by MITRE affiliates, the NVD team, and other actors involved in the coordination. One way to think about this internal coordination is to consider the delay metric as an *indirect* quantity for measuring how fast work items in a *backlog* or an *inventory* are transferred to complete deliveries @MaloneCrowston94 [@Ruohonen19ACI]. Due to data limitations, however, it is currently neither possible to observe the internal within-MITRE (or within-NVD) coordination directly nor to explicitly measure the work items in the CVE backlog. Therefore, the delay metric used provides a sensible but not entirely reliable way to approximate the typical coordination delays that affected one particular CVE coordination channel. Bipartite Email and Infrastructure Networks ------------------------------------------- The socio-technical coordination and communication characteristics are proxied by observing so-called task-based @Wolf09a or person-task @Sierra18 social networks. Thus, the underlying social network structure is bipartite, meaning that there are two types of vertices. An edge connecting any two vertices always contains both types; there are no edges that would connect a vertex of one type to a vertex of the same type. In addition, the network structure is unweighted and undirected. More formally: $$\label{eq: social network} G_s = (P \cup A, E) ,$$ where $G_s$ denotes an observed social network constructed from the messages that were posted on the mailing list, from the first message posted in 2008 to the last message in 2016. The two disjoint vertex sets $P$ and $A$ refer to participants and CVE identifiers, respectively. If a participant $p \in P$ has sent a message containing a CVE identifier $a \in A$, an undirected edge, $(p, a) = (a, p)$, is present in the edge set $E$. Therefore, individual participants are linked together through CVE identifiers (see Fig. \[fig: example networks\]). Due to the bipartite structure, it holds that $P \cap A = \emptyset$ and $$\label{eq: degrees} \vert E \vert = \sum_{p \in P}\operatorname{Deg}(p) = \sum_{a \in A}\operatorname{Deg}(a) ,$$ where $$\label{eq: deg} \operatorname{Deg}(x) = \sum_{(x, y) \in E} y .$$ Furthermore, another network is constructed from the uniform resource locators (URLs) embedded in the hyperlinks present in the emails that contained also CVEs. The structure is again undirected, unweighted, and bipartite: $$\label{eq: domain network} G_d = (D \cup B, F) ,$$ where $D$ denotes a set of domain names extracted from the URLs sent by participants in $P$, $B$ denotes another set of CVEs, and $F$ is an edge set containing edges from the domain names to the CVEs. If a participant $p \in P$ sent an email containing a $b \in B$ and a domain name $d \in D$ extracted from a URL in a hyperlink, an undirected edge $(b, d) = (d, b)$ is present in the set $F$. Thus, CVEs are linked also to domain names through the participants who posted messages containing both CVE identifiers and hyperlinks. Therefore, $B \subseteq A$ and $n = \vert A \vert \geq \vert B \vert$ because the subset of CVE identifiers stored to $B$ are required to also have mappings to domain names. ![Example Networks<span data-label="fig: example networks"></span>](fig_ist_networks.pdf){width="\linewidth" height="4.5cm"} The network $G_s$ is a *social* network in the traditional sense; even though participants are linked to each other through abstract identifiers, the participants are still *human* beings. The network $G_d$, in contrast, resembles more the so-called domain name system graphs within which domain names are connected to each other via Internet protocol (IP) addresses or by other technical relations @Ruohonen17CIT [@Stevanovic16]. Consequently, it would be possible to manipulate $G_d$ by resolving the addresses of the domain names or by considering only the second-level domain names @Ruohonen16MESSA. Given the historical context, however, no attempts are made to resolve the domain names, many of which are nonexistent today. Instead, only three semantic validation checks are enforced: (a) the length of each $d \in D$ is asserted to be at least three characters; (b) each $d$ is required to contain a dot character; (c) and no entry in $D$ is allowed to refer to a semantically valid IPv4 address. Explanatory Metrics {#subsec: explanatory metrics} ------------------- The three research questions are evaluated by regressing the coordination delays $y_1, \ldots, y_n$ against the metrics enumerated in Table \[tab: metrics\]. Six *models* ($\operatorname{M}$) are used for the statistical computations; the integer $k$ denotes the cumulative number of metrics included in the consecutively estimated models, including the intercept. The table shows also the scale of the metrics; there are a few *continuous* ($\operatorname{\mathcal{C}}$) metrics but most of the metrics are *dichotomous* ($\operatorname{\mathcal{D}}$) dummy variables. The metrics enumerated can be further elaborated according to the models within which these first appear. \[t!\] $\operatorname{RQ}_i$ $\operatorname{M}_j$ Name Scale Description ----------------------- ---------------------- ---------------------- ------------------------------ ----------------------------------------------------------------------------------------------- – $\operatorname{M}_1$ $2009, \ldots, 2016$ $\operatorname{\mathcal{D}}$ True for CVEs assigned on a given year according to $T_\textmd{\texttt{oss-security}}$. $(k = 21)$ Feb, $\ldots$, Dec $\operatorname{\mathcal{D}}$ True for CVEs assigned on a given month according to $T_\textmd{\texttt{oss-security}}$. WEEKEND $\operatorname{\mathcal{D}}$ True for CVEs assigned on Saturday or Sunday according to $T_\textmd{\texttt{oss-security}}$. $\operatorname{RQ}_1$ $\operatorname{M}_2$ SOCDEG $\operatorname{\mathcal{C}}$ The degree of all CVE-labeled vertices in $G_s$. $(k = 25)$ MITREDEV $\operatorname{\mathcal{D}}$ True for CVEs in the neighborhood of the three labeled vertices in Fig. \[fig: core\]. MSGSLEN $\operatorname{\mathcal{C}}$ The amount of characters divided by $100$ in the emails mentioning a CVE. MSGSENT $\operatorname{\mathcal{C}}$ For a given CVE, the Shannon entropy of the emails mentioning the CVE. $\operatorname{RQ}_2$ $\operatorname{M}_3$ INFDEG $\operatorname{\mathcal{C}}$ The degree of CVE-labeled vertices in $G_d$ (zero for any $b \in B$ but $b \not\in A$). $(k = 31)$ NVDREFS $\operatorname{\mathcal{C}}$ Number of reference URLs given in NVD for the CVEs observed. VULNINF $\operatorname{\mathcal{D}}$ True for CVEs linked to vulnerability infrastructures via $G_d$. BUGS $\operatorname{\mathcal{D}}$ True for CVEs linked to bug tracking and related systems via $G_d$. REPOS $\operatorname{\mathcal{D}}$ True for CVEs linked to version control and related systems via $G_d$. SUPPORT $\operatorname{\mathcal{D}}$ True for CVEs linked to vendors' support channels via $G_d$. $\operatorname{RQ}_3$ $\operatorname{M}_4$ IMPC $\operatorname{\mathcal{D}}$ True for CVEs having a partial or a complete impact on confidentiality $(k = 34)$ IMPI $\operatorname{\mathcal{D}}$ True for CVEs having a partial or a complete impact on integrity IMPA $\operatorname{\mathcal{D}}$ True for CVEs having a partial or a complete impact on availability $\operatorname{M}_5$ EXPNET $\operatorname{\mathcal{D}}$ True for CVEs that may be exploited only with a network access. $(k = 37)$ EXPCPLX $\operatorname{\mathcal{D}}$ True for CVEs with a high or a medium access complexity for exploitation. EXPAUTH $\operatorname{\mathcal{D}}$ True for CVEs that can be exploited only through authentication. $\operatorname{M}_6$ CWE-264 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of permissions, privileges, and access controls. $(k = 47)$ CWE-119 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of buffer-related bugs. CWE-79 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of cross-site scripting (XSS). CWE-20 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of input validation. CWE-200 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of information leaks. CWE-399 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of resource management bugs. CWE-189 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of numeric bugs. CWE-352 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of cross-site request forgery (CSRF). CWE-89 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of structured query language (SQL) injection. CWE-310 $\operatorname{\mathcal{D}}$ True for CVEs in the domain of cryptographic bugs. ### Control Metrics Temporal aggregation of social network data should be done only after a careful consideration @Nia10. Previous work in the `oss-security` context also indicates that the social networks for the open source CVE coordination changed over the years @Ruohonen17IWSMMensura. In contrast to what has been claimed to characterize open source projects @Ngamkajornwiwat08, the coordination effort did not diminish over time. In fact, the list became more popular, which resulted in more participants and more coordinated CVEs. These transformations also changed the social network structure, although the core of the network structure remained centered to MITRE affiliates. A notable change occurred also in the structure of this network core: Kurt Seifried from Red Hat joined the CVE editorial board @MITRE15c and took an active role also on `oss-security`. This activity reduced the reliance on a single MITRE affiliate for the CVE coordination through the list, resulting in the network cores illustrated in Fig. \[fig: core\]. Instead of explicitly modeling these changes through separate annual social networks—as has been typical in applied social network research @Ruohonen16AICCSA [@Ruohonen17IWSMMensura; @Toral10; @Zanetti13b], the longitudinal dimension is approximately controlled with eight dummy variables. Each of these is zero for the $i$:th identifier unless the CVE identifier was assigned on a given year between 2009 and 2016 according to the corresponding $T_{\textmd{\texttt{oss-security}}_i}$ timestamp used in . The initial year 2008 acts as the reference category against which the effects of these annual dummy variables are compared against. Given the fairly complex time series dynamics arising from the archiving of vulnerabilities to NVD and related tracking infrastructures @JohMalaiya17 [@Tang19], a further set of eleven dummy variables is included for controlling potential monthly variation in the coordination delays. The $T_{\textmd{\texttt{oss-security}}_1}, \ldots, T_{\textmd{\texttt{oss-security}}_n}$ timestamps are again used for computation, and January acts as the reference month. The third and final longitudinal control metric is named WEEKEND. It scores a value one for a CVE posted to the mailing list on Saturday or Sunday according to the coordinated universal time, taking a value zero otherwise. The rationale relates to observations that the days of week may affect the likelihood of introducing bugs during software development @Sliwerski05. Although the empirical evidence is mixed regarding this assertion @Eyolfson11, it is reasonable to extend it toward vulnerability coordination. For instance: if the $i$:th requested CVE would have otherwise ended up to NVD rapidly after two days, it may be that an additional delay, say $\epsilon > 0$, is present in case the request was posted on a weekend, such that $y_i = 2 + \epsilon$. The same rationale applies to the monthly effects. In other words, annual holidays presumably taken by the MITRE affiliates and others participants may well affect the coordination delays. ### Social Network and Communication Metrics {#subsec: social network and communication metrics} Four metrics are used for soliciting an answer to \[rq: social\]. The first is the amount of participants linked to CVEs: $$\label{eq: socdeg} \textmd{SOCDEG} = [\operatorname{Deg}(a_1 \in A), \ldots, \operatorname{Deg}(a_n \in A) ],$$ where $n = \vert A \vert$ and the set $A$ is assumed to be ordered. In other words, the metric equals the degree of the CVE-labeled vertices in $G_s$. This degree centrality conveys a clear theoretical rationale. In the software engineering context this rationale relates to the saying "too many cooks spoil the broth". The essence behind the saying is that increasing number of participants increases the coordination requirements, which translate into delays in completing software engineering tasks @Bird06 [@Johnson16a]. There exists also some evidence for an assertion that bug resolution delays increase with increasing number of participants in the resolution processes @Bhattacharya11. Although the existing empirical evidence seems weak, the same dictum can be extended to a further hypothesis that "too many developers" increase the probability of introducing vulnerabilities during software development @Meneely10. Given analogous reasoning, SOCDEG can be expected to lengthen the coordination delays. If a given $a \in A$ has a high degree, meaning that many participants posted messages containing the CVE, it may be that the vulnerability in question was particularly interesting or controversial. Either way, a longer delay could be expected for such a vulnerability. In theory, also other vertex-specific centrality metrics could be used for modeling the delays. There are a couple of reasons to avoid additional centrality metrics, however. The first reason relates to interpretation ambiguities. For instance, the so-called closeness centrality is often used for quantifying information flows among a group of human participants @BettenburgHassan13. In the context of sender-receiver type of email networks @Lubarski12 [@Tang14], this quantification rests on the assumption that a reply to an email constitutes an information flow. In reality, however, the lack of a reply does not imply the absence of an information flow; a participant may read an email without replying @Nia10. In addition to these theoretical limitations, the bipartite structure of $G_s$ makes the interpretation of many vertex centrality metrics challenging. For instance, the so-called betweenness centrality is often interpreted to reflect communication gatekeepers and information brokers @Bird06 [@PooCaamano17], but it is difficult to theorize how a CVE identifier would be a gatekeeper. The second, more practical reason stems from multicollinearity issues induced by the inclusion of additional centrality metrics. As is typical @Ruohonen16MESSA [@Schoch17], many of the centrality metrics are correlated. In particular, SOCDEG is highly correlated with the betweenness centrality values. Instead of explicitly computed centrality, the second social network metric takes a simpler approach to quantify the concept of core developers in the open source context. The definitions for such developers differ. For instance, some authors have identified core developers with a cutoff point for vertex degrees @Licorish14 [@Toral10], while others have relied on documents about developer responsibilities and commit accesses @Conaldi13 [@Crowston17]. In the present context the relevant social network core is composed by the MITRE affiliates. Thus, MITREDEV is a dummy variable that scores one for all CVEs linked to the three labeled participants in Fig. \[fig: core\]. The two remaining metrics are not explicitly related to social networks *per se*, although both of these still proxy communication practices. Namely: the metric MSGSLEN counts the length of strings in all emails posted with a given CVE identifier divided by one hundred, while the metric MSGSENT records the Shannon entropy of these emails. Both metrics have been used previously in the software engineering context @BettenburgHassan13. The effect of both metrics upon $y_i$ can be also expected to be positive. Given the rationale of the mailing list for coordinating CVE identifiers, lengthy email exchanges and increasing entropy are both likely to increase the coordination delays. In other words, high values for these two metrics both run counter to the short "use CVE-2016-6527"@Openwall16c communication patterns preferred on the list during the period observed. ### Infrastructure Metrics {#subsec: infrastructure metrics} Six metrics are used for soliciting an answer to \[rq: infrastructure\]. The first of these, INFDEG, is defined analogous to but by using the vertex set $B$ present in the network $G_d$. Thus, this metric counts the number of semantically valid domain names in the adjacency of the CVE identifiers observed. The analytical meaning is similar to the number of hyperlinks in reports posted within bug tracking systems, which have been hypothesized to reflect bugs that are particularly difficult to remedy [@BettenburgHassan13]. By translating the same hypothesis to the vulnerability coordination context, the number of domain names extracted from the hyperlinks could be expected to increase the coordination delays. Accordingly, a CVE that accumulates many hyperlinks may correspond with a vulnerability that is particularly difficult to interpret. Another explanation may be that the signal of relevant information is lost to the noise of numerous hyperlinks. However, the effect of INFDEG could be alternatively speculated to shorten the delays. The rationale for this alternative speculation relates to the prerequisite constraints in typical software engineering coordination. A requested CVE identifier is likely to end up in NVD in case it is backed by sufficient and valid technical information. In addition to INFDEG, these prerequisite constraints can be also retrospectively approximated through the references provided in NVD to the primary information sources. Like the historical @Bozorgi10 and contemporary @Ruohonen17TIR databases, NVD maintains a list of reference sources that is frequently polled for new vulnerability information @Christey13. If a vulnerability appears in multiple sources, it is probable that a CVE for the vulnerability appears rapidly in NVD due to information gains about the technical details. Thus, the metric NVDREFS counts the number of NVD's reference sources for all CVEs observed. Even when a CVE was coordinated through `oss-security`, shorter delays can be expected for an identifier with multiple alternative sources for confirming the corresponding vulnerability. Metric Regular expression for all $d \in D$ --------- -------------------------------------------------- VULNINF `cert.`, `exploit-db.com`, `first.org`, `mitre.org`, `nist.gov`, `osvdb.org` BUGS `bugs.`, `bugzilla.`, `gnats.`, `issues.`, `jira.`, `redmine.`, `trac.`, `tracker.` REPOS `code.`, `cvs.`, `cvsweb.`, `download.`, `downloads.`, `ftp.`, `git.`, `gitweb.`, `hg.`, `packages.`, `svn.`, `webcvs.`, `websvn.` SUPPORT `blog.`, `blogs.`, `dev.`, `doc.`, `docs.`, `forum.`, `forums.`, `help.`, `info.`, `lists.`, `support.`, `wiki.` : Infrastructure Regular Expressions<span data-label="tab: domain regex"></span> ![An Example Domain Name Neighborhood<span data-label="fig: example domain neighborhood"></span>](fig_ist_infexample.pdf){width="8cm" height="6cm"} The same rationale can be used for justifying a more fine-grained look at the hyperlinks explicitly posted on the mailing list. The four remaining infrastructure metrics are dummy variables that approximate the content behind the domain names stored to the set $D$ in $G_d$. Given a domain name neighborhood of each CVE identifier in the vertex set $B$, the value of these metrics is zero unless at least one domain name in the neighborhood matches the regular expressions listed in Table \[tab: domain regex\]. To further elaborate the operationalization of these metrics, Fig. \[fig: example domain neighborhood\] shows the neighborhood of a vertex labeled as CVE-2008-4688 in $B \in G_d$. For this CVE, the INFDEG metric takes a value six because there are six domain names linked to the identifier. Given the regular expressions, also the metrics VULNINF, BUGS, and REPOS each score a value one due to the vertices $d_2$, $d_3$, and $d_6$ (for the corresponding email see @Openwall08b). Of course, the expressions used are only approximations for automatically probing the nature of the primary tracking infrastructures. For instance, hosting services such as GitHub are mostly (but not entirely) excluded due to the matching primarily based on subdomains. Nevertheless, these infrastructure metrics convey a clear theoretical expectation. For instance, patches are often provided faster for high-profile vulnerabilities that affect multiple vendors, particularly in case computer emergency response teams are involved @Ruohonen16AICCSA [@Temizkan12]. For analogous reasons, VULNINF could be expected to decrease the coordination delays observed. Given the open source way of coordinating defects in open bug trackers, also BUGS can be expected to show a significant negative impact upon the delays. In other words, a good bug report often goes a long way in satisfying a prerequisite constraint for a CVE assignment in the open source context. ### Vulnerability Metrics {#subsec: vulnerability metrics} The remaining metrics are used for evaluating whether the severity and technical characteristics of the vulnerabilities coordinated affect the delays. Given the existing bug tracking research, there are good reasons to expect a correlation. Although the empirical evidence seems to be again somewhat mixed @Bhattacharya11, the severity of bugs have been observed to correlate with resolution delays in bug tracking systems @ZhangHassan12 [@ZhouGupta15]. Furthermore, the complexity of source code has been observed to correlate with bug resolution times @Kula10. In terms of the metrics used, however, the bug tracking literature differs from vulnerability research. Most bug tracking systems contain categories for assessing the severity and impact of bugs. The severity assignments within these systems are typically done either by bug reporters or by the associated developers. Due to the human work involved, the severity assignments have been observed to be relatively inconsistent and unreliable @TianHassan16. In contrast to bug tracking systems, the severity of vulnerabilities archived to NVD are evaluated by experts by using the Common Vulnerability Scoring System (CVSS). Although it remains debatable whether the severity assignments are suitable for assessing the security risks involved @Allodi14 [@YonisMalaiya16], the assignments themselves are highly consistent across different databases and evaluation teams @Johnson17. Therefore, in technical terms, CVSS provides a good and reliable framework for seeking an answer to the question \[rq: technical\]. The metrics in the models $\operatorname{M}_4$ and $\operatorname{M}_5$ are all based on the second version of the CVSS standard @FIRST07. (It is worth also remarking that only data based on this version is provided in NVD for the historical archival material relevant to the case studied.) This CVSS version classifies the severity of the vulnerabilities archived according to two dimensions: *exploitability* possibilities and *impact* upon successful exploitation. The impact dimension contains three metrics (confidentiality, integrity, and availability). Each of these can take a value from three options (`NONE`, `PARTIAL`, or `COMPLETE`). For instance, successfully exploiting the recently discovered and disclosed high-profile Meltdown vulnerability results in complete loss of confidentiality, although integrity and availability remain unaffected @NVD18a. Due to multicollinearity issues, the three impact metrics IMPC, IMPI, and IMPA are based on collapsing the three value categories according to: $$\label{eq: cvss impact} g(x) = \begin{cases} 0 & \textmd{if} ~x~ \textmd{is}~\texttt{NONE}, \\ 1 & \textmd{if}~x~\textmd{is}~ \texttt{PARTIAL}~\textmd{or}~\texttt{COMPLETE} . \\ \end{cases}$$ An analogous operationalization is used for the three dummy variables EXPNET, EXPCPLX, and EXPAUTH (see Table \[tab: metrics\]). These approximate the exploitability dimension based on the access *vector* (whether exploitation requires a network or a local access), access *complexity* (how complex are the access conditions for exploitation), and *authentication* (whether exploitation requires prior authentication) metrics in the CVSS v.2 standard. The CVSS standard and the associated data from NVD provide a wealth of information for approaching different security questions, but it is unclear how the re-coded impact and exploitability metrics may affect the coordination delays. There exists some evidence for a conjecture that the difficulty of reliable severity assessments vary in terms of what is being assessed and who is doing the assessments. This conjecture applies both to the CVSS framework @Allodi17d and to quantitative security assessments in general @Johnston18. However, analogous reasoning does not seem to hold in the context of NVD and the second version of the CVSS standard; the content of the standard does not notably affect the delays for CVSS assignments @Ruohonen19ACI. In addition to these empirical observations, it is difficult to speculate why some particular CVSS metric would either increase or decrease the coordination and related delays @Ruohonen17COMSIS. For these reasons, the CVSS metrics are used in the models without prior theoretical expectations. The assumption merely is that the severity of the vulnerabilities coordinated is statistically associated with the delays observed. The final set of metrics is based on the Common Weakness Enumeration (CWE) framework maintained and developed by MITRE in cooperation with the associated governmental partners and volunteers @MITRE18a. In essence, this comprehensive framework is used to catalog the typical "root causes" (weaknesses) that may lead to exploitable vulnerabilities in software products. The examples range from buffer and integer overflows to race conditions and software design flaws. By using a subset of frequent CWE identifiers @MITRE18b, NVD maintainers derive the underlying weaknesses behind the vulnerabilities archived either during the CVE coordination or in retrospect. Currently, the CWE framework covers over 700 distinct weaknesses. Partially due to this large amount, some of the CWEs are difficult to evaluate in practice @GosevaPopstojanova15 [@WuGandhi10]. This difficulty does not necessarily imply that the framework itself would be problematic. Rather, many vulnerabilities chain a lot of distinct weaknesses together. For instance, CWEs can be used to exemplify that buffer overflow vulnerabilities posit a complex "mental model" for developers due to the tangled web of distinct programming mistakes involved @WuGandhi10. Given analogous reasoning, it can be hypothesized that NVD-based CWE information can also explain a portion of the variation in the CVE coordination delays. As an example: when compared to XSS and CSRF vulnerabilities (as identified with CWE-79 and CWE-352), it may be that buffer overflow vulnerabilities (CWE-119) result in slower coordination because such vulnerabilities are usually technically more complex than simple web vulnerabilities. In other words, the effort required for interpretation may vary from a vulnerability to another, and such variation may show also in the coordination delays. To probe this general assumption, the final model $\operatorname{M}_6$ includes the ten most frequent CWEs in the sample. Given that about 90% of the CVEs in the sample have valid CWE entries in NVD, together these top-10 entries account for about 79% of the CWEs available. Estimation {#subsec: estimation} ---------- The coordination delay vector ${\textup{\textbf{\textrm{y}}}}= [y_1, \ldots, y_n]^\prime$ represent count data: the observations count the days between CVE assignments on the mailing list and the publication of the CVEs assigned within the primary global tracking database. Therefore, Poisson regression @Ruohonen19ACI [@Ruohonen17COMSIS] and regression estimators for survival
{'source': 'arXiv_out/2007'}