DOD’s guidance states that its FFRDCs are created to (1) provide strategic value through independent, intellectually rigorous, relevant, and timely products and services; and (2) support the department’s goals of long-term improvement in operations and enhanced national security. They are managed by various military departments or divisions within the department, called primary sponsors. FFRDCs are operated by universities, other not-for-profit or nonprofit organizations, or private firms—called parent organizations—under long-term contracts. They provide special research and development services that generally cannot be readily satisfied by government personnel or private contractors. For example, the Lincoln Laboratory develops key radar and electronic warfare technologies for integrated air and missile defense systems. The Software Engineering Institute provides cybersecurity solutions for defense entities. DOD’s FFRDCs are grouped into three categories: research and development laboratories, study and analysis centers, and systems engineering and integration centers. DOD oversees 10 FFRDCs (see table 1). According to the Director of Laboratories and Personnel within the OUSD for Research and Engineering, he took over responsibility for managing FFRDCs in July 2018, following a reorganization of OUSD for Acquisition, Technology and Logistics. DOD and each FFRDC have a sponsoring agreement, which is a stand- alone, bilateral, written agreement between the primary sponsor and the parent organization. It must be approved by the Under Secretary of Defense for Research and Engineering prior to award of an FFRDC contract and is incorporated into the contract. According to DOD’s guidance, the sponsoring agreement defines the FFRDC’s purpose and mission, establishes the conditions under which DOD may award an FFRDC contract, and describes the overarching requirements for operation of the FFRDC. For example, the primary sponsor must include provisions in the sponsoring agreement to prevent real or perceived organizational and personal conflicts of interest. As part of that, sponsors are to require FFRDC parent organizations to establish and maintain policies and procedures to protect information, such as sensitive data, from disclosure and provide training that covers ethics and conflicts of interest. We reported in December 2019 that representatives from the five study and analysis center FFRDCs said they provide annual training covering ethics and conflicts of interest for all personnel. DOD may use FFRDCs to perform work that is closely associated with the performance of inherently governmental functions or that is critical to maintaining control of the department’s missions and operations. Work could include activities such as support for financial analyses, policy development, acquisition planning, source selection, and contract management. In the course of performing work, FFRDCs may need access to acquisition data collected from DOD’s prime contractors and program offices. FFRDCs may obtain these data through DOD personnel, government databases, or directly from prime contractors. Government- held data may be stored and managed in department-wide databases or by individual program offices. For example, the Cost Assessment Data Enterprise is a web application that allows users access to various reports that include information such as major defense acquisition programs’ cost, software, and technical data. On December 21, 2017, DOD issued implementing guidance that marked the launch of its 3-year pilot program. According to DOD officials, prior to the start of the pilot program, FFRDC researchers needed to obtain permission from each data owner (e.g., DOD prime contractor or supplier), typically by signing a nondisclosure agreement. According to DOD officials that requested the authority to allow FFRDCs to have increased access, one of the purposes of the pilot was to allow for a streamlined nondisclosure agreement process. Under the pilot program, FFRDC researchers no longer have to obtain nondisclosure agreements with each data owner. To participate in the pilot, the FFRDC and DOD sponsor must first take steps to ensure certain protections are in place to protect against unauthorized disclosure or use of the data being accessed. For example, according to the statute, in order to be eligible, participating FFRDCs and its personnel (FFRDC researchers) had to agree to be subject to and comply with appropriate ethics standards and requirements applicable to government personnel, including the Ethics in Government Act of 1978, the Trade Secrets Act, and the Procurement Integrity Act. After the protections are in place, the FFRDC and DOD sponsor can enroll individual projects in the pilot program. Per the implementing guidance, the FFRDCs and DOD sponsors agree to collect and provide information about the enrolled projects. For example, DOD sponsors must provide the Laboratories and Personnel Office quarterly updates on a project’s progress obtaining data and, once the project is complete, information on the results of its access to sensitive data under the pilot program. Six of 10 DOD FFRDCs elected to participate in the pilot program during its first 21 months (figure 1). According to FFRDC representatives, the decision regarding whether an FFRDC would participate in the pilot program primarily depended on two factors: (1) the data needs of the FFRDC’s projects and (2) the ability of FFRDCs to access necessary data without the pilot program. Representatives from the six participating FFRDCs told us they elected to participate because they required access to sensitive data and, in some cases, lacked viable options for obtaining that data. For five of these FFRDCs, representatives said their researchers had identified specific projects for which they were interested in using the pilot to gain access to data sources with sensitive data from numerous contractors. Representatives from the four nonparticipating FFRDCs said that the existing processes the FFRDCs have in place provide the access they need for their projects. For example, Lincoln Laboratory representatives said their researchers are often working with an individual program or working to advance a specific technology; therefore, their work is generally with a limited number of contractors. In cases where they have needed access to sensitive data to do this work, they have executed a blanket nondisclosure agreement with their primary sponsor and, in some cases, have executed more tailored nondisclosure agreements with companies when obtaining information directly from a defense contractor. Aerospace has a blanket nondisclosure agreement with the Air Force Space and Missile Systems Center and the center included a provision in its contracts that requires its prime contractors to directly share information with the FFRDC. A Center for Communications and Computing representative said existing processes already provide the access they need for their projects. According to the FFRDC’s sponsor, its work focuses more on technological development rather than acquisitions analysis. Participating FFRDCs reported 33 projects enrolled in the pilot program from January 2018 through September 2019. Pilot projects represented about 1.5 percent of these FFRDCs’ total number of projects as of June 2019. Of the projects enrolled in the pilot program, 11 were complete and 10 were ongoing as of the end of September 2019. In addition, 11 projects enrolled in the pilot initially, only to realize they did not require access to the requested data and thus were removed. One project was put on hiatus pending a decision about whether it will continue. Table 2 summarizes the status of the projects in the pilot program as of September 2019. According to DOD officials, the fiscal year 2018 reorganization of OUSD for Acquisitions, Technology and Logistics into two offices, coupled with changes in leadership, shifted attention away from the pilot program design and implementation. For example, an official from the Acquisition and Sustainment office also told us it missed an opportunity to conduct outreach with its FFRDC because the office did not hold its biannual meetings in 2017 or 2018 due to the reorganization. In these biannual meetings, he explained, they would have discussed the department’s future research priorities and how the pilot program may have helped. This official—who was involved in the pilot’s implementation—also noted that the shift in attention meant they did not engage with the offices that maintain the data repositories as fully as they would have liked. We found it took sponsors and FFRDCs from a few weeks to 7 months to resolve questions about pilot program requirements and update the FFRDCs’ sponsoring agreements to incorporate the pilot protections. During that time, FFRDCs were unable to move forward with certain analyses for their proposed projects. Of the six projects we selected for further review (shown in table 3), four have been completed and FFRDCs reported benefits from their pilot program participation. The two remaining projects are on hiatus or removed. DOD officials and representatives from the four completed projects shared with us the following benefits: Systems and Analyses Center assessment of the U.S. munitions defense industrial base capacity. Portions of the research required access to sensitive data about the availability and production levels of manufacturing parts for a large number of contractors and suppliers. The FFRDC researchers used these data in their microlevel assessments of the manufacturing capacity and supply chain resiliency of the U.S. defense munitions industrial base. They said they were able to provide DOD’s Industrial Policy office with a more complete picture by combining these microlevel analyses with broader analyses of employment trends and economic outputs. A DOD industrial policy official who requested the work also said that the analysis enabled her office to meet an executive branch reporting requirement, which DOD did not have the manpower to conduct. National Defense Research Institute support for analysis of munitions industrial base. FFRDC researchers worked in collaboration with government officials to perform analyses on the adequacy of the munitions and missiles industrial base using government-held data from prime contractors and subcontractors. For example, the researchers supported working groups examining propulsion and chemicals in munitions and provided analysis for a report to Congress on solid rocket motors. The DOD official that requested the work and National Defense Research Institute representatives said that, without the pilot, the FFRDC would not have been able to access the data used to support DOD in these efforts. The official also noted that in this case the FFRDC helped fill a gap in DOD’s workforce to meet a congressional reporting requirement. National Security Engineering Center and Software Engineering Institute analysis of software acquisitions practices. FFRDCs supported a Defense Innovation Board study that aimed to identify correlations between software complexity, cost, and schedule evolution. FFRDC researchers’ access to and use of the data provided important insights about the quality and reliability of the department’s data. Specifically, DOD gained further insight into the kinds of software data the department holds and the significant gaps that would need to be addressed to improve overall DOD-held data quality. The Defense Innovation Board’s report included findings related to the quality of the software data accessed and analyzed by these two FFRDCs. Project AIR FORCE assessment of contractor risk. According to FFRDC representatives and an Air Force official involved in the work, the pilot program facilitated the FFRDC’s access to sensitive data held by the Defense Contract Management Agency that researchers used to identify early indicators of contractor performance risks. In response to the results of this work, the Air Force has funded a follow- on project to further research the potential of data analytics to provide early indicators of challenges in contract execution, according to an Air Force official involved with project. In addition to the benefits at the project level, several DOD sponsor officials and FFRDC representatives also noted the benefits of using the streamlined nondisclosure agreement process to accomplish their work. According to several DOD sponsor officials and FFRDC researchers we spoke to, completing the requested analysis without the pilot program would have required individual nondisclosure agreements with hundreds of individual contractors and suppliers. Systems and Analyses Center representatives said this would have been essentially impossible, and would have prevented researchers from completing important parts of the analyses. In another case, a Software Engineering Institute representative told us that, before the pilot, their team could not access software data when attempting to complete a 2017 project involving DOD software costs and production time frames. For that project, the DOD organization responsible for the data repository had recommended researchers send out a data request letter to each of the contractors with data in the system. Researchers sent out roughly one hundred requests to contractors for permission, but received no responses. They pointed out this was in part because contractors have no incentive to respond to an FFRDC’s request for access to their data. As a result, Software Engineering Institute was unable to use updated data for the 2017 report. While several sponsoring agency officials noted benefits of using the data for analyses to inform key program decisions, they also noted that a causal relationship between the pilot program activities and acquisition process improvements would be hard to establish, in part due to the length of time needed for projects to effect change. DOD officials responsible for two completed projects examining the munitions industrial base said they expect the analyses performed will lead to improved acquisition processes but that it would take many years to see the benefits. Specifically, they said the FFRDCs’ work helped identify areas for improvement in the department’s budget and acquisition strategy to better signal future demand to its lower tier munitions industrial base suppliers. In addition to noting these expected improvements, several DOD officials also acknowledged that expanding access of sensitive data to more people increases the potential for unauthorized use or disclosure but said that the pilot program put in place important protections to help mitigate these risks. DOD’s guidance to implement the pilot program outlined protections the FFRDCs must agree to, in order to guard against unauthorized disclosure or use of sensitive data, and required that these protections be incorporated into the sponsoring agreements between the FFRDCs and the DOD sponsor. However, we found some instances where details of the required protections were not incorporated into the agreements. We also found that the Laboratories and Personnel Office, which is responsible for managing the pilot program, does not have a procedure to verify whether protections were implemented, in part because it has not developed a process for doing so. Table 4 explains the protections. Some of these protections were already part of the FFRDCs’ business operations, while others are new. For example, the prohibition on their use of sensitive data to compete against a third party was already a fundamental aspect of FFRDCs’ role in supporting DOD. Similarly, participating FFRDC representatives told us that certain protections, such as implementing nondisclosure agreements and training, required only small adjustments to their existing procedures. However, the pilot’s financial disclosure program, annual certifications by parent organizations, and instructions for researchers to notify contracting officers of employment offers when supporting source selection decisions were new and specific to the pilot, according to a DOD official involved in the pilot’s implementation. We found that not all the details of protections were incorporated into the sponsoring agreements we reviewed. According to DOD’s implementing guidance, to participate in the pilot program, FFRDCs must agree to and follow these protections, which are to be incorporated into FFRDCs’ sponsoring agreements. We found that all six participating FFRDCs’ sponsoring agreements were updated and that most of the protections were incorporated. However, none included the instructions for FFRDC personnel involved in source selections to notify contracting officers if they are contacted about employment by an entity whose proposal is being evaluated and recuse themselves. We also found that the sponsoring agreements omitted one of the three officials that should be notified in the event of a Trade Secrets Act violation. These details were also not included in the templates DOD provided sponsors to use when updating FFRDC sponsoring agreements. When we raised these gaps to the attention of the DOD office responsible for managing the pilot program, the officials we spoke with were unaware of these omissions. In addition, the Laboratories and Personnel Office has not taken steps to ensure that another protection—the certification of the annual review of financial disclosure forms—has occurred, even though it was incorporated into the sponsoring agreements. The implementing guidance states that FFRDCs’ parent organizations must certify the annual review of financial disclosure forms and archive these forms for 6 years. However, only two FFRDC parent organizations provided us with this certification. According to representatives from parent organizations of the other four FFRDCs, the review of financial disclosures is generally performed as part of their conflict of interest programs. They review the disclosures on an annual or rolling basis when researchers are assigned to new projects but had not certified, as the sponsoring agreements require, that they have taken this step for the pilot program. We found that the Laboratories and Personnel Office had not taken steps to verify FFRDC parent organization compliance with this protection, such as collecting or reviewing the certification. When we raised this gap to the attention of the DOD office responsible for managing the pilot program, the officials were unaware of the missing annual certifications. By not ensuring the annual review is occurring, DOD has limited information about FFRDCs’ adherence to this pilot program protection. The pilot’s implementing guidance also states that, before government personnel provide access to sensitive data, the FFRDCs and researchers must have addressed these protections. However, the Laboratories and Personnel Office has not taken steps to ensure it is done. In our review of the six specific projects, we found that different people were checking that some of the protections were in place. For example, For two of the six projects, a primary sponsor official had a copy of the FFRDC addendum, and collected and reviewed the nondisclosure agreements and certifications of financial disclosure for individual researchers on each project. For a third project, a DOD official in the office that requested the project told us she confirmed that FFRDC researchers working on the project were part of the pilot program and told the official from the data repository that he could share information with the researchers. For the remaining three projects, representatives for a data repository that provided researchers with data access told us they confirmed that the addendum was incorporated into the sponsoring agreement and that researchers had the individual protections, such as a nondisclosure agreement, in place before providing access to the data. Standards for internal controls in the federal government state that responsibilities for control activities, such as sponsors ensuring the protections are incorporated into the agreements and that FFRDCs are following these protections, should be documented through policy and procedures. Without a process that includes clearly defined roles and responsibilities to ensure the protections are followed, DOD cannot ensure that its goal to safeguard sensitive data is achieved. DOD established what information sponsors must collect about the projects enrolled in the pilot in its implementing guidance to sponsors and notified FFRDCs about these responsibilities. The requirements include: pre-action information to be collected when the project is enrolled in the pilot, which includes basic details about the project, the data required, and planned analysis; quarterly status updates, which include progress obtaining access to sensitive data and any challenges or barriers to access; and post-action information regarding the results of pilot access when the project’s analyses are completed, which includes a summary of how the pilot supported FFRDC research, and any benefits accrued to DOD from pilot participation. The implementing guidance also instructs sponsors to collect information about the project’s results again 6 months after the project is completed. The Laboratories and Personnel Office, which is responsible for managing the pilot program, sends an email quarterly requesting that sponsors submit information. However, we found that the Laboratories and Personnel Office did not receive pre-action information from 11 of the 33 projects in the pilot program. For example: Systems and Analyses Center has not submitted pre-action information for six of its eight projects. An official from the Systems and Analyses Center’s primary sponsor office told us that in his view, pre-action information can be obtained by other means, and he had not requested it. Project AIR FORCE and Arroyo Center have submitted project pre- action information for five projects to the FFRDCs’ primary sponsors: Air Force and Army, respectively. However, the primary sponsors have not provided this information to the Laboratories and Personnel Office. An Air Force official explained that he gets a request from the Laboratories and Personnel Office for the quarterly reports, but not the pre-action information, and thus had not provided it. In addition, the office did not collect a quarterly report for three projects in the pilot, and, as of September 2019, two completed projects had passed the 6-month post-completion time frame and only one had submitted post-action information. These gaps in reporting have occurred because the Laboratories and Personnel Office is not monitoring the project information it receives to ensure sponsors are submitting all required reporting. DOD’s implementing guidance states that primary sponsors will collect and submit this information for each project enrolled in the pilot program. Further, GAO’s leading practices for pilot design state, among other things, that a well-designed pilot program should have a clear approach to gathering information for the purpose of supporting the future evaluation of the pilot and tracking the pilot program’s implementation and performance. Consistency in collecting pre-action, quarterly, and post- action reports is important because each contains different information, which could be useful for the department to track the pilot program’s progress and in an evaluation of the pilot program. For example, without the pre-action information from Systems and Analyses Center, Arroyo Center, and Project AIR FORCE, the Laboratories and Personnel Office will not have general descriptions of their pilot projects or information about the kinds of data these FFRDCs initially planned to access. Without complete information, DOD will not be able to effectively evaluate the pilot program and inform future decisions about the program’s status. We found that DOD followed some but not all of the leading practices for evaluating its pilot program. According to GAO’s leading practices for pilot design, a well-developed and documented pilot program can help ensure that agency assessments produce information needed to make effective program and policy decisions. Such a process enhances the quality, credibility, and usefulness of evaluations, in addition to helping to ensure that time and resources are used effectively. Five leading practices form a framework for effective pilot design and evaluation. (See figure 2.) We found DOD generally addressed the first of the leading practices of pilot design—establish objectives—by establishing goals for the pilot program, summarized below. Make sensitive data previously restricted or unavailable available for analysis. Use sensitive data in accordance with the FFRDC contract. Safeguard sensitive data. Document results of pilot program. Document risks or costs of FFRDC access to sensitive data. Gain analytic value from FFRDC access to sensitive data. Demonstrate benefits to government from sharing sensitive data. Inform future actions for making FFRDC access to sensitive data permanently available. However, DOD has not fully addressed the other leading practices. Specifically, we found that the Laboratories and Personnel Office does not have: a plan that (1) includes an assessment methodology to ensure DOD is collecting the correct information to evaluate whether the pilot has met the department’s goals and (2) defines how DOD will use the information collected to evaluate the implementation and performance of the pilot program, when the evaluation will take place, and by whom; a plan for identifying or documenting lessons learned; and a plan for gathering input from stakeholders, such as DOD sponsors, FFRDCs, and officials from DOD’s data repositories, for the pilot program’s evaluation. According to officials involved in its implementation, DOD did not consider creating such plans when developing the pilot program. The pilot program guidance, however, stated that information collected would be used for the department to assess the ongoing efficacy of the pilot program and GAO’s evaluation. These officials explained that, when the pilot’s guidance was formulated, the department was in the process of reorganizing the former Acquisition, Technology, and Logistics Office and they pointed out that GAO was to do the assessment of the pilot program. They said the reason the program collected information—such as quarterly reports—was to inform our review. Thus, they had no plan to assess the information collected and no plans to talk to stakeholders or to collect and share lessons learned. While our review occurred during pilot implementation, an evaluation of the pilot conducted by DOD after more projects are completed would provide an opportunity to identify lessons learned and gather valuable input from stakeholders—such as the offices that manage the data repositories and the sponsors requesting the projects. We found cases where FFRDC researchers had problems accessing data and where gathering this input from stakeholders involved with pilot projects would have been useful for DOD. For example, some FFRDC researchers described barriers when trying to gain access to certain government- and department-wide databases. In one case, Project AIR FORCE researchers reported not being able to access information in the Electronic Document Access database and various databases containing contractor performance information because researchers lacked military or government email addresses. Further, some of the databases that FFRDC researchers and a DOD official said would be useful are not owned by DOD. We found that guidance for one such database explicitly prohibits disclosure of contractor evaluation data to any contractor or non- government entity. In addition, the researchers were able to gain only partial access to DOD’s Acquisition Information Repository (a database containing acquisition documents for DOD’s major weapons programs) and, as a result, were unable to access individual documents, such as program assessment reports. They told us the repository is set up such that the researchers must request access to individual documents directly from document owners, who set permissions when uploading documents, rather than from a central source that can grant access across the repository. Without further evaluation of the pilot, DOD is missing an opportunity to benefit from gathering input from its stakeholders and identifying lessons learned, such as learning and understanding more about these barriers to accessing certain databases. There is still time for DOD to develop an evaluation plan with elements described in our leading practices. The pilot program ends December 21, 2020. Our review comes at a time when 11 of the overall 33 projects have been completed; therefore, information exists to report on outcomes. Officials and representatives from the Laboratories and Personnel Office and participating sponsors and FFRDCs expressed a continued need for access to the sensitive data. Without an evaluation plan, DOD will have difficulty determining the effectiveness of the pilot to meet its goal of accruing more analytic value for the department while also safeguarding sensitive data. The FFRDC pilot program has already provided DOD with some benefits, as a few FFRDCs have reported success in completing analysis that would not have been possible without it. However, in implementing the pilot, DOD has room for improvement. A key control of this pilot that provides access to sensitive data is ensuring protections are in place to prevent improper disclosure. Another control is to establish a process to ensure these protections are followed, yet the responsible office within DOD has not done so. Further, despite the fact that the pilot is past the midpoint of implementation, this office still has an opportunity to develop a plan on how to evaluate it. But to do this, it must develop a mechanism to ensure it is collecting complete information on the pilot activities. Ensuring comprehensive reporting and implementing a well-developed evaluation plan will help DOD understand and articulate the benefits the department has accrued because of FFRDC’s access to sensitive data. Further, through identifying lessons learned and obtaining stakeholder input, the Laboratories and Personnel Office has an opportunity to better understand the challenges FFRDCs and the department face when attempting to access and use sensitive data included in government- and department-wide databases. Such an evaluation could help inform Congress’ decision whether to extend, make FFRDC access permanent, or end the pilot. We are making the following six recommendations to the Department of Defense: The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the details of the pilot program’s data protections are incorporated into the existing agreements. (Recommendation 1) The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to take steps to ensure that the FFRDCs and sponsors are implementing the pilot program’s protections for sensitive data. (Recommendation 2) The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to establish a monitoring and oversight mechanism to ensure that primary sponsors submit complete information on pilot projects, as required by DOD’s guidance for the pilot program. (Recommendation 3) The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan that outlines the methodology by which DOD will assess the pilot and how and when information collected will be analyzed to evaluate the pilot program. (Recommendation 4) The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan to identify and evaluate lessons learned from the pilot program. (Recommendation 5) The Under Secretary of Defense for Research and Engineering should direct the Laboratories and Personnel Office to develop a plan for obtaining input from stakeholders on the pilot program. (Recommendation 6) We provided a draft of this product to DOD for comment. DOD provided a letter response, reproduced in Appendix I. DOD agreed with our recommendations and described actions that it intends to take in response. We also provided excerpts of this product to FFRDCs for comment, of which three provided technical comments that we incorporated as appropriate. We are sending copies of this report to the appropriate congressional committees and the Secretary of Defense. In addition, the report is available at no charge on the GAO website at http://www.gao.gov. If you or your staff have any questions about this report, please contact me at (202) 512-4841 or makm@gao.gov. Contact points for our Offices of Congressional Relations and Public Affairs may be found on the last page of this report. GAO staff who made key contributions to this report are listed in Appendix II. In addition to the contact named above, Tatiana Winger (Assistant Director), Leslie Ashton (Analyst-in-Charge), Evan Nemoff, Tanya Waller, Jenny Chanley, Laura Greifner, Christine Pecora, and Roxanna Sun made key contributions to this report.
FFRDCs provide federal agencies with research and development functions, technical systems engineering capabilities, and policy development and decision-making studies, among other services. The Federal Acquisition Regulation states that FFRDCs have a special relationship with DOD, which can give FFRDCs access to sensitive data beyond what would commonly be shared with contractors. The National Defense Authorization Act for Fiscal Year 2017 directed DOD to establish a 3-year pilot program that allows FFRDCs streamlined access to sensitive data maintained by DOD. It also included a provision for GAO to report on the pilot program within 2 years of implementation. This report addresses the extent to which (1) FFRDCs are using the pilot program, (2) DOD put procedures in place to protect data accessed, and (3) DOD is evaluating the pilot program. GAO reviewed DOD guidance and FFRDC processes, pilot reports for January 2018 through September 2019, and DOD's plans and efforts for evaluating the pilot program. GAO also selected a nongeneralizable sample of six projects—at least one from each FFRDC with an enrolled project as of December 2018—for further review. In addition, GAO assessed the pilot program against leading practices for pilot design. The Department of Defense (DOD) launched a 3-year pilot program in December 2017 to enable a streamlined process to share certain sensitive data, such as data collected from its contractors, with its Federally Funded Research and Development Centers (FFRDC). At times, FFRDCs need to access such data to support DOD. The pilot was intended to reduce the burden on FFRDCs to seek permission from hundreds of contractors to access information needed for their research. Six of DOD's 10 FFRDCs have taken part in the pilot, enrolling a combined total of 33 projects, as shown in the table. DOD officials and FFRDC representatives reported that the streamlined process made the use of sensitive data feasible. As a result, FFRDCs with completed projects in GAO's sample indicated they were able to provide more robust analyses or insights to DOD. DOD guidance for the pilot program established procedures to protect sensitive data. But GAO found that DOD did not incorporate all of the details of the required protections into its agreements with FFRDCs. Further, GAO found that not all FFRDCs were performing annual certification of financial disclosure forms, as required by its agreements with DOD. DOD does not have a process to ensure that all the protections pertaining to FFRDCs' streamlined access to sensitive data are being followed. Without a process that defines roles and responsibilities, DOD cannot ensure that FFRDCs adhere to the protections. DOD developed goals for the pilot program and outlined what information was to be obtained for each participating project, actions that are consistent with GAO's leading practices for pilot design. However, DOD has not developed a plan for evaluating the program nor has it consistently collected information on about a third of the pilot projects. Leading practices for pilot design call for an evaluation plan, which should include an assessment methodology and identify responsibilities as to how the evaluation will be conducted. Without an evaluation plan and a mechanism to collect information on pilot projects, DOD will not be positioned to identify the effectiveness of the pilot program and benefit from lessons learned. Such information will be useful as Congress considers the path forward after the pilot ends in December 2020. GAO is making six recommendations, including that DOD take steps to ensure data protections are in the agreements and followed, collect information on projects, and evaluate the pilot. DOD agreed with the recommendations.