In addition, although DHS  established a variety of NCPS-related metrics, none provide insight into  the value derived from the functions of the system. NCPS Has Limited Ability to Detect Intrusions within Observed Network Traffic

NCPS’s intrusion detection capability is intended to provide DHS with the  ability to scan network traffic for signs of potentially malicious activity. By employing only signature-based intrusion detection, NCPS is unable to  detect intrusions for which it does not have a valid or active signature  deployed. However, NCPS is not currently evaluating all types of network traffic. This is accomplished by monitoring  network traffic to and from a customer agency’s network and taking some  action to stop traffic (e.g., blocking an e-mail) that has characteristics  matching pre-defined indicators of malicious traffic. Specifically, DHS has enhancements planned through  fiscal year 2018. As a result, DHS has yet to develop a majority of  planned functionality for the information-sharing capability of NCPS. As a result no substantive actions have yet been taken to develop this  capability. Additionally, DHS did not always solicit, and agencies did not always  provide, feedback on the notifications. One reason DHS and agencies do not agree about whether notifications  were received may be that DHS does not always explicitly ask for  feedback or confirmation of receipt of the notification. DHS Identified Future NCPS Needs, but Has Developed Limited Requirements

While DHS developed an executive road map for the intrusion detection,  prevention, analytics, and information sharing objectives that describes  future NCPS capabilities to be developed through fiscal year 2018, it has  not defined requirements, as called for by OMB guidance and best  practice, for two intrusion detection capabilities to be provided in fiscal  year 2016. In addition, although DHS officials stated that they do consider  threat information as part of the required risk-based approach for  determining future capabilities to protect federal information systems, they  do not consider specific vulnerabilities affecting agencies’ networks and  systems, as information on these is not currently available. Agencies have had mixed results in adopting NCPS capabilities. However, NSD documents indicated that only 5 of the 23 agencies were  receiving intrusion prevention services. Further, four of the five selected agencies in our review reported that not  all of their traffic was being sent to NCPS intrusion detection sensors. This agency will also not be able to implement the e-mail  intrusion prevention capability. This occurred in part because NSD did not provide guidance to customer  agencies on how to securely route their information to the Internet service  providers. The system’s intrusion prevention capability is less fully developed, with  limited deployment across different types of network traffic, such as  content from websites, limiting its ability to prevent malicious code from  penetrating agencies’ networks. While the adoption of the intrusion detection capabilities is widespread  among the 23 agencies required to use NCPS, the implementation of  intrusion prevention capabilities is more limited due to policy and  implementation challenges that DHS is working to overcome. Recommendations for Executive Action

We recommend the Secretary of Homeland Security direct:
NSD to determine the feasibility of enhancing NCPS’s current  intrusion detection approach to include functionality that would detect  deviations from normal network behavior baselines;
NSD to determine the feasibility of developing enhancements to  current intrusion detection capabilities to facilitate the scanning of  traffic not currently scanned by NCPS;
US-CERT to update the tool it uses to manage and deploy intrusion  detection signatures to include the ability to more clearly link  signatures to publicly available, open-source data repositories;
US-CERT to consider the viability of using vulnerability information,  such as data from the Continuous Diagnostics and Mitigation program  as it becomes available, as an input into the development and  management of intrusion detection signatures;
US-CERT to develop a timetable for finalizing the incident notification  process, to ensure that customer agencies are being sent notifications  of potential incidents, which clearly solicit feedback on the usefulness  and timeliness of the notification;
The Office of Cybersecurity and Communications to develop metrics  that clearly measure the effectiveness of NCPS’s efforts, including the  quality, efficiency, and accuracy of supporting actions related to  detecting and preventing intrusions, providing analytic services, and  sharing cyber-related information;
The Office of Cybersecurity and Communications to develop clearly  defined requirements for detecting threats on agency internal  networks and at cloud service providers to help better ensure effective  support of information security activities;
NSD to develop processes and procedures for using vulnerability  information, such as data from the Continuous Diagnostics and  Mitigation program as it becomes available, to help ensure DHS is  using a risk-based approach for the selection/development of future  NCPS intrusion prevention capabilities; and
NSD to work with their customer agencies and the Internet service  providers to document secure routing requirements in order to better  ensure the complete, safe, and effective routing of information to  NCPS sensors. Appendix I: Objectives, Scope, and Methodology

Our objectives were to determine the extent to which (1) the National  Cybersecurity Protection System (NCPS) meets stated objectives, (2) the  Department of Homeland Security (DHS) has designed requirements for  future stages of the system, and (3) federal agencies have adopted the  system. To determine the extent to which DHS has designed requirements for  future stages of the system, we reviewed NCPS program planning  documentation and interviewed program officials in order to identify how  future capabilities are planned.
Why GAO Did This Study

Cyber-based attacks on federal systems continue to increase. GAO has designated information security as a government-wide high-risk area since 1997. This was expanded to include the protection of critical cyber infrastructure in 2003 and protecting the privacy of personally identifiable information in 2015. NCPS is intended to provide DHS with capabilities to detect malicious traffic traversing federal agencies' computer networks, prevent intrusions, and support data analytics and information sharing.
Senate and House reports accompanying the 2014 Consolidated Appropriations Act included provisions for GAO to review the implementation of NCPS. GAO determined the extent to which (1) the system meets stated objectives, (2) DHS has designed requirements for future stages of the system, and (3) federal agencies have adopted the system. To do this, GAO compared NCPS capabilities to leading practices, examined documentation, and interviewed officials at DHS and five selected agencies. This is a public version of a report that GAO issued in November 2015 with limited distribution. Certain information on technical issues has been omitted from this version.

What GAO Found

The Department of Homeland Security's (DHS) National Cybersecurity Protection System (NCPS) is partially, but not fully, meeting its stated system objectives:
Intrusion detection: NCPS provides DHS with a limited ability to detect potentially malicious activity entering and exiting computer networks at federal agencies. Specifically, NCPS compares network traffic to known patterns of malicious data, or “signatures,” but does not detect deviations from predefined baselines of normal network behavior. In addition, NCPS does not monitor several types of network traffic and its “signatures” do not address threats that exploit many common security vulnerabilities and thus may be less effective.
Intrusion prevention: The capability of NCPS to prevent intrusions (e.g., blocking an e-mail determined to be malicious) is limited to the types of network traffic that it monitors. For example, the intrusion prevention function monitors and blocks e-mail. However, it does not address malicious content within web traffic, although DHS plans to deliver this capability in 2016.
Analytics: NCPS supports a variety of data analytical tools, including a centralized platform for aggregating data and a capability for analyzing the characteristics of malicious code. In addition, DHS has further enhancements to this capability planned through 2018.
Information sharing: DHS has yet to develop most of the planned functionality for NCPS's information-sharing capability, and requirements were only recently approved. Moreover, agencies and DHS did not always agree about whether notifications of potentially malicious activity had been sent or received, and agencies had mixed views about the usefulness of these notifications. Further, DHS did not always solicit—and agencies did not always provide—feedback on them.
In addition, while DHS has developed metrics for measuring the performance of NCPS, they do not gauge the quality, accuracy, or effectiveness of the system's intrusion detection and prevention capabilities. As a result, DHS is unable to describe the value provided by NCPS.
Regarding future stages of the system, DHS has identified needs for selected capabilities. However, it had not defined requirements for two capabilities: to detect (1) malware on customer agency internal networks or (2) threats entering and exiting cloud service providers. DHS also has not considered specific vulnerability information for agency information systems in making risk-based decisions about future intrusion prevention capabilities.
Federal agencies have adopted NCPS to varying degrees. The 23 agencies required to implement the intrusion detection capabilities had routed some traffic to NCPS intrusion detection sensors. However, only 5 of the 23 agencies were receiving intrusion prevention services, but DHS was working to overcome policy and implementation challenges. Further, agencies have not taken all the technical steps needed to implement the system, such as ensuring that all network traffic is being routed through NCPS sensors. This occurred in part because DHS has not provided network routing guidance to agencies. As a result, DHS has limited assurance regarding the effectiveness of the system.

What GAO Recommends

GAO recommends that DHS take nine actions to enhance NCPS's capabilities for meeting its objectives, better define requirements for future capabilities, and develop network routing guidance. DHS concurred with GAO's recommendations.