Since 2011, the Army has conducted  five of them, and has projected the cumulative cost of the events at $791  million. Army Not Taking Full Advantage of Potential Knowledge from NIEs

The Army has made steady improvements in the NIE process since its  inception and the evaluations continue to give the Army useful information  and helpful insights into current and emerging networking capabilities. However, some resulting Army decisions are at odds with knowledge  produced during the NIEs. Additionally, as  we reported previously, the Army has not yet tapped into the potential to  use the NIE to gain insight into the effectiveness and performance of the  overall tactical network. The NIEs allowed the  Army to  formulate a network architecture baseline that will serve as the  foundation upon which the Army plans to add networking capabilities  in the future; evaluate industry-developed systems that may help address Army- identified capability gaps in the future; integrate the new capability sets into operational units and to create  new tactics, techniques, and procedures for using the new systems in  operations; and provide soldiers with an opportunity to both provide input into the  designs of networking systems and to integrate the systems before  the Army fields them to operational brigades. According to DOT&E, the intended NIE objective to test and evaluate  network components together in a combined event is sound, as is the  opportunity to reduce overall test and evaluation costs by combining test  events. NIEs also offer the opportunity for a more comprehensive  evaluation of a mission command network instead of piecemeal  evaluation of individual network components. Two of these SUTs performed poorly during developmental testing. However, the  Army has been unable to buy many of these systems because it did not  have a strategy in place to rapidly buy promising technologies. Even so, Army officials did not develop  alternative acquisition approaches before they began the NIE process. The Army also has an  opportunity to work more closely with the test community to further  improve NIE execution and results. Army Is Implementing Corrective Actions to Improve NIE Process and Network Modernization

The Army has identified inefficiencies or less-than-optimal results in its  network modernization and the NIE process and has begun implementing  corrective actions to mitigate some of them. Through this initiative, the Army performs technology evaluations,  assessments, and integration of vendor systems. The Army is now developing a strategy to address these barriers. DOD Has Begun to Define Network Outcome-based Performance Metrics

In our initial report on the Army’s tactical network, we concluded that it will  also be important for the Army to assess the cost effectiveness of   Moreover, to  individual initiatives before and during implementation.facilitate oversight, we concluded that it is important for the Army and  DOD to develop metrics to assess the actual contributions of the initial  capability set the Army will field in fiscal year 2013 and use the results to  inform future investments. The Army has  worked closely with the test community to plan, conduct, and evaluate the  NIEs. Tension between the acquisition and testing communities has been long- standing. Additional opportunities exist for leadership of the Army and the test  community to work together to further improve NIE execution and results. A good starting point would be for the Army to consider addressing the  test community observations and recommendations from previous NIEs. On the other hand, even with a new strategy for  procurement of emerging capabilities to fill capability gaps, the Army may  still face an expectation gap with industry. Until it has clearly demonstrated the means to rapidly buy  and field emerging capabilities and provided this information to industry,  the Army may need to manage industry expectations of how many new  networking systems it can buy and how rapidly. Recommendations for Executive Action

To improve outcomes for its entire network modernization strategy, we  recommend that the Secretary of Defense direct the Secretary of the  Army to take the following four actions:
Require that network systems from major defense acquisition  programs obtain a positive Assessment of Operational Test  Readiness (now called a Developmental Test and Evaluation  Assessment) recommendation before being scheduled for operational  testing during the NIE;
Correct network system performance and reliability issues identified  during the NIEs before moving to buy and field these systems;
Provide results to industry on the Army’s actual experience in buying  and fielding successfully demonstrated systems under evaluation and  the length of time it has taken to date; and
Collaborate with all network stakeholder organizations to identify and  correct issues that may result in improved network outcomes,  including addressing the observations and recommendations of the  test community related to the NIEs. Appendix I: Scope and Methodology

Our objectives were to evaluate (1) the results of the Network Integration  Evaluations (NIE) conducted to date and identify the extent to which the  Army has procured and fielded proposed network solutions; and (2) Army  actions and additional opportunities to enhance the NIE process.
Why GAO Did This Study

In 2011, the Army began a major undertaking to modernize its tactical network to improve communication and provide needed information to soldiers on the battlefield. The Army has identified the network as its number one modernization priority requiring approximately $3 billion per year indefinitely. NIEs provide semi-annual assessments of newly developed systems. Given the importance of the network, GAO was asked to examine elements of the process the Army is using to acquire network capabilities. This report examines (1) the results of the NIEs conducted to date and the extent to which the Army has procured and fielded network solutions, and (2) Army actions to enhance the NIE process. To conduct this work, GAO analyzed key documents, observed testing activities, and interviewed acquisition and testing officials.

What GAO Found

Since 2011, the Army has conducted five Network Integration Evaluations (NIE), which have provided extensive information and insights into current network capabilities and potential solutions to fill network capability gaps. According to senior Department of Defense (DOD) test officials, the NIE objective to test and evaluate network components together in a combined event is sound, as is the opportunity to reduce overall test and evaluation costs by combining test events. Further, the NIEs offer the opportunity for a more comprehensive evaluation of the broader network instead of piecemeal evaluation of individual network components. However, the Army is not taking full advantage of the potential knowledge that could be gained from the NIEs, and some resulting Army decisions are at odds with knowledge accumulated during the NIEs. For example, despite poor results in developmental testing, the Army moved forward to operational testing for several systems during the NIEs and they demonstrated similarly poor results. Yet the Army plans to buy and field several of these systems. Doing so increases the risk of poor performance in the field and the need to correct and modify deployed equipment. On the other hand, the Army has evaluated many emerging network capabilities--with generally favorable results--but has bought very few of them, in large part because it did not have a strategy to buy these promising technologies. Army officials have stated that the success of network modernization depends heavily on industry involvement but, with few purchases, it is unclear whether industry will remain interested. Finally, the Army has not yet developed metrics to determine how network performance has improved over time, as GAO recommended in an earlier report.
The Army has several actions under way or planned to enhance the NIE process and has further opportunities to collaborate with the test community. The Army has identified issues in the NIE process and its network modernization strategy that were causing inefficiencies or less-than-optimal results and has begun implementing actions to mitigate some of those issues. For example, the Army has begun performing technology evaluations, and integration of vendor systems in a lab environment to weed out immature systems before they get to the NIE. The Army has also developed a strategy and has an acquisition plan to address requirements, funding, and competition issues that will help enable it to buy emerging capabilities rapidly. However, the Army will need to validate the new strategy and plan and provide results to industry, which could help to manage industry expectations about how many of and how quickly it can buy these capabilities. DOD has started to identify and evaluate network metrics and to re-focus NIEs to gather additional data and insights. Taking these actions will ultimately allow the periodic review and evaluation of the actual effectiveness of network capabilities and the likely effectiveness of proposed investments. The test community has worked closely with the Army on the NIEs but has also voiced various concerns about the NIEs including their being a schedule-driven event. Tension between the acquisition and test communities has been long-standing. Additional opportunities exist for Army leadership and the test community to work together to further improve NIE execution and results and to reduce tensions between the two communities. A good starting point for the Army would be to take a fresh look at the test community observations and recommendations from previous NIEs.

What GAO Recommends

To improve outcomes for the Armys network modernization strategy, GAO recommends that the Secretary of Defense direct the Army to (1) require successful developmental testing before moving to operational testing at an NIE, (2) correct issues identified during testing at NIEs prior to buying and fielding systems, (3) provide results to industry on Armys efforts to rapidly acquire emerging capabilities, and (4) pursue additional opportunities for collaboration with the test community on the NIEs. DOD agreed with the recommendations to varying degrees, but generally did not offer specific actions to address them. GAO believes all recommendations remain valid.