This type of  contract is commonly referred to as “seat management.” Generally  speaking, under seat management, contractor-owned desktop and other  computing hardware, software, and related services are bundled and  provided on the basis of a fixed price per unit (or seat). However, the Navy did not  implement this plan, choosing instead to focus on defining and measuring  contractually specified SLAs. Without effective performance  management, the Navy is increasing the risk that the program will continue  to fall short of its goals and expected results. However, the Navy has not implemented its 2000 performance management  plan. Customer satisfaction survey: The target was to have 85 percent of  NMCI end users satisfied. Specifically, the information superiority and  innovation goals that were used to justify the program have yet to be  attained. Contractor Has Largely Met Many but Has Not Met Other SLAs

Our analysis of Navy contractor performance data since September 2004  shows that the extent to which the site-specific agreements have been met  for all operational seats (regardless of site) varies widely by individual  agreement, with some always being met but others having varied  performance over time and by seat type. Our analysis also showed that,  although the contractor has met most of the enterprisewide agreements  during this time period, it has not met a few. 5). 9). NMCI Customer Groups’ Satisfaction Levels Vary, but Overall Customer Satisfaction Is Low

The Navy’s three groups of NMCI customers—end users, organizational  commanders, and network operators—vary in the extent to which they are  satisfied with the program, but collectively these customers are generally  not satisfied. End User Surveys Show Dissatisfaction with NMCI

Despite reported improvements in end user satisfaction levels since 2002,  end user responses to quarterly satisfaction surveys have been consistently  at the low end of the range of scores that the Navy defines the term  “satisfied” to mean, and the percentage of end users that Navy counts as  being “satisfied” have consistently been below the Navy’s satisfaction  target level. Exacerbating this long-standing shortfall in meeting end user satisfaction  expectations is the fact that the Navy considers a “satisfied” end user to  include users that are at best marginally satisfied and arguably somewhat  dissatisfied. Customer Satisfaction Improvement Efforts Are Not Being Guided by Effective Planning

The NMCI program office reports that improving customer satisfaction is a  program priority. Expanded network services. Given that the Navy reports that it has  already invested about 6 years and $3.7 billion in NMCI, the time to develop  a comprehensive understanding of the program’s performance to date, and  its prospects for the future, is long overdue. Recommendations for Executive Action

To improve NMCI performance management and better inform investment  decision making, we recommend that the Secretary of Defense direct the  Secretary of the Navy to ensure that the NMCI program adopts robust  performance management practices that, at a minimum, include (1)  evaluating and appropriately adjusting the original plan for measuring  achievement of strategic program goals and provides for its  implementation in a manner that treats such measurement as a program  priority; (2) expanding its range of activities to measure and understand  service level agreement performance to provide increased visibility into  performance relative to each agreement; (3) sharing the NMCI  performance results with DOD, Office of Management and Budget, and  congressional decision makers as part of the program’s annual budget  submissions; and (4) reexamining the focus, scope, and transparency of its  customer satisfaction activities to ensure that areas of dissatisfaction  described in this report are regularly disclosed to the aforementioned  decision makers and that customer satisfaction improvement efforts are  effectively planned and managed. Key contributors to this report are listed in appendix V.

Objectives, Scope, and Methodology

Our objectives were to review (1) whether the Navy Marine Corps Intranet  (NMCI) is meeting its strategic goals, (2) the extent to which the contractor  is meeting its service level agreements (SLA), (3) whether customers are  satisfied with the program, and (4) what is being done to improve customer  satisfaction. To determine whether NMCI is meeting its strategic goals, we  reviewed documents provided by Department of the Navy describing  the mission need for NMCI, strategic goals, performance measures, and  data gathered on actual performance,  conducted interviews with officials from the offices of the Department  of Defense Chief Information Officer (CIO), Department of the Navy  CIO, and Assistant Secretary of the Navy for Research, Development,  and Acquisition, including officials in the NMCI program office,  identified the NMCI strategic goals, related performance categories,  associated performance targets, and actual performance data through  document reviews and interviews,  developed an analysis showing NMCI’s performance relative to the  strategic goals, performance categories, and targets based upon  available actual performance data, and  shared our analysis with program officials and adjusted the analysis  based on comments and additional data they provided.
Why GAO Did This Study

The Navy Marine Corps Intranet (NMCI) is a 10-year, $9.3 billion information technology services program. Through a performance-based contract, the Navy is buying network (intranet), application, and other hardware and software services at a fixed price per unit (or "seat") to support about 550 sites. GAO prepared this report under the Comptroller General's authority as part of a continued effort to assist Congress and reviewed (1) whether the program is meeting its strategic goals, (2) the extent to which the contractor is meeting service level agreements, (3) whether customers are satisfied with the program, and (4) what is being done to improve customer satisfaction. To accomplish this, GAO reviewed key program and contract performance management-related plans, measures, and data and interviewed NMCI program and contractor officials, as well as NMCI customers at shipyards and air depots.

What GAO Found

NMCI has not met its two strategic goals--to provide information superiority and to foster innovation via interoperability and shared services. Navy developed a performance plan in 2000 to measure and report progress towards these goals, but did not implement it because the program was more focused on deploying seats and measuring contractor performance against contractually specified incentives than determining whether the strategic mission outcomes used to justify the program were met. GAO's analysis of available performance data, however, showed that the Navy had met only 3 of 20 performance targets (15 percent) associated with the program's goals and nine related performance categories. By not implementing its performance plan, the Navy has invested, and risks continuing to invest heavily, in a program that is not subject to effective performance management and has yet to produce expected results. GAO's analysis also showed that the contractor's satisfaction of NMCI service level agreements (contractually specified performance expectations) has been mixed. Since September 2004, while a significant percentage of agreements have been met for all types of seats, others have not consistently been met, and still others have generally not been met. Navy measurement of agreement satisfaction shows that performance needed to receive contractual incentive payments for the most recent 5-month period was attained for about 55 to 59 percent of all eligible seats, which represents a significant drop from the previous 9-month period. GAO's analysis and the Navy's measurement of agreement satisfaction illustrate the need for effective performance management, to include examining agreement satisfaction from multiple perspectives to target needed corrective actions and program changes. GAO analysis further showed that NMCI's three customer groups (end users, commanders, and network operators) vary in their satisfaction with the program. More specifically, end user satisfaction surveys indicated that the percent of end users that met the Navy's definition of a satisfied user has remained consistently below the target of 85 percent (latest survey results categorize 74 percent as satisfied). Given that the Navy's definition of the term "satisfied" includes many marginally satisfied and arguably somewhat dissatisfied users, this percentage represents the best case depiction of end user satisfaction. Survey responses from the other two customer groups show that both were not satisfied. GAO interviews with customers at shipyards and air depots also revealed dissatisfaction with NMCI. Without satisfied customers, the Navy will be challenged in meeting program goals. To improve customer satisfaction, the Navy identified various initiatives that it described as completed, under way, or planned. However, the initiatives are not being guided by a documented plan(s), thus limiting their potential effectiveness. This means that after investing about 6 years and $3.7 billion, NMCI has yet to meet expectations, and whether it will is still unclear.