Our analysis of the September 2003 DTS economic analysis found that two  key assumptions used to estimate cost savings were not based on reliable  information. Two primary areas—personnel savings and reduced  CTO fees—represented the majority of the over $56 million of estimated  annual net savings DTS was expected to realize. Air Force and Navy DTS  program officials stated that they did not anticipate a reduction in the  number of personnel with the full implementation of DTS, but rather the  shifting of staff to other functions. Also, as part of the  Navy’s overall evaluation of the economic analysis, program officials  stated that “the Navy has not identified, and conceivably will not  recommend, any personnel billets for reduction.” Finally, the Naval Cost  Analysis Division (NCAD) October 2003 report on the economic analysis  noted that it could not validate approximately 40 percent of the Navy’s  total costs, including personnel costs, in the DTS life-cycle cost estimates  because credible supporting documentation was lacking. DOD strongly objected to our finding that the personnel savings are  unrealistic. However, based on our review and analysis of documentation and  discussion with department personnel, we found that the underlying  assumptions in support of the $54.1 million were not valid, particularly in  regard to the amounts estimated for the Navy and Air Force. Savings Associated with Reduction of CTO Fees Are Unknown

According to the September 2003 economic analysis, DOD expected to  realize annual net savings of $31 million through reduced fees paid to the  CTOs because the successful implementation of DTS would enable the  majority of airline tickets to be acquired with either no or minimal  intervention by the CTOs. Rather, the  sole support provided by the PMO-DTS was an article in a travel industry  trade publication. The article was not based on information related to  DTS, but rather on the experience of one private sector company. This contract applies to the Defense Travel Region  6 travel area. Specific examples of failures to effectively implement the DOD policy on  conducting economic analyses include the (1) DTS life-cycle cost  estimates portion of the economic analysis was not independently  validated as specified in DOD’s guidance and (2) September 2003 DTS  economic analysis did not undertake an assessment of the effects of the  uncertainty inherent in the estimates of benefits and costs, as required by  DOD and OMB guidance. Because an economic analysis uses estimates  and assumptions, it is critical that a sensitivity analysis be performed to  understand the effects of the imprecision in both underlying data and  modeling assumptions. Presently, the reported DTS utilization is based on a DTS Voucher Analysis  Model that was developed in calendar year 2003 using estimated data, but  over the years has not been completely updated with actual data. This lack of accurate and pertinent  utilization data hinders management’s ability to monitor its progress  toward the DOD vision of DTS as the standard travel system, as well as to  provide consistent and accurate data to Congress. DOD Has Taken Steps to Improve DTS Utilization, but Further Action Is Needed

Although the military services have issued various memorandums aimed at  increasing the utilization of DTS, the military service DTS program  officials all pointed to ineffective training as a primary cause of DTS not  being utilized to a far greater extent. While these actions are a positive step  forward, they do not address the fundamental problem that DTS’s  requirements are still ambiguous and conflicting—a primary cause of the  previous problems. Our February 2006  analysis of selected flight information disclosed that DOD still did not have  reasonable assurance that DTS displayed flights in accordance with its  stated requirements. For example, DOD has retained a  requirement to display 25 flights for each inquiry. Until DOD improves DTS requirement management  practices, it will not have this assurance. Equally important, however, will be the department’s  ability to resolve the long-standing difficulties that DTS has encountered  with its requirements management and system testing.
Why GAO Did This Study

In 1995, the Department of Defense (DOD) began an effort to implement a standard departmentwide travel system. The Defense Travel System (DTS) is envisioned as DOD's standard end-to-end travel system. This testimony is based on GAO's September 2006 related report. Today's testimony highlights GAO's key findings with regard to the following objectives: (1) Were the two key assumptions made in the September 2003 economic analysis reasonable? (2) Was DOD taking action to ensure full utilization of DTS and gathering the data needed to monitor DTS utilization? and (3) Has DOD resolved several functional problems associated with weak system requirements and testing? To address these objectives, GAO (1) reviewed the September 2003 DTS economic analysis, (2) analyzed DTS utilization data, and (3) analyzed DTS flight information.

What GAO Found

GAO's analysis of the September 2003 DTS economic analysis found that the two key assumptions used to estimate annual net savings were not based on reliable information. Two cost components represent the majority of the over $56 million in estimated net savings--personnel savings and reduced commercial travel office (CTO) fees. In regard to the personnel savings, GAO's analysis found that the $24.2 million of personnel savings related to the Air Force and the Navy were not supported. Air Force and Navy DTS program officials stated that they did not anticipate a reduction in the number of personnel, but rather the shifting of staff from the travel function to other functions. The Naval Cost Analysis Division stated that the Navy will not realize any tangible personnel cost savings from the implementation of DTS. In regard to the CTO fees, the economic analysis assumed that 70 percent of all DTS airline tickets would either require no intervention or minimal intervention from the CTOs, resulting in an estimated annual net savings of $31 million. However, the sole support provided by the DTS program office was an article in a trade industry publication. The article was not based on information related to DTS, but rather on the experience of one private sector company. Furthermore, the economic analysis was not prepared in accordance with guidance prescribed by the Office of Management and Budget and DOD. DOD guidance stated that the life-cycle cost estimates should be verified by an independent party, but this did not occur. The economic analysis did not undertake an assessment of the effects of the uncertainty inherent in the estimates of benefits and costs. Because an economic analysis uses estimates and assumptions, it is critical that the imprecision in both the underlying data and assumptions be understood. Such an assessment is referred to as a sensitivity analysis. DOD acknowledged that DTS is not being used to the fullest extent possible, but lacks comprehensive data to effectively monitor its utilization. DOD's utilization data are based on a model that was developed in calendar year 2003. However, the model has not been completely updated to reflect actual DTS usage. The lack of accurate utilization data hinders management's ability to monitor progress toward the DOD vision of DTS as the standard travel system. GAO also found that the military services have initiated actions that are aimed at increasing the utilization of DTS. Finally, GAO found that DTS still has not addressed the underlying problems associated with weak requirements management and system testing. While DOD has acted to address concerns GAO previously raised, GAO found that DTS's requirements are still ambiguous and conflicting. For example, DTS displaying up to 25 flights for each inquiry is questionable because it is unclear whether this is a valid requirement. Until DOD improves DTS's requirements management practices, the department will not have reasonable assurance that DTS can provide the intended functionality.