Congress created DOT&E in response to reports of conflicts of interest in the acquisition community’s oversight of operational testing leading to inadequate testing of operational suitabilityand effectiveness and the fielding of new systems that performed poorly. By law, DOT&E serves as the principal adviser on operational test and evaluation in DOD and bears several key responsibilities, including monitoring and reviewing all operational test and evaluation in DOD, reporting to the Secretary of Defense and congressional committees whether the tests and evaluations of weapon systems were adequate and whether the results confirmed that the system is operationally suitable and effective for combat before a decision is made to proceed to full-rate production, and submitting to the Secretary of Defense and congressional decisionmakers an annual report summarizing operational test and evaluation activities during the preceding fiscal year. The following year, DOD proposed legislative changes that would have reduced the scope and authority of DOT&E. The impact of DOT&E oversight varies with the system under development. DOT&E Recommended Follow-On Operational Test and Evaluation

When DOT&E concludes that a weapon system has not fully demonstrated operational suitability or effectiveness, or if new testing issues arise during initial operational test and evaluation, it may recommend that follow-on operational test and evaluation be done after the full-rate production decision. However, several current trends have the potential to adversely affect DOT&E’s independence and its ability to affect operational test and evaluation, including (1) service challenges to DOT&E’s authority to require and oversee follow-on operational testing and evaluation, (2) declining resources available for oversight, (3) the management of limited resources to address competing priorities, (4) DOT&E’s participation in the acquisition process as a member of the program manager’s working-level integrated product teams, and (5) greater integration of developmental and operational testing. DOT&E’s impact on operational testing is dependent upon its ability to manage these divergent forces while maintaining its independence. The foundation of interagency (i.e., DOT&E and service operational test agencies) relations is based on the independence of DOT&E, its legislative mandate, and its independent reporting to Congress. An unfavorable DOT&E report does not necessarily prevent full-rate production. DOT&E oversight reduces, but does not eliminate, the risk that new systems will not be operationally effective and suitable. In conducting our 13 case studies, we assessed the strengths and weaknesses of the organizational framework in DOD for operational testing via test agency representatives, an assessment on the origins and implementation (exemplified by the 13 cases) of the title 10 amendments creating and empowering DOT&E, and a review of the literature. Efforts to enhance the efficiency of acquisition, in general—and in operational testing, in particular—need to be well balanced with the requirement to realistically and thoroughly test operational suitability and effectiveness prior to the full-rate production decision. 2.
Why GAO Did This Study

Pursuant to a congressional request, GAO reviewed the Department of Defense's (DOD) Office of the Director of Operational Test and Evaluation's (DOT&E) operations and organizational structure for overseeing operational testing, focusing on: (1) DOT&E's efforts and their impact on the quality of operational testing and evaluation in DOD; and (2) the strengths and weaknesses of the current organizational framework in DOD for operational testing.

What GAO Found

GAO noted that: (1) GAO's review of 13 case studies indicated that DOT&E oversight of operational testing and evaluation increased the probability that testing would be more realistic and more thorough; (2) specifically, DOT&E was influential in advocating increasing the reliability of the observed performance and reducing the risk of unknowns through more thorough testing, conducting more realistic testing, enhancing data collection and analysis, reporting independent findings, and recommending follow-on operational test and evaluation when suitability or effectiveness was not fully demonstrated prior to initiating full-rate production; (3) the independence of DOT&E--and its resulting authority to report directly to Congress--is the foundation of its effectiveness; (4) that independence, along with its legislative mandate, provides sufficient freedom and authority to exercise effective oversight of the operational testing and evaluation of new systems before a decision is made to begin full-rate production; (5) DOT&E can reduce the risk that systems are not adequately tested prior to the full-rate production decision but DOT&E cannot ensure that: (a) only systems whose operational effectiveness and suitability have been demonstrated through operational testing will proceed to the full-rate production decision; or (b) new fielded systems will accomplish their missions as intended or that the fielded systems are safe, survivable, and effective; (6) DOT&E management must balance its oversight responsibilities for operational testing with the broader acquisition priorities of program managers and service test agencies; (7) though supportive of DOT&E's mission and independence, program and service representatives frequently considered the time, expense, and resources expended to accommodate DOT&E concerns to be ill-advised; (8) several current trends may challenge DOT&E's ability to manage its workload and its ability to impact operational test and evaluation: (a) service challenges to DOT&E's authority to require and oversee follow-on operational testing and evaluation; (b) a decline in resources available for oversight; (c) an expansion of DOT&E involvement in activities other than oversight of major acquisition programs; (d) participation of DOT&E in the acquisition process as a member of working-level integrated product teams; and (e) greater integration of developmental and operational testing; and (9) these trends make it imperative that DOT&E prioritize its workload to achieve a balance between the oversight of major defense acquisition programs and other initiatives important to the quality of operational test and evaluation.