DOD Has Developed Guidance for the Joint Exercise Program and Has Implemented an Approach to Assess Program Effectiveness DOD Has Developed a Body of Guidance for the Joint Exercise Program and Is Working to Update a Key Guidance Document That Is Outdated

DOD has developed a body of guidance for the Joint Exercise Program  and is working to update a key outdated guidance document that  identifies overarching roles and responsibilities for military training in  accordance with a congressional requirement in a House Committee on  Armed Services report accompanying the National Defense Authorization  Act for Fiscal Year 2017. The Director of the Joint Assessment and Enabling  Capability office reviews the performance measures created by the  combatant commands against the SMART rubric and provides input and  coaching on improving the measures through an ongoing and  collaborative process. DOD Uses Two Key Information Technology Systems to Manage the Joint Exercise Program, but the Data in the System That Tracks the Most Recent Funding Execution Data Does Not Have Required Supporting Documentation and Its Data Quality Is Questionable

DOD uses two key information technology systems—JTIMS and the  Execution Management System—to manage the execution of the Joint  Exercise Program, but DOD does not have assurance that the Execution  Management System produces quality information. In addition, TRANSCOM officials  randomly selected exercises in JTIMS to show us the type of information  entered in the system and we noted that the level of detail provided  sometimes varied significantly by combatant command. Furthermore,  officials from two of the four combatant commands we visited stated that  sometimes the information captured in JTIMS is not useful and could  negatively affect their ability to coordinate training with other combatant  commands or extract pertinent information about exercises from the  system that would be helpful in planning them. Consequently, to help  improve the consistency and standardization of information across  combatant commands, the Office of the Chairman of the Joint Chiefs of  Staff published a user guide for JTIMS that is intended to mitigate  inconsistencies in the information entered there, standardize the use of  the system across DOD, and improve the overall understanding of the  system. Based on our review, we found that two— STRATCOM and NORTHCOM—of the four combatant commands we  visited had uploaded supporting documentation, as required by the  Execution Management System guidance, for fiscal years 2013-16. Based on our  review of a nongeneralizable sample of supporting documentation  for fiscal years 2014 through 2016 that was uploaded into the  Execution Management System, we found that the sum of the  individual expenditures reported in supporting documentation did not  match the corresponding total expenditures entered in the system for  any of the four combatant commands we visited. Without  ensuring that supporting documentation is uploaded and implementing  effective internal controls to ensure the completeness and accuracy of  financial information captured for the Joint Exercise Program, DOD and  other key decision makers may not have the correct financial information  to defend the Joint Exercise Program’s budget. Recommendations for Executive Action

To better ensure quality financial execution information is available to  guide the Joint Exercise Program, we recommend that the Secretary of  Defense direct the Office of the Assistant Secretary of Defense for  Readiness to take the following two actions: direct the combatant commanders to take steps to comply with current  Execution Management System guidance to upload supporting  documentation that is reconcilable to funds executed from the  Combatant Commanders Exercise Engagement and Training  Transformation account; and as the department implements financial improvement plans in  accordance with the FIAR guidance, it should include specific internal  control steps and procedures to address and ensure the  completeness and accuracy of information captured for the Joint  Exercise Program’s Combatant Commanders Exercise Engagement  and Training Transformation account. In its  written comments, which are summarized below and reprinted in  appendix IV, DOD partially concurred with both recommendations. Key contributors to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

This report (1) describes guidance the Department of Defense (DOD) has  developed for its Joint Exercise Program and DOD’s approach to assess  the effectiveness of the program and (2) evaluates the extent to which  DOD uses two key information systems—the Joint Training Information  Management System (JTIMS) and the Execution Management System–to  manage the Joint Exercise Program. We  reviewed DOD’s approach to make performance measures specific,  measurable, achievable, realistic, and time-phased (commonly referred to  as the SMART rubric) to assess the return on investment for the Joint  Exercise Program.
Why GAO Did This Study

The Joint Exercise Program is the principal means for combatant commanders to maintain trained and ready forces, exercise contingency and theater security cooperation plans, and conduct joint and multinational training exercises. These exercises are primarily aimed at developing the skills needed by U.S. forces to operate in a joint environment and can also help build partner-nation capacity and strengthen alliances.
House Report 114-102 included a provision for GAO to review DOD's Joint Exercise Program. This report (1) describes guidance DOD has developed for its Joint Exercise Program and DOD's approach to assess the effectiveness of the program and (2) evaluates the extent to which DOD uses two information technology systems to manage the program. GAO observed data in JTIMS and analyzed fiscal years 2014-16 financial data and supporting documentation in the Execution Management System.

What GAO Found

The Department of Defense (DOD) has developed a body of guidance for the Joint Exercise Program and has implemented an approach to assess the effectiveness of the program. In addition to the body of guidance for the program, DOD is working to update a key guidance document for military training in accordance with a congressional requirement. DOD's approach to assess the effectiveness of the Joint Exercise Program is aimed at ensuring that its performance measures are specific, measurable, achievable, realistic, and time-phased (commonly referred to as the SMART rubric). The Joint Assessment and Enabling Capability office reviews the performance measures created by the combatant commands against this rubric and provides input and coaching on improving the measures through an ongoing and collaborative process.
DOD uses two key information technology systems—the Joint Training Information Management System (JTIMS) and the Execution Management System—to manage the execution of the Joint Exercise Program, but does not have assurance that funding execution data in the Execution Management System are reliable. JTIMS is the system of record for the Joint Exercise Program that combatant commanders use to plan and manage their joint training exercises. GAO observed significant variation in the type and quality of information entered in JTIMS. Combatant command and Joint Staff officials stated that information in JTIMS lacked consistency in the level of detail provided, sometimes making it difficult to coordinate training with other combatant commands or extract pertinent information about exercises from the system that would be helpful in planning other exercises. Consequently, to help improve the consistency and standardization of information across combatant commands, the Joint Staff published a user guide for JTIMS.
Regarding the Execution Management System, a web-based database DOD uses to track the most recent funding execution data for the Joint Exercise Program, GAO found that DOD does not have assurance that the system produces quality information because supporting documentation is not consistently uploaded into the system and, when it is uploaded, it is not reconcilable to the data entered there. Only U.S. Strategic Command and U.S. Northern Command uploaded supporting documentation for fiscal years 2013-16 as required by the Execution Management System guidance. Reviewing a nongeneralizable sample of uploaded supporting documentation for fiscal years 2014-16, GAO found that the sum of the individual expenditures reported in supporting documentation did not match corresponding total expenditures entered in the system for any of the four combatant commands included in GAO's review. Further, the four combatant commands GAO visited, the Office of the Chairman of the Joint Chiefs of Staff, and the Office of the Assistant Secretary of Defense for Readiness had not implemented effective internal controls similar to those identified in the Standards for Internal Control in the Federal Government to ensure the completeness and accuracy of financial information captured for the Joint Exercise Program. Without such internal controls, DOD and other key decision makers may not have the financial information of sufficient quality to defend the Joint Exercise Program's budget.

What GAO Recommends

GAO recommends that DOD comply with current guidance to upload supporting documentation in the Execution Management System and implement effective internal controls to ensure the completeness and accuracy of financial information. DOD partially concurred with both recommendations, noting existing controls in other related systems of record. GAO believes the recommendations remain valid, as discussed in this report.