For example, aspects of modeling and data  were inadequate in some areas because data were lacking and some of the  models used could not simulate all relevant aspects of the missions. Relevant research standards require that models  used are adequate for the intended purpose, represent a complete range of  conditions, and that data used are properly generated and complete. For example, the MCS modeled hypothetical homeland defense missions  rather than homeland defense demands derived from a well defined and  approved concept of operations for homeland defense, because the  specific details of the missions were still being determined, and DOD  acknowledged that the data used are incomplete. The MCS also was unable to model the flexible deterrent  options/deployment order process to move units and equipment into  theater because of lack of data on how deployment orders have been  issued in the past for major combat operations. Exclusive Use of Warfighting Metrics in MCS Analyses Limited Usefulness of Report

While the MCS concluded that combined U.S. and host nation  transportation assets were adequate to meet U.S. objectives with  acceptable risk, the report, in describing the use of warfighting metrics in  its analyses, does not provide a clear understanding of the direct  relationship of warfighting objectives to transportation capabilities. The  report acknowledged that further analysis is required to understand the  operational impact of increased or decreased strategic lift on achieving  warfighting objectives. Relevant generally accepted research standards  require that conclusions be supported by analyses. Results Are Not Always Complete or Presented Clearly and Are Qualified or Contingent on Further Study or Analysis

In some cases, the MCS results were incomplete, unclear, or contingent on  further study, making it difficult to identify findings and evaluate evidence. Relevant research standards require results to be presented in a complete,  accurate, and relevant manner; conclusions to be sound and complete; and  recommendations to be supported by analyses. Our analysis of the MCS  report found that it contains several recommendations for further studies  and assessments, five of which are under way. However, the report does not explain the potential  effect of these ongoing studies on the MCS results after the studies are  complete, nor does DOD have plans to report the effect of these studies on  the MCS results. In addition, the report contains qualified information that is not presented  clearly in the report, such as varying assessments of intratheater assets in  three different places. Conclusions

The methodological limitations in the MCS that we identified—some of  which were acknowledged by DOD in the MCS report—raise questions  about the adequacy and completeness of the study and its report. Furthermore, in the absence of an explanation of how identified modeling  and data limitations could affect the study results or how such limitations  could affect projected mobility capability requirements, the accuracy of  the study’s finding that projected capabilities are adequate to achieve U.S.  objectives with an acceptable level of risk during the period from fiscal  years 2007 through 2013 is unclear. As a result, we suggest that  Congress and other decision makers exercise caution in using the MCS to  make programmatic investment decisions. Recommendation for Executive Action

To provide decision makers with adequate and complete information  concerning mobility capabilities so they are able to clearly understand the  operational implications of the study and make fully informed  programmatic investment decisions, and to improve the usefulness of  future mobility capabilities studies, we recommend that the Secretary of  Defense take the following three actions, when conducting future mobility  capabilities studies beginning with any study currently underway:    develop models and data for all critical missions, such as homeland  defense, and processes, such as the flexible deterrent options/deployment  order process;  include in study reports an explanation of how stated limitations might  impact the study results and, at a minimum, describe how recommended  future studies might be conducted to enhance the results of the original  study; and  incorporate both mobility and warfighting metrics in determining  capabilities.
Why GAO Did This Study

The Department of Defense (DOD) issued the Mobility Capabilities Study (MCS), which was intended to identify and quantify the mobility capabilities needed to support U.S. strategic objectives into the next decade. The MCS found that projected capabilities are adequate to achieve U.S. objectives with an acceptable level of risk--that is, current U.S. inventory of aircraft, ships, prepositioned assets, and other capabilities are sufficient, in conjunction with host nation support, and assuming planned investments take place. The Senate report accompanying the bill for the fiscal year 2005 Defense Authorization Act required GAO to report on the adequacy and completeness of the MCS. GAO assessed the extent to which the MCS met generally accepted research standards that this type of study would be expected to meet to be considered sound and complete.

What GAO Found

DOD used an innovative approach in conducting the study and acknowledged methodological limitations in its report; however, it did not fully disclose how these limitations could affect the MCS conclusions and recommendations. Therefore, it is not transparent how the analyses done for the study support DOD's conclusions. Measured against relevant generally accepted research standards, GAO has identified limitations in the MCS and its report that raise questions about their adequacy and completeness. GAO suggests that Congress and other decision makers exercise caution in using the MCS to make investment decisions. Among GAO's findings: Aspects of modeling and data were inadequate in some areas because data were lacking and the models used could not simulate all relevant aspects of the missions. The report did not explain how these limitations could affect the study results or what the impact on projected mobility capabilities might be. Generally accepted research standards require that models used are adequate for the intended purpose, represent a complete range of conditions, and that data used are properly generated and complete. For example, the MCS modeled hypothetical homeland defense missions rather than homeland defense demands derived from a well defined and approved concept of operations for homeland defense, because the specific details of the missions were still being determined and the data used may be incomplete. The MCS also was unable to model the flexible deterrent options/deployment order process to move units and equipment into theater because of lack of data, but the study assumed a robust use of this process. In addition, the MCS report contains over 80 references to the need for improved modeling or data. While the MCS concluded that combined U.S. and host nation transportation assets were adequate, in describing the use of warfighting metrics in its analyses, the report does not provide a clear understanding of the direct relationship of warfighting objectives to transportation capabilities. Additionally, the report stated that further analysis is required to understand the operational impact of increased or decreased strategic lift on achieving warfighting objectives. Relevant generally accepted research standards require that conclusions be supported by analyses. The use of both warfighting and mobility metrics would allow decision makers to know whether combat tasks were achieved and how much strategic transportation is needed to accomplish those tasks. In some cases, the MCS results were incomplete, unclear, or contingent on further study, making it difficult to identify findings and evaluate evidence. Relevant research standards require results to be presented in a complete, accurate, and relevant manner. For example, the report contains recommendations for further studies and assessments, five of which are under way. However, DOD has no plans to report the impact of these studies on the MCS results after the studies are complete. In addition, the report contains qualified information that is not presented clearly, such as varying assessments of intra-theater assets in three different places.