DOD Has Participation Data, but Has Not Developed Performance Measures or Clarified Roles and Responsibilities to Help Ensure Effective Program Implementation

DOD has data on participation in and costs of the Armed Forces Sports  Program, but has not taken steps, including developing performance  measures and clarifying roles and responsibilities, that are needed to help  ensure that the program is implemented effectively. Servicemember Participation

In analyzing the number of servicemembers participating in the program,  we found that servicemember participation changed from 968  servicemembers in fiscal year 2012 to 848 servicemembers in fiscal year  2016. Table 4 breaks out these data for  each year from fiscal years 2012 through 2016. Program costs ranged from about $2.1 million to about $2.8  million from fiscal years 2014 through 2016. While these data provide important  context about the program’s size and reach, they are outputs and do not  constitute performance measures because they do not exhibit several of  the key attributes previously discussed. While the  Sports Council Secretariat’s data included the “actual” number of program  participants, they did not identify projected performance targets that  would enable program officials to determine how far they have  progressed toward a desired outcome or end state. Specifically, officials cited the program’s  emphasis on a higher level of physical fitness than is otherwise required  by the services as contributing to individual servicemember readiness,  and involvement in national and international sports championships as  aiding recruiting efforts because it showcases some of the unique  opportunities open to those in the services. However, outside of participation  and cost data and some anecdotal examples, officials did not have  specific measures for or data on the Armed Forces Sports Program’s  contribution to the services’ readiness, recruiting, and retention efforts. Third, while DOD has program participation data, it does not track  baseline and trend data in order to assess the program’s performance  and progress over time. DOD’s Implementation of Armed Forces Sports Program’s Roles and Responsibilities Differs from Those Currently Defined in Policy

The roles and responsibilities that are currently being implemented for the  Armed Forces Sports Program differ from the program’s roles and  responsibilities specified in DOD policy. DOD Instruction 1330.04 and the  Armed Forces Sports Council’s standard operating procedures specify  that the Armed Forces Sports Program includes training or national  qualifying events in preparation for participation in International Military  Sports Council events, the Pan American Games, the Olympic Games,  the Paralympic Games, and other international competitions. While this  is how the program is defined in key program documents, the Office of  the Secretary of Defense, Sports Council Secretariat, and service officials stated that all responsibilities, including costs, associated with  servicemember participation in the Pan American, Olympic, and  Paralympic Games are, in practice, handled by the services. The Office of the Secretary of Defense and Sports Council Secretariat  officials stated they plan to review DOD Instruction 1330.04 and make  necessary updates but did not indicate what specific changes would be  made to clarify the program’s roles and responsibilities. Recommendations for Executive Action

To improve the management of the Armed Forces Sports Program and  better determine whether the program is achieving its desired results, we  recommend that the Secretary of Defense direct the Under Secretary of  Defense for Personnel and Readiness to develop and implement  performance measures for the Armed Forces Sports Program that  measure the desired outcomes for the program and, at a minimum,  demonstrate linkage to the program’s goals or mission, have a  measurable target, and include a baseline that can be used to  demonstrate program performance. DOD concurred with our recommendation to develop and implement  performance measures for the Armed Forces Sports Program that  measure the desired outcomes for the program and, at a minimum,  demonstrate linkage to the program’s goals or mission, have a  measurable target, and include a baseline that can be used to  demonstrate program performance but also noted potential limitations on  establishing measures. However, DOD stated that there are limitations  on establishing metrics for several of the program’s objectives, such as  goodwill and positive image, which are challenging to measure. The time frame of the participant and cost data that we obtained differs  because DOD officials stated that fiscal year 2014 was the most recent  year that cost data were available from all the services.
Why GAO Did This Study

For nearly a century, the U.S. Armed Forces (i.e., the Army, the Navy, the Marine Corps, the Air Force, and the Coast Guard) have organized and participated in international and national sporting competitions in part because of the intended benefits for servicemember morale and the unique opportunity that participation provides to foster diplomatic relations.
House Report 114-537 accompanying a bill for the National Defense Authorization Act for Fiscal Year 2017 included a provision for GAO to review the Armed Forces Sports Program and its impact on the military services' readiness. This report assesses the effectiveness of DOD's implementation of the Armed Forces Sports Program. GAO analyzed participation data for fiscal years 2012 through 2016 and cost data for fiscal years 2014 through 2016, compared DOD data with attributes of successful performance measures, compared roles and responsibilities specified in policy with those being implemented, and interviewed DOD officials.

What GAO Found

The Department of Defense (DOD) has data on participation in and costs of the Armed Forces Sports Program, but has not taken steps, including developing performance measures and clarifying roles and responsibilities that are needed to help ensure the program is implemented effectively. DOD officials stated that they use sport and competition participation data to measure the performance and effectiveness of the program. According to these data, servicemember participation changed from 968 servicemembers in fiscal year 2012 to 848 servicemembers in fiscal year 2016, and program costs ranged from about $2.1 million to about $2.8 million in fiscal years 2014 through 2016. While these data provide important context about the program's size and reach, they do not exhibit several key attributes, such as linkage, a measurable target, and baseline and trend data that GAO has found are key to successfully measuring a program's performance.
First, these data do not exhibit linkage because no relationship has been established to show how the number of servicemember participants contribute to achievement of the program's objectives, such as promoting goodwill among and a positive image of the U.S. Armed Forces through sports.
Second, these data were not associated with a measurable target that would enable program officials to determine how far the program has progressed toward a desired outcome or end state.
for measures that are able to assess the program's performance and progress over time.
Without performance measures that demonstrate these attributes, DOD will be unable to effectively demonstrate that it is achieving the intended benefits of the program, such as improving readiness, recruitment, and retention as well as promoting the goodwill of the U.S. Armed Forces. Officials cited the program as aiding recruiting because it showcased unique opportunities open to those in the U.S. Armed Forces. However, outside of participation and cost data and some anecdotal examples, officials did not have specific measures for or data on the Armed Forces Sports Program's contribution to the services' readiness, recruiting, and retention efforts.
The roles and responsibilities that are currently being implemented for the program differ from the program's roles and responsibilities specified in DOD policy. DOD Instruction 1330.04 specifies that the program includes training or national qualifying events in preparation for participation in International Military Sports Council events, the Pan American Games, the Olympic Games, the Paralympic Games, and other international competitions. While this is how the program is defined in key program documents, DOD officials stated that all responsibilities, including costs, associated with servicemember participation in the Pan American, Olympic, and Paralympic Games are handled by the services. DOD officials stated that they plan to review DOD Instruction 1330.04 and make necessary updates, but have not yet determined what specific changes would be made to clarify the program's roles and responsibilities.

What GAO Recommends

GAO recommends that DOD develop and implement performance measures for the Armed Forces Sports Program that, at a minimum, demonstrate linkage to the program's goals or mission, have a measurable target, and include a baseline that can be used to demonstrate program performance. DOD concurred with the recommendation, noting potential limitations on establishing measures. GAO acknowledges these limitations, but continues to believe that measures are important to evaluating the program's effectiveness.