Data provided by the Army on the first 6 months of contractor operations indicated that Cendant had received and processed 793 relocation authorizations and arranged (booked) 581 moves. Cendant also assisted members in initiating 181 Do-It-Yourself moves. The Army’s Plans to Evaluate the Pilot Project Need to Be Further Refined

The Army has developed an evaluation plan to facilitate data collection on quality of life, total cost, and impact on small business; define agency responsibilities; and explain which data will be used for comparative purposes. Transportation and storage costs under the pilot will be compared with what the Army would have paid under the MTMC program. Prior to commenting on a draft of this report, the Army had not explained what constituted success for the pilot, how much weight each factor (quality of life, total cost, and impact on small business) should have in determining overall success, and whether there were thresholds that specific factors should attain to be considered successful. Such advance determinations are typically part of a sound study methodology to enhance the credibility of the results and to avoid any perception of bias. The information provides thresholds for determining the success of the overall Hunter pilot and the relative importance of each evaluation factor. The Army Audit Agency, however, has not yet completed its validation of those responses. Other Pilot Projects or Tests Underway or Planned

Other pilots or tests are also underway or are being planned in DOD. The Army Hunter pilot, on the other hand, is limited to one location, Hunter, and one service, the Army, although it provides for move management services, including entitlement counseling and destination relocation assistance, for all Hunter Army families, relocating to anywhere in the world. USTRANSCOM will also maintain oversight of the personal property program pilots and tests and assure consistent evaluation criteria and assessment of the pilot program results. Notwithstanding this and other pilots and tests planned or underway, DOD and Army officials briefed us at the end of this review about plans to expand the Hunter pilot by testing it at selected DOD sites. The information, with some minor exceptions, represents important steps toward formulating a sound evaluation plan. However, the evaluation method has not been finalized or made a formal part of the evaluation plan, and questions remain about the definition of terms and small business group measurements. We recognize that further issues could be identified as the evaluation method is finalized. In agreeing with our draft recommendation, DOD provided additional information regarding the Army’s method for determining what results it plans to use to judge the pilot a success. We reviewed the data and observed how it was being collected. GAO Comments

1. 2.
Why GAO Did This Study

Pursuant to a legislative requirement, GAO provided information on the Army's Hunter pilot program, an alternative approach for providing relocation services for its personnel stationed at Hunter Army Airfield, Savannah, Georgia, focusing on: (1) data collected to date; (2) GAO's observations on how the Army plans to evaluate the data; and (3) other pilot tests that are under way or planned in the Department of Defense (DOD).

What GAO Found

GAO noted that: (1) through the first 6 months of operation, the Hunter pilot contractor had received and processed nearly 800 requests for transportation and relocation services, arranged for transportation of nearly 600 personal property shipments, and assisted Army members initiate another 200 do-it-yourself moves; (2) because many of the shipments had not been completed at the time of GAO's review, the Army had paid for only about 200 of these shipments, at a cost of about $500,000, but it was actively collecting data on all the factors--quality of life, cost, and impact on small businesses--it was planning to use in its evaluation of the pilot; (3) however, at the time GAO completed its work, the Army had not determined what it would consider successful within each factor or how much weight each factor would have in determining overall success; (4) specifically, the Army had not determined criteria to judge the success or failure of the pilot to help it assess whether the pilot was performing better or worse than the existing program; (5) making such determinations before pilot program data are analyzed would be important to enhancing the credibility of the Army's assessment and for use in making comparisons with other pilot programs that are under way or planned in DOD; (6) these include pilots under way by the Navy or planned by the Military Traffic Management Command; (7) also, DOD and the Army are considering expanding the pilot now being tested at Hunter Army Airfield to other DOD sites; (8) subsequently, in response to GAO's draft report, the Army provided it with new information on how it will determine the results of the pilot; (9) the information, with some minor exceptions, represents important steps toward formulating a sound evaluation plan; (10) however, the evaluation method has not yet been finalized or made a formal part of the evaluation plan, and questions remain about the definition of terms and small business group measurements; and (11) GAO recognized that further issues could be identified as the evaluation method is refined and finalized.