The Federal Air Marshal Service’s Operational Approach to Achieving Its Core Mission Is Based on Risk-Related Factors

FAMS’s operational approach (concept of operations) is based on risk- related factors, such as assessments of threat, vulnerability, and  consequences. FAMS is guided by the provisions of the Aviation and  Transportation Security Act that specify the deployment of federal air  marshals on flights presenting high-security risks, such as nonstop, long- distance flights targeted on September 11, 2001. FAMS seeks to maximize  coverage of high-risk flights by establishing coverage goals for 10 targeted  critical flight categories. An Independent Assessment Concluded That the Federal Air Marshal Service’s Approach for Achieving Its Core Mission Was Reasonable; Recommendations for Enhancing the Approach Are Being Implemented

In its 2003 PART review of FAMS, OMB concluded that an independent  evaluation should be conducted to assess FAMS’s performance related to  aspects of the agency’s concept of operations—particularly aspects  involving flight coverage risk categories, the distribution of covered  flights, and target levels of coverage. The Homeland Security Institute, a  federally funded research and development center, performed this  evaluation and issued a final report in July 2006. The Homeland Security Institute recommended, for  example, that FAMS increase randomness or unpredictability in selecting  flights and otherwise diversify the coverage of flights within various risk  categories. Also,  the institute examined FAMS’s scheduling processes and analyzed outputs  in the form of “coverage” data reflecting when and where air marshals  were deployed on flights. We reviewed the Homeland Security  Institute’s evaluation methodology and generally found it to be reasonable. As presented in table 1, FAMS had implemented or  had ongoing efforts to implement all of the recommended enhancements,  as of October 2008. The Federal Air Marshal Service Modified Its Dress Code and Hotel Policies to Further Protect Air Marshals’ Anonymity

To further protect the anonymity of air marshals while on missions, and in  response to air marshals’ feedback and the working groups’  recommendations, FAMS management revised the dress code policy and  the hotel policy for air marshals in August 2006 and February 2007,  respectively. The Federal Air Marshal Service Conducted a Workforce Satisfaction Survey in 2007; the Potential Usefulness of Future Surveys Could Be Enhanced by Improving the Response Rate and Redesigning Certain Questions

In late fiscal year 2007, FAMS conducted a workforce satisfaction survey  of all staff—not just air marshals—to help determine issues affecting the  ability of agency personnel to perform their jobs—and to obtain feedback  on the effectiveness of measures already taken by management to address  relevant issues. For example, although the 2007 workforce satisfaction  survey had an overall response rate (46 percent) that constituted less than  half of the FAMS workforce, 79 percent of the respondents indicated that  there had been positive changes from the prior year. Regarding future  plans, FAMS expects to administer a workforce satisfaction survey every 2  years. However, the 46 percent response rate was substantially less than the 80  percent rate OMB encourages for federal surveys that require its approval. Several key improvements in  FAMS policies and procedures have resulted from these efforts. Among  other improvements, for example, FAMS amended its policy for flight  check-in and boarding procedures to better ensure the anonymity of air  marshals in mission status. The current FAMS Director, after being designated  in June 2008 to head the agency, issued a broadcast message to all  employees, expressing a commitment to continue applicable processes  and initiatives, including the working group process, listening sessions,  field office visits, and the internal Web site for agency personnel to  provide anonymous feedback to management on any topic. FAMS, to its credit, has established a number of processes and  initiatives—including a workforce satisfaction survey—to address various  operational and quality-of-life issues that affect the ability of air marshals  and other FAMS personnel to perform their aviation-security mission. To what extent has the Federal Air Marshal Service’s operational  approach for achieving its core mission been independently assessed? This report is the public version of a restricted report that we provided to  congressional requesters in December 2008. U.S. commercial passenger carriers make  roughly 28,000 domestic and international flights each day.
Why GAO Did This Study

By deploying armed air marshals onboard selected flights, the Federal Air Marshal Service (FAMS), a component of the Transportation Security Administration (TSA), plays a key role in helping to protect approximately 29,000 domestic and international flights operated daily by U.S. air carriers. GAO was asked to examine (1) FAMS's operational approach or "concept of operations" for covering flights, (2) to what extent this operational approach has been independently evaluated, and (3) the processes and initiatives FAMS established to address workforce-related issues. GAO analyzed documented policies and procedures regarding FAMS's operational approach and a July 2006 classified report based on an independent evaluation of that approach. Also, GAO analyzed employee working group reports and other documentation of FAMS's processes and initiatives for addressing workforce-related issues, and interviewed the FAMS Director, other senior officials, and 67 air marshals (selected to reflect a range in levels of experience). This report is the public version of a restricted report (GAO-09-53SU) issued in December 2008.

What GAO Found

Because the number of air marshals is less than the number of daily flights, FAMS's operational approach is to assign air marshals to selected flights it deems high risk--such as the nonstop, long-distance flights targeted on September 11, 2001. In assigning air marshals, FAMS seeks to maximize coverage of flights in 10 targeted high-risk categories, which are based on consideration of threats, vulnerabilities, and consequences. In July 2006, the Homeland Security Institute, a federally funded research and development center, independently assessed FAMS's operational approach and found it to be reasonable. However, the institute noted that certain types of flights were covered less often than others. The institute recommended that FAMS increase randomness or unpredictability in selecting flights and otherwise diversify the coverage of flights within the various risk categories. As of October 2008, FAMS had taken actions (or had ongoing efforts) to implement the Homeland Security Institute's recommendations. GAO found the institute's evaluation methodology to be reasonable. To address workforce-related issues, FAMS's previous director, who served until June 2008, established a number of processes and initiatives--such as working groups, listening sessions, and an internal Web site--for agency personnel to provide anonymous feedback to management on any topic. These efforts have produced some positive results. For example, FAMS revised its policy for airport check-in and aircraft boarding procedures to help protect the anonymity of air marshals in mission status, and FAMS adjusted its flight scheduling process for air marshals to support a better work-life balance. The air marshals GAO interviewed expressed satisfaction with FAMS efforts to address workforce-related issues. Further, the current FAMS Director, after being designated in June 2008 to head the agency, issued a broadcast message to all employees, expressing a commitment to continue applicable processes and initiatives. Also, FAMS has plans to conduct a workforce satisfaction survey of all employees every 2 years, building upon an initial survey conducted in fiscal year 2007. Although the 2007 survey indicated positive changes since the prior year, it was answered by 46 percent of the workforce, well short of the 80-percent response rate that the Office of Management and Budget (OMB) encourages for ensuring that results reflect the views of the target population. OMB guidance gives steps, such as extending the cut-off date for responding, that could improve the response rate of future surveys. Also, several of the 2007 survey questions were ambiguous, and response options were limited. Addressing these design considerations could enhance future survey results.