As shown in figure 3 , FEMA  announced in March 2010 that TFER would not continue past the pilot  stage, but allowed the pilot states to draw down the remaining TFER funds  and continue to complete initiatives started under TFER through August  2011. FEMA Implemented Some, but Not All Elements of Sound Management Practices in Its Administration of the TFER Pilot Program, and Does Not Have Policy Guidance on Developing Pilot Programs

FEMA developed program objectives and a data collection plan—elements  of sound management practices—but did not implement other elements of  sound management practices, such as documenting standards for  determining the program’s success and did not always follow its stated  processes and procedures for TFER. However,  officials in four states reported that it would have been helpful if FEMA  had provided more guidance on plan development. FEMA Took Steps to Collect and Analyze Data from States to Evaluate TFER, but Did Not Always Include Elements of Sound Management Practices

FEMA developed a data collection plan but did not reliably collect data for  the pilot evaluation, and FEMA did not develop a data analysis plan to  describe how it would track TFER’s performance and evaluate its  effectiveness based on standards to determine program performance. FEMA administered the TFER pilot program, but other key stakeholders in  federal disaster response, such as the pilot states and DOD, could benefit  from the results of FEMA’s evaluation of the TFER pilot program. In the absence of this, FEMA does not have a systematic  approach for designing, administering, and evaluating such programs. By developing and implementing policy guidance that  includes sound management practices, FEMA could be better positioned  to ensure its pilot programs meet their intended goals. All Pilot States Have Made Progress in Addressing the Pilot Program Objectives, but No State Has Fully Addressed All Three Objectives

All five states have taken steps to follow FEMA’s TFER Information  Package and TFER Evaluation Criteria document to address the TFER  pilot program’s three objectives—develop plans, build relationships, and  document lessons learned—and these efforts are ongoing. Second, all five states built relationships with  stakeholders, including FEMA and state agencies, but coordination with  DOD—specifically, with the Defense Coordinating Officers and  Emergency Preparedness Liaison Officers—was limited. Two of Five States Have Made Substantial Progress in Developing Plans, and TFER Planners in All States Reported Using FEMA’s Planning Process to Develop Initiatives that Fill Gaps in State Catastrophic Planning

Four of the states are developing plans and all five pilot states reported  they are using the planning process outlined in FEMA’s CPG 101. All Five States Made Progress in Building Relationships with Stakeholders such as FEMA and State Agencies, but Coordination with Federal Military Stakeholders Was Limited

relevant stakeholders. All Five Pilot States Have Made Some Progress in Documenting Lessons Learned, but Four States Have Not Exercised Plans to Evaluate TFER’s Effectiveness

All states have documented lessons learned to date, but four states have  not exercised TFER plans to determine their effectiveness in the event of  an emergency. For  example, all five states considered the pilot a success; Hawaii,  Massachusetts, South Carolina, and West Virginia officials reported that  TFER allowed their states to conduct catastrophic planning that otherwise  would not have occurred, and Washington reported that TFER advanced  its catastrophic and emergency logistics planning by at least 2 years. Conclusions

Hurricane Katrina highlighted gaps in the nation’s preparedness to  respond effectively to catastrophic incidents. Appendix I: Scope and Methodology

To identify the extent to which the Federal Emergency Management  Agency (FEMA) followed sound management practices in developing,  administering, and evaluating the Task Force for Emergency Readiness  (TFER) pilot program, we analyzed program guidance and other key  documents including the TFER Pilot Information Package and the  Guidance and Application Kit. We conducted site visits to all five participant states from  June through September 2010. Our assessment of the extent to which each state addressed FEMA’s  stated objectives for the TFER pilot is presented in appendix IV. 1. Please provide copies of completed / draft plans.
Why GAO Did This Study

Hurricane Katrina in 2005 highlighted gaps in the nation's preparedness to respond effectively to catastrophic incidents. The Department of Homeland Security's (DHS) Federal Emergency Management Agency (FEMA) designed the Task Force for Emergency Readiness (TFER) pilot program to advance and integrate state and federal catastrophic planning efforts. TFER, first envisioned by the Department of Defense (DOD), ran from September 2008 to September 2010, and FEMA provided the five participating states--Hawaii, Massachusetts, South Carolina, Washington, and West Virginia--with $350,000 each to develop plans, build relationships with stakeholders, and document lessons learned (i.e., TFER's stated objectives). As requested, GAO evaluated the extent to which (1) FEMA followed sound management practices in designing, administering, and evaluating TFER and (2) the five participating states satisfied TFER's stated objectives. GAO analyzed FEMA guidance, such as the TFER Pilot Information Package, conducted site visits to all five participant states, and met with relevant FEMA and DOD officials, to evaluate FEMA's management of TFER and the states' implementation of it.

What GAO Found

FEMA developed program objectives and procedures for administering the TFER pilot, but did not develop other elements of sound management practices in designing, administering, and evaluating pilot programs that GAO identified from its prior work and social science literature. FEMA developed objectives for the pilot, but did not document standards for determining the pilot's success. FEMA also provided resources such as funding, training, and support, but FEMA did not always follow the procedures it established for TFER. For example, FEMA did not consistently conduct biweekly conference calls with the states, and four states reported that it would have been helpful if FEMA provided more guidance. FEMA did not develop a data analysis plan, which could have better ensured FEMA collected data on the extent to which the pilot states coordinated with key stakeholders and provided reasonable assurance that FEMA conducted a systematic assessment of TFER using comparable data across the five pilot states. In Spring 2010, FEMA announced TFER would not continue past the pilot stage before evaluating TFER on its merits in strengthening and advancing state catastrophic planning, but FEMA allowed the states to draw down the remaining TFER funds and continue to complete initiatives started under TFER through August 2011. GAO previously reported in April 2009 that FEMA faced challenges in assessing pilot program data, and FEMA officials reported the agency does not have pilot program policy guidance. In the absence of this, FEMA lacks a systematic approach to developing, administering, and evaluating pilot programs. FEMA could better ensure other pilot programs meet their intended goals by developing policies and guidance that include sound management practices. All five states have taken steps to follow FEMA's guidance to address TFER's objectives, but no state has fully addressed them all. First, two of the five states have completed draft catastrophic plans, and all five states reported following FEMA's planning process. Second, all five states built relationships with stakeholders such as state agencies and FEMA, but state officials said coordination with DOD--a key federal stakeholder who may be called upon to assist in disaster response--was limited. State officials reported not coordinating with DOD because they did not have draft plans for DOD officials to review. Third, all states have documented lessons learned to date, but four states have not exercised TFER plans to determine their effectiveness in the event of an emergency.

What GAO Recommends

GAO recommends that FEMA develop policies and guidance that follow sound management practices for future pilot programs, and share TFER results with stakeholders. FEMA agreed with GAO's recommendations.