	Background

Federal agencies are increasingly expected to demonstrate effectiveness in  achieving agency or governmentwide goals. The first three classifications are represented in our case selection of ACF,  NHTSA, and NSF. Administration for Children and Families (ACF)

ACF, in the Department of Health and Human Services (HHS), oversees  and helps finance programs to promote the economic and social well- being of families, individuals, and communities. Housing and Urban Development (HUD)

The HUD Office of Community Planning and Development (CPD) provides  financial and technical assistance to states and localities in order to  promote community-based efforts to develop housing and economic  opportunities. National Highway Traffic Safety Administration (NHTSA)

To promote highway safety, DOT’s NHTSA develops regulations and  provides financial and technical assistance to states and local  communities. These elements include an evaluation culture, data quality, analytic  expertise, and collaborative partnerships. Agencies  demonstrated an evaluation culture through commitment to self- examination and learning through experimentation. Agency collaboration with federal and other program  partners helped leverage resources and expertise for evaluation. HUD and  the Coast Guard had more ad hoc arrangements in place when questions  about specific initiatives or issues created the demand for evaluations. Analytic Expertise

The agencies reviewed sought access to analytic expertise to ensure  assessments of program results would be systematic, credible, and  objective. Many agencies share  goals with others. Second,  they improved administrative systems or turned to special data collections  to obtain better quality data. The Coast Guard selects experts who  already have military experience. The Coast Guard also contracts for special studies with the agency’s  Research and Development Center, the Center for Naval Analyses, and the  American Bureau of Shipping. Providing Technical Expertise to Program Partners

Because of their reliance on state and local agencies for both  implementing and evaluating their programs, some of the reviewed  agencies found it necessary, in order to improve data quality, to help  develop state and local evaluation expertise. Building Collaborative Partnerships

The five agencies used collaborative partnerships to obtain access to  needed data and expertise for evaluations. Factors That Impede Building Evaluation Capacity

Although agencies used a variety of strategies to maximize evaluation  capacity, they also cited factors that impede conducting evaluations or  improving evaluation capacity, including the following:    Constraints on spending program resources on oversight: Some agency  officials claimed that the lack of a statutory mandate or dedicated funds  for evaluation impeded investing program funds to conduct studies or to  improve administrative data. Observations

The five agencies we reviewed employed various strategies to obtain  useful evaluations of program effectiveness. As other agencies aim to  develop evaluation capacity, the examples in this report may help them  identify ways to obtain the data and expertise needed to produce useful  and credible information on results. Office of National Drug Control Policy. Grant Programs: Design Features Shape Flexibility, Accountability, and  Performance Information.
Why GAO Did This Study

Agencies are increasingly asked to demonstrate results, but many programs lack credible performance information and the capacity to rigorously evaluate program results. To assist agency efforts to provide credible information, GAO examined the experiences of five agencies that demonstrated evaluation capacity in their performance reports: the Administration for Children and Families (ACF), the Coast Guard, the Department of Housing and Urban Development (HUD), the National Highway Traffic Safety Administration (NHTSA), and the National Science Foundation (NSF).

What GAO Found

In the five agencies GAO reviewed, the key elements of evaluation capacity were an evaluation culture--a commitment to self-examination, data quality, analytic expertise, and collaborative partnerships. ACF, NHTSA, and NSF initiated evaluations regularly, through a formal process, while HUD and the Coast Guard conducted them as specific questions arose. Access to credible, reliable, and consistent data was critical to ensure findings were trustworthy. These agencies needed access to expertise in both research methods and subject matter to produce rigorous and objective assessments. Collaborative partnerships leveraged resources and expertise. ACF, HUD, and NHTSA primarily partnered with state and local agencies; the Coast Guard partnered primarily with federal agencies and the private sector. The five agencies used various strategies to develop and improve evaluation: Commitment to learning from evaluation developed to support policy debates and demands for accountability. Some agencies improved administrative systems to improve data quality. Others turned to specialized data collection. All five agencies typically contracted with experts for specialized analyses. Some agencies provided their state partners with technical assistance. These five agencies used creative strategies to leverage resources and obtain useful evaluations. Other agencies could adopt these strategies--with leadership commitment--to develop evaluation capacity, despite possible impediments: constraints on spending, local control over flexible programs, and restrictions on federal information collection. The agencies agreed with our descriptions of their programs and evaluations.