Following the crosscutting review, NASA’s Chief  Operating Officer (COO) determined final ratings during a briefing  attended by the PIO and each of the strategic objective leaders. Clarify and Clearly Define Measurable Outcomes for Each Strategic Objective to Be Reviewed

A strategic review starts with framing the outcome or impact the agency  seeks to achieve. In some cases, defining and measuring the outcome related to a strategic  objective may be relatively straightforward. According to PIO staff, for the success statements, objective  leaders and staff were asked to characterize or define the outcomes of  success in implementing their objectives in the next 10 years by  answering questions such as, “What will the agency have completed,  obtained, contributed, advanced?” NASA officials told us that because it  can be difficult to measure progress towards long-term, scientific  discovery-oriented outcomes, they also rely on underlying multiyear  performance goals, annual performance indicators, and milestones to  better plan for and understand near-term progress towards those  objectives. Identify the Strategies and Other Factors That Influence the Outcomes and Determine Which Are Most Important

It is critical to identify, at a conceptual level, the various strategies and  factors that can help or hinder achievement of the strategic objective. As illustrated in figure 5, the logic model shows how the output of FNS’s  programs (left column) contribute to relevant near-term and long-term  outcomes (the three columns to the right). Identify Key Stakeholders to Participate in the Review

Because the achievement of outcomes may be complex and involve a  variety of contributors from within an agency, or include other federal  agencies, levels of government, and sectors, it is critical to consider which  key stakeholders should be involved in a strategic review. Although no one outside of HUD directly participated in the review, HUD  officials stated that they leveraged their existing relationship with officials  at the U.S. Interagency Council on Homelessness (Interagency Council)  and the 18 other federal agencies that comprise it to better understand  how other federal programs are contributing to progress in ending   This included attending and  homelessness for the target populations.participating in various meetings, including quarterly meetings with the  Interagency Council principals, staff-level coordinating meetings, and  targeted working groups, such as bimonthly meetings of the chronic and  family homelessness working group. For its objective to promote sustainable and livable communities, OSWER  officials developed what they called a “ladder of evidence”—a framework  and inventory of relevant performance information, scientific studies,  academic research, and program evaluations, which they then assessed  and categorized by strength. Assess Effectiveness in Achieving Strategic Objectives and Identify Actions Needed to Improve Implementation and Impact

Using relevant evidence, strategic review participants should assess  whether strategies are being implemented as planned and whether they  are having the desired effect, as well as whether other factors are  influencing results. DHS Is Developing New Performance Measures to Address Gaps in Data Identified during Strategic Review

Although CBP, TSA, ICE, and Coast Guard officials determined during  their review that sufficient progress was being made on DHS’s Goal 2.2 to  Safeguard and Expedite Lawful Trade and Travel, they also identified  gaps to address in performance monitoring. For example, agencies could use their existing quarterly  performance review processes to monitor progress on strategic review  action items, in line with the emphasis in OMB’s guidance for using  existing agency management processes for strategic reviews. HUD’s performance staff told us they work with responsible parties to  update the status of each action item and provide a report to the Deputy  Secretary regularly. Agency Comments

We provided a draft of the report to the Director of the Office of  Management and Budget, the Secretary of the Department of Agriculture,  the Secretary of the Department of Education, the Secretary of the  Department of Homeland Security, the Secretary of the Department of  Housing and Urban Development, the Administrator of the National  Aeronautics and Space Administration, and the Administrator of the  Environmental Protection Agency for comment. OMB staff and officials from the six agencies generally agreed with the  findings presented in this report. Appendix I: Objective, Scope, and Methodology

We are required to review implementation of the GPRA Modernization Act  of 2010 (GPRAMA) at several critical junctures. Our specific objective for this report was to  identify and illustrate, through case agency examples, practices that  facilitate effective strategic reviews by federal agencies. To identify practices, we analyzed and synthesized information gathered  from a literature review we conducted, which covered public  administration and public policy journals, business administration  journals, our body of work on performance management and program  evaluation, and other sources on policies and practices that can facilitate  or challenge the effectiveness of strategic reviews as a decision-making  tool. As a proxy, we used  relevant agency-level results on selected items from our 2013 survey of  federal managers on performance and management issues to  approximate if agencies had robust review processes and selected  agencies with varying levels of robustness. We also considered the extent to which agency strategic review  processes had a greater chance of addressing areas of fragmentation,  overlap, and duplication, and high-risk issues identified in our past work. We also  conducted interviews with officials involved in conducting strategic  reviews at the six selected agencies—which included agency  Performance Improvement Officers and their staff, strategic objective  leaders, and strategic review participants—and staff from OMB and the  PIC.
Why GAO Did This Study

The GPRA Modernization Act of 2010 (GPRAMA) provides important tools that can help inform federal decision making. In implementing GPRAMA, the Office of Management and Budget (OMB) established a strategic review process in which agencies, beginning in 2014, were to annually assess their progress in achieving each strategic objective—the outcome the agency is intending to achieve—in their strategic plans.
GPRAMA requires GAO to periodically review its implementation. This report identifies and illustrates practices that facilitate effective strategic reviews.
To identify such practices, GAO analyzed and synthesized information from a variety of sources, including GPRAMA's requirements; OMB guidance; a review of relevant literature; and interviews with experts in performance management and evaluation and OMB staff. To refine and illustrate the practices, GAO reviewed strategic review documentation and interviewed relevant officials from six selected agencies: USDA, Education, DHS, HUD, EPA, and NASA. GAO selected these agencies based on several factors. This included the extent to which agency strategic review processes had a greater chance of addressing areas identified in GAO's work on fragmentation, overlap, and duplication or high-risk issues, and agency results on selected items in GAO's 2013 survey of federal managers on performance and management issues.
In commenting on a draft of this report, OMB and the six selected agencies generally agreed with the findings.

What GAO Found

GAO identified seven practices federal agencies can employ to facilitate effective strategic reviews and illustrated aspects of those practices through examples from the strategic review processes conducted at the Departments of Agriculture (USDA), Education (Education), Homeland Security (DHS), and Housing and Urban Development (HUD), and the Environmental Protection Agency (EPA), and the National Aeronautics and Space Administration (NASA).
1. Establish a process for conducting strategic reviews. NASA developed a strategic review process that involved senior leaders in individual assessments and a rating of each strategic objective, a crosscutting review to identify themes and provide independent rating recommendations, and a briefing to the Chief Operating Officer to determine final ratings.
2. Clarify and clearly define measurable outcomes for each strategic objective. NASA officials defined what would constitute success in 10 years for each strategic objective and used underlying performance goals, indicators, and milestones to better plan for and understand near-term progress towards their long-term scientific outcomes.
3. Review the strategies and other factors that influence the outcomes and determine which are most important. USDA's Food and Nutrition Service developed a model showing how the output of its programs contribute to relevant near-term and long-term outcomes related to the department's objective to improve access to nutritious foods. The model also identifies external factors that could influence progress, such as food prices.
4. Identify and include key stakeholders in the review. Contributors from various agencies, levels of government, and sectors may be involved in achieving an outcome. While the six agencies involved internal stakeholders in their strategic reviews, GAO did not find instances of external stakeholder involvement. In some cases, agencies took steps to incorporate external perspectives, such as HUD leveraging its existing relationship with officials at the U.S. Interagency Council on Homelessness to better understand how other federal programs are contributing to progress towards its objective to end homelessness for target populations.
5. Identify and assess evidence related to strategic objective achievement. For EPA's objective to promote sustainable and livable communities, officials developed a framework and inventory of relevant performance information, scientific studies, academic research, and program evaluations, which they then assessed and categorized by strength.
6. Assess effectiveness in achieving strategic objectives and identify actions needed to improve implementation and impact. For DHS's goal to safeguard and expedite lawful trade and travel, officials determined that sufficient progress was being made, but identified gaps in monitoring efforts, such as a lack of performance measures related to travel. DHS officials are taking steps to develop measures to address the gaps.
7. Develop a process to monitor progress on needed actions. HUD broadened its existing process for tracking progress on actions items identified at its quarterly performance reviews to also cover those from strategic reviews. HUD staff update the status of each action item regularly—planned to be biweekly following the 2015 strategic reviews.