AOC’s Policies and Guidance Conform to Leading Practices for Two Characteristics of Reliable Cost Estimates, but Fall Short in Two Others

As previously discussed, the GAO Cost Estimating and Assessment  Guide defines 12 leading practices related to four characteristics— comprehensive, well documented, accurate, and credible—that are  important to developing high-quality, reliable estimates. We discuss characteristics using five rating categories— does not meet, minimally meets, partially meets, substantially meets, or  fully meets. AOC guidance is specific about sources of data. However, AOC’s guidance does  not require that cost estimates be updated with actual costs during a  project’s construction phase—a leading practice. Because of the weaknesses in AOC’s policies and guidance pertaining to  two of the four characteristics that are important to developing high-  quality, reliable estimates, the project cost estimates that AOC produces  may not always be reliable. Without reliable cost estimates, AOC’s  projects risk experiencing cost overruns or budget surpluses, missed  deadlines, and performance shortfalls. Furthermore, potential limitations  in the reliability of its estimates may impair Congress’s ability to make  well-informed funding decisions and affect how AOC allocates resources  across competing projects in its capital program. Our final  assessment found that  the Cannon Building renewal estimate to be substantially  comprehensive, well documented, and accurate while lacking in  elements affecting its credibility, and  the Capitol Dome restoration estimate to be substantially  comprehensive and well documented while lacking in areas pertaining  to accuracy and credibility. For the credibility characteristic, we found the estimate partially met  leading practices. Conclusions

AOC project’s cost-estimating guidance policy and two resulting project  estimates that we reviewed may not be fully reliable because they  incorporated some, but not all, leading practices in cost estimating. We recognize that incorporating GAO’s cost- estimating best practices into AOC’s cost-estimating policy and guidance  may involve additional costs—such as for conducting a risk and  uncertainty analysis for projects and conveying the confidence level of the  estimate to Congress and AOC managers. Recommendations

To improve the Architect of the Capitol’s project-cost-estimating process,  enhance the transparency of its related process, and allow for more  informed decision making related to projects’ costs, we recommend that  the Architect of the Capitol take the following two actions, to the extent  that the benefits exceed the costs:  incorporate leading practices we identified as lacking for cost  estimating in the AOC’s cost-estimating guidance and policies, and  for ongoing and future projects, submit the confidence level derived  from risk and uncertainty analyses along with budget documentation  to appropriate congressional decision makers, so that Congress is  aware of the range of likely costs and AOC’s associated confidence  levels. This report addresses the  extent to which AOC’s policies and guidance for developing cost  estimates conform to leading practices identified in GAO’s Cost  Estimating and Assessment Guide and provide a reliable basis to  support funding and capital program decisions. When the tasks  associated with the leading practices that define a characteristic are  mostly or completely satisfied, we consider the characteristic to be  “substantially” or “fully” met. In addition, our review of the Cannon Building estimate is  responsive to a mandate in the House Appropriations Committee report  accompanying the fiscal year 2010 Legislative Branch Appropriations Bill  (H.R. Appendix III: GAO’s Summary Assessments of AOC’s Project Cost Estimates for the Cannon Building’s Renewal and Capitol Dome’s Restoration

Appendix III: GAO’s Summary Assessments of  AOC’s Project Cost Estimates for the Cannon  Building’s Renewal and Capitol Dome’s  Restoration Estimating leading practice characteristic and rationale for assessment   Comprehensive  The cost estimate should include both government and contractor costs of the program over its full life cycle, from  inception of the program through design, development, deployment, operation and maintenance, to retirement of the  program. AOC  did not document variances between planned and actual costs. Modeling the aggregated risks precludes AOC  from identifying relationships among risk elements and determining  which risks have the greatest influence on project costs.
Why GAO Did This Study

AOC is responsible for the maintenance, renovation, and new construction of the U.S. Capitol complex, which comprises more than four dozen facilities. Reliable cost estimates for projects are crucial to AOC's capital-planning and construction processes. The House Appropriations Committee report accompanying the fiscal year 2014 Legislative Branch Appropriations bill mandated that GAO review AOC's cost-estimating methodology.
This report addresses the extent to which AOC's policies and guidance for developing cost estimates adheres to leading practices. GAO analyzed AOC's cost-estimating guidance, interviewed AOC officials, and compared AOC's cost-estimating guidance and documentation and two projects' cost estimates to leading practices in GAO's Cost Guide . When most or all of the practices associated with each characteristic of a high-quality, reliable estimate are followed, GAO considers the characteristic to be “fully” or “substantially” met. When, in turn, all four characteristics are at least “fully” or “substantially” met, GAO considers a cost estimate to be reliable.

What GAO Found

GAO's Cost Estimating and Assessment Guide ( Cost Guide ) defines 12 leading practices that are associated with four characteristics— c omprehensive, well documented, accurate , and credible —that are important to developing high-quality, reliable project-cost estimates. Using the Cost Guide , GAO determined that the Architect of the Capitol's (AOC) cost-estimating guidance conforms to leading practices for developing estimates that are, in general, comprehensive and well-documented . However, AOC's guidance does not substantially conform to leading practices related to developing cost estimates that are accurate and credible . For example, pertaining to the credible characteristic, AOC's guidance does not require determining the confidence level of estimates or quantifying the extent to which a project's costs could vary due to changes in key assumptions. GAO found the strengths and weaknesses of AOC's guidance generally reflected in the cost estimates for AOC's Cannon House Office Building's (Cannon Building) renewal project ($753 million) and Capitol Dome's restoration project ($125 million).
Cannon Building renewal—GAO found the estimate is substantially comprehensive, well documented, and accurate, but several factors that affect its credibility are lacking. For example, AOC's risk analysis does not allow for determination of which risks have the greatest influence on project costs and may overstate the effect of the risks.
Capitol Dome restoration—GAO found the estimate is substantially comprehensive and well documented, but lacking key analysis that support accurate and credible estimates. For example, AOC did not use actual costs from completed phases to update its estimates and did not complete a risk and uncertainty analysis.
Overall, AOC's cost-estimating guidance may not enable fully reliable estimates because it incorporates some, but not all, leading practices. Without reliable cost estimates that convey their confidence levels, AOC's projects risk experiencing cost overruns or budget surpluses, missed deadlines, and performance shortfalls. Potential limitations in the reliability of AOC's estimates may make it difficult for Congress to make well-informed funding decisions and affect how AOC allocates resources across competing projects in its capital portfolio.
Source: GAO analysis of AOC documents and data.
= Fully Meets =Substantially Meets =Partially Meets =Minimally Meets = Does Not Meet
Note: A characteristic is fully met when the associated tasks of underlying leading practices are completely satisfied; substantially met when a large portion of the associated tasks are satisfied; partially met when about half of the associated tasks are satisfied; minimally met when a small portion of the associated tasks are satisfied; and not met when none of the associated tasks are satisfied.

What GAO Recommends

GAO recommends that AOC incorporate additional leading practices into its cost-estimating guidance and submit the confidence levels of project estimates to Congress. AOC concurred with the recommendations and provided context and clarification on its cost-estimating guidance and policies.