In February  2011, the Corps submitted its initial report to Congress summarizing its  experiences implementing the peer review process in response to the  requirement in section 2034 of WRDA 2007. It Is Unclear How Many Peer Reviews Have Been Completed in Response to Section 2034 Requirements

Since enactment of WRDA 2007, 49 Corps civil works project studies  have undergone peer review as of January 2012, but it is unclear how  many of these reviews were performed in response to the requirements in  section 2034. This is because the Corps does not make specific  determinations or track whether a peer review is being conducted in  response to the requirements of section 2034. Nevertheless, because the Corps did not distinguish which studies had  been selected for peer review in accordance with section 2034, we  believe that it did not provide Congress with the type of information  required by section 2034 that would help congressional decision makers  evaluate the trial program. Completed Peer Reviews Have Cost Millions of Dollars in Direct and Indirect Costs

The 49 peer reviews conducted by the Corps since November 2007  resulted in direct costs of about $9 million in contract costs and contract  administration fees. Furthermore, the addition of peer review to the Corps study process has  resulted in indirect costs by altering project study schedules because of  the additional time required to complete the peer review. In some cases  where a peer review was not planned for during the early stages of the  study process, significant delays in the project studies have resulted from  the addition of the peer review. In addition, Corps staff resources were also used to manage the  peer reviews, but these costs are not fully quantifiable. In contrast, according to some Corps division and planning- center-of-expertise officials, when the project manager had built in time  for the peer review and had identified funding for it early, the peer review  process had much less of an impact on the overall project study  schedule. The Corps’ Process for Initiating Peer Review Is More Expansive and Less Flexible Than Section 2034 Requirements

The Corps’ process for determining whether a project study is subject to  peer review is more expansive than section 2034 requirements because it  uses broader criteria; this has resulted in peer reviews of studies that are  outside the scope of section 2034. In addition, the process the Corps  uses does not include the flexibility provided in section 2034 to exclude  certain project studies from peer review. Moreover, some studies are  undergoing peer review that do not warrant it, according to some Corps  officials we spoke with. Gaps Exist in the Corps’ Process for Screening Its Contractors Who Are Responsible for Selecting Experts for Peer Review Panels

The Corps has a process to review general information on contractors’  conflicts of interest and independence during its contractor selection  process, but it does not have a process for reviewing project-specific  information provided by contractors to determine if conflicts of interest  and independence exist at the project level. The Corps’ contractors,  however, have a process for reviewing the appropriate information related  to the conflicts of interest and independence of the experts selected for  peer review panels at the project level. The Corps Has Adopted Most Peer Review Recommendations, Resulting in Technical Improvements but Generally No Changes in Project Decisions

The Corps has adopted and incorporated most of the peer review  recommendations it has received. Adoption of these recommendations  has resulted in some technical improvements to project study reports but  generally has not changed the Corps’ decisions in selecting preferred  project designs. According to some Corps officials we spoke with, this is  the result of the review occurring too late in the process to effect a  change in decision making. As a result, some recommendations about alternatives may not have  been implemented because the decision on the preferred design had  already been made. Appendix I: Objectives, Scope, and Methodology

Our objectives for this work were to examine (1) the number of Corps  project studies that have undergone independent peer review in response  to section 2034 of the Water Resources and Development Act (WRDA) of  2007, (2) the cost of these peer reviews, (3) the extent to which the U.S.  Army Corps of Engineers’ (Corps) process for determining if a project  study is subject to peer review is consistent with section 2034, (4) the  process the Corps uses to ensure that the contractors it hires and the  experts the contractors select to review project studies are independent  and free from conflicts of interest, and (5) the extent to which  recommendations from peer reviews have been incorporated into project  studies.
Why GAO Did This Study

Section 2034 of the Water Resources Development Act of 2007 requires that certain U.S. Army Corps of Engineers (Corps) civil works project studies undergo independent external peer review to assess the adequacy and acceptability of the methods, models, and analyses used. In the act, Congress established a 7-year trial period for this requirement and also required the Corps to submit two reports on its experiences with the peer review process.
GAO was asked to examine (1) the number of Corps project studies that have undergone independent peer review in response to section 2034, (2) the cost of these peer reviews, (3) the extent to which the Corps process for determining if a project study is subject to peer review is consistent with section 2034, (4) the process the Corps uses to ensure that the contractors it hires and the experts the contractors select to review project studies are independent and free from conflicts of interest, and (5) the extent to which peer review recommendations have been incorporated into project studies. GAO reviewed relevant laws, agency guidance, and documents and interviewed Corps officials and contractors.

What GAO Found

Since enactment of the Water Resources Development Act of 2007, 49 project studies have undergone peer review but it is unclear how many were performed in response to section 2034 requirements because the Army Corps of Engineers (Corps) does not make specific determinations or track if a peer review is being conducted under section 2034. In February 2011, in response to section 2034, the Corps submitted its initial report to Congress summarizing its implementation of the peer review process. In its report, however, the Corps did not distinguish which studies had been selected for peer review in accordance with section 2034 and therefore, did not provide Congress information that would help decision makers evaluate the requirements of section 2034 at the end of the trial period.
The 49 peer reviews resulted in both direct and indirect costs. Specifically, these peer reviews resulted in direct costs of over $9 million in contract costs and fees. In addition, Corps staff resources were used to manage the reviews, although these costs are not fully quantifiable. Furthermore, the addition of peer review to the Corps study process has resulted in indirect costs by altering project study schedules to allow for time needed to complete peer reviews. In some cases where a peer review was not planned during the early stages of the study process, significant delays to project studies occurred while funds were sought to pay for the peer review. In contrast, according to some Corps officials, when project managers have built in time and identified funding for peer reviews early, the process has had less of an impact on project study schedules.
The Corps process for determining whether a project study is subject to peer review is more expansive than section 2034 requirements because it uses broader criteria, resulting in peer reviews of studies outside the scope of section 2034. In addition, the process the Corps uses does not include the flexibility provided in section 2034, which allows for the exclusion of certain project studies from peer review. Moreover, some studies are undergoing peer reviews that do not warrant it, according to some Corps officials GAO spoke with.
The Corps has a process to review general information on contractors conflicts of interest and independence when selecting them to establish peer review panels, but it does not have a process for reviewing project-level information on conflicts of interest and independence. As a result, it cannot be assured that contractors do not have conflicts at the project-level. In contrast, the Corps contractors do have a process for reviewing information related to conflicts of interest and the independence of experts selected for each peer review panel.
The Corps has adopted and incorporated into its project study reports most of the peer review recommendations it has received. Doing so has resulted in some technical improvements to study reports but generally has not changed the Corps decisions about project alternatives, in part because the peer review process occurs too late in the project study process to affect decision making, according to some Corps officials GAO spoke with. As a result, some recommendations about alternatives may not have been implemented because the decision on the preferred design had already been made.

What GAO Recommends

GAO recommends that the Department of Defense direct the Corps to, among other actions, better track peer review studies, revise the criteria for determining which studies undergo peer review and the timing of these reviews, and improve its process for ensuring contractor independence. The department generally concurred with these recommendations.