Reliability of NMFS’ Pacific Groundfish Stock Assessments Is Questionable

The reliability of NMFS’ stock assessments is questionable for the Pacific  hake and four rockfish species we reviewed, although they were based on  the best information available at the time the assessments were conducted. The reliability of the stock assessments we reviewed is questionable  because (1) four of the assessments did not have at least one  NMFS-collected data source of sufficient scope and accuracy; (2) NMFS  lacked a standard process for assessing the reliability of non-NMFS data  used in all five assessments; and (3) for four of the assessments, the stock  assessment reports did not adequately identify the uncertainty of the  biomass estimates. The 2003 NMFS report also found that darkblotched groundfish are  less abundant in untrawlable waters, while canary and bocaccio species  are more abundant in untrawlable waters. Lacking a standard process, some  assessors reviewed the quality of the raw non-NMFS data, while others did  not. Assessors who reviewed for data quality found mistakes that they  believed made some of the data unusable or that could have impaired the  accuracy of the stock assessments. In a review of  the 2002 canary assessment, the stock assessment review panel  recommended that standard estimates of uncertainty be included in future  assessments because it is difficult to determine the reliability of the stock  assessment without them. The Northwest Center has  concentrated most of its efforts on implementing recommendations aimed  at obtaining more data. Recommendations aimed at increasing the types of  data and improving their quality have not yet been fully implemented for a  variety of reasons, such as staffing and funding limitations. Extended the geographic range of the groundfish shelf and slope bottom  trawl survey. The surveys are now coastwide from Cape Flattery,  Washington to the Mexican border, adding over 300 more miles along  the southern California coast. Finally, the improvement plan recommended that NMFS prepare a  comprehensive plan that combines the improvement plan with other  complementary plans, such as the NOAA Fisheries Data Acquisition Plan  and the NMFS Social Sciences Plan. However, the actual cost of  implementing remaining improvements to Pacific groundfish stock  assessments may be even higher because the Northwest Center’s budget  requests primarily reflect the amount of money the Center believed it could  realistically obtain, rather than the actual cost of the improvements. Remaining Improvements Are Estimated to Cost at Least $8.9 Million, but Estimate Is Likely Understated

According to NMFS, the Northwest Center needs at least $8.9 million to  complete ongoing and planned improvements for Pacific groundfish stock  assessments: $2.6 million that NMFS’ Northwest Center requested but did  not receive between fiscal years 2001 to 2003 and $6.3 million the Center  requested for fiscal years 2004 and 2005. Specifically, as shown in table 2,  the Northwest Center records have identified the following funding needs   $7.7 million to improve the types of data used, including $2.4 million for  surveys of untrawlable waters, $2.1 million to expand acoustic and  recruitment surveys, and $3.2 million to collect ecosystem data; and  $1.2 million to improve the quality of data used in stock assessments,  including $600,000 to enhance the calibration of vessel equipment;  $525,000 to develop and implement methods to collect information on  stock identification, structure, and movement; and $75,000 to  standardize trawl survey procedures. Objectives, Scope, and Methodology

We reviewed National Marine Fisheries Service (NMFS) stock assessments  for five species of Pacific groundfish: Pacific hake (Pacific whiting) as well  as four types of rockfish—bocaccio, canary, darkblotched, and yelloweye. Specifically, for these five species you asked us to (1) assess the reliability  of NMFS’ stock assessments, (2) identify which relevant recommendations  from the stock assessment improvement plan have been implemented and  which have not, and (3) identify the estimated costs associated with  planned and ongoing improvements to groundfish stock assessments. We revised the report to clearly show that NMFS has not collected  enough ecosystem data and that the frequency and range of  recruitment surveys are limited. The NMFS data used in the bocaccio, canary, and darkblotched  assessments were limited because NMFS conducted its surveys in  trawlable waters only. NMFS data were unavailable for  the yelloweye assessment.
Why GAO Did This Study

Because of concerns raised about the accuracy of National Marine Fisheries Service (NMFS) stock assessments, GAO reviewed the assessments for five species of Pacific groundfish: Pacific hake and four types of rockfish--bocaccio, canary, darkblotched, and yelloweye. Specifically, for these five species GAO (1) assessed the reliability of NMFS' stock assessments, (2) identified which relevant recommendations from NMFS' stock assessment improvement plan have been implemented and which have not, and (3) identified the costs associated with planned and ongoing improvements to groundfish stock assessments.

What GAO Found

The reliability of the NMFS assessments is questionable for the five species GAO reviewed, although the assessments were based on the best information available at the time they were conducted. According to NMFS officials and a National Research Council report, to obtain reliable results each stock assessment should include at least one NMFS data source of sufficient scope and accuracy because such data are derived from unbiased, statistical designs. However, in the yelloweye assessment, no NMFS data were used, and in the darkblotched, canary, and bocaccio assessments, the NMFS data were limited because the NMFS' surveys were conducted in trawlable waters only. A 2003 NMFS report concluded that darkblotched groundfish are less abundant and bocaccio and canary are more abundant in untrawlable waters. Also for all five species, NMFS lacks a standard approach for ensuring the reliability of non-NMFS data used in stock assessments. Some assessors reviewed the quality of non-NMFS data; others did not. The assessors who reviewed the quality of the non-NMFS data found errors that made some of the data unusable or that could have impaired the reliability of certain stock assessments. Finally, for four species, the stock assessment reports were questionable because they did not present the uncertainty associated with the population estimates. For example, the canary stock assessment review panel recommended that standard estimates of uncertainty be included in the assessment report because without them it is difficult to determine their reliability. NMFS has taken steps to implement some of the recommendations contained in the NMFS stock assessment improvement plan, but much remains to be done. NMFS has concentrated its efforts mostly on improving data quantity. For example, NMFS increased the frequency of groundfish stock assessments and extended the geographic ranges of the shelf and slope surveys to cover over 300 more miles along the southern California coast. However, because of staffing and funding limitations, NMFS has not yet implemented many of the recommendations aimed at obtaining more types of data and improving data quality. For example, NMFS has not collected enough ecosystem data, and the frequency and range of recruitment surveys (estimated production of new members of a fish population) are limited. Finally, because of other program priorities, NMFS has not implemented the recommendation to create a comprehensive plan that combines the improvement plan and its complementary plans. NMFS records indicate at least $8.9 million is needed to complete ongoing and planned stock assessment improvements--$2.6 million that NMFS' Northwest Fisheries Science Center requested but did not receive in fiscal years 2001 to 2003, and $6.3 million requested for fiscal years 2004 and 2005. It will cost about (1) $7.7 million to improve the types of data used, such as more untrawlable water and recruitment surveys and (2) $1.2 million to improve the quality of data used in stock assessments, such as enhanced calibration of vessel equipment and standardized trawl survey procedures. The actual cost of the remaining improvements may be even higher than the $8.9 million estimated because the estimates primarily reflect the amount of money that agency officials believed could be realistically obtained, rather than what the improvements might cost.