PNNL manages the  deployment of radiation detection equipment for DHS. DHS’s  current plans call for completing the deployment of radiation portal  monitors at U.S. ports of entry by September 2013. The current technology  portal monitors, known as plastic scintillators or PVTs, cost about $55,000  per unit, while the advanced spectroscopic portal monitors, known as  ASPs, will cost around $377,000 per unit. DNDO Ignored Its Own Performance Test Results and Instead Relied on the Potential Performance of New Portal Monitors and Unreliable Estimates of Current Equipment Performance

DHS is developing new portal monitors, known as ASPs that, in addition to  detecting nuclear or radiological material, can also identify the type of  material. NIST compared the 10 ASP systems, and in June  2006 submitted a report to DHS on the results of that testing. Performance tests of ASPs showed that they did not meet DNDO’s main  performance assumption in the cost-benefit analysis of correctly  identifying HEU 95 percent of the time it passes through portal monitors. Consequently, DNDO officials  told us the analysis was based on the performance of PVT monitors that  PNNL tested during 2004 in New York. In our view, the results of such testing are  critical to any decision by DNDO to employ new technology, such as ASPs,  that might help protect the nation from nuclear smuggling. DNDO’s Cost-Benefit Analysis was Incomplete and Used Inflated Cost Estimates for PVT Equipment

DNDO officials told us they did not follow the DHS guidelines for  performing cost-benefit analyses in conducting their own cost-benefit  analysis. However, DNDO’s analysis omitted  many factors that could affect the cost of new radiation portal monitors. DNDO also underestimated the life-cycle costs for operations and  maintenance for both PVT and ASP equipment over time. In recent testimonies before Congress, DNDO’s Director has cited the  primary benefit of deploying ASP monitors as reducing unnecessary  secondary inspections. For example, it  did not estimate the costs of secondary inspections as they are carried out  today. DNDO’s analysis needs these baseline costs to compare alternatives  because without them, it is impossible to determine whether the use of  ASPs, as planned, will result in cost savings for secondary inspections. In addition, the ASP contract award has exceeded DNDO’s estimate for  total cost by about $200 million. The cost-benefit analysis shows the total  cost for deploying PVT and ASP monitors to be about $1 billion, which  covers all costs related to acquisition, design, maintenance, and physical  inspection over 5 years (for both PVT and ASP). However, in July 2006,  DHS announced that it had awarded contracts to develop and purchase up  to $1.2 billion worth of ASP portal monitors over 5 years. According to DNDO’s  Director, DNDO chose the highest published price because the current  contract for portal monitors at that time was to expire, and the portal  monitors will probably cost more in the future. However, the information  DNDO provided us does not explain why DNDO assumes that the future  price will be more than double what DHS was currently paying, as  assumed in DNDO’s analysis. In conclusion, DNDO’s approach to the cost-benefit analysis omitted many  factors that could affect the cost of new radiation portal monitors. For  these reasons, DHS’s cost-benefit analysis does not meet the intent of our  March 2006 report recommendation to fully assess the costs and benefits  before purchasing any new equipment.
Why GAO Did This Study

The Department of Homeland Security (DHS) is responsible for addressing the threat of nuclear smuggling. Radiation detection portal monitors are key elements in our national defenses against such threats. DHS has sponsored R&D and testing activities to develop a "next generation" portal monitor, known as the advanced spectroscopic portal monitor. However, each one costs 6 times more than a current portal monitor. In March 2006, we recommended that DHS conduct a cost-benefit analysis to determine whether the new portal monitors are worth the additional cost. In June 2006, DHS issued its analysis. In October 2006, we issued our report that assessed the DHS study. GAO's statement, based on our October 2006 report, addresses whether DHS's cost-benefit analysis provides an adequate basis for its decision to purchase and deploy the next generation portal monitors.

What GAO Found

DHS's cost-benefit analysis does not provide a sound analytical basis for its decision to purchase and deploy the new portal monitor technology. Our review of the analysis determined that it had the following problems. Regarding the performance of the portal monitors: instead of using the results of its own portal monitor tests conducted in 2005, DHS assumed that the new portal monitor technology would correctly detect and identify highly enriched uranium (HEU) 95 percent of the time--a performance level that far exceeds the new technology's current capabilities. To determine the performance of the current generation of portal monitors in detecting HEU, DHS used data from limited tests carried out in 2004 that test officials concluded were unreliable for such purposes. DHS's analysis of the new technology portal monitors was incomplete because the analysis focused on identifying HEU, but did not fully consider how well the new portal monitor technology could correctly detect or identify other dangerous radiological or nuclear materials. Regarding cost estimates: in comparing the costs of the new and current technologies, the procurement costs of the current generation portal monitors were highly inflated because DHS assumed a unit cost of about $131,000. However, the contract price at the time of the analysis was about $55,000. According to officials who manage the contract, it was to expire and they expected portal monitor prices to increase, but not nearly as much as DHS assumed. DHS stated that the primary benefit of deploying the new portal monitors is reducing unnecessary secondary inspections. However, DHS's analysis does not fully estimate today's baseline costs for secondary inspections, which makes it impossible to determine whether the use of the new portal monitors as currently planned, will result in significant cost savings for these inspections. The new portal monitor contract price has exceeded DHS's total cost estimate by about $200 million. The cost-benefit analysis shows the total cost for deploying both current and new portal monitors to be about $1 billion. However, in July 2006, DHS announced that it had awarded contracts to develop and purchase up to $1.2 billion worth of the new portal monitors over 5 years. DHS's cost-benefit analysis omitted many factors that could affect the cost of new portal monitors, such as understating the life-cycle costs for operating and maintaining the equipment over time. For these reasons, DHS's cost-benefit analysis does not meet the intent of our March 2006 report recommendation to fully assess the costs and benefits before purchasing any new equipment.