Responsibilities for Oversight of Imported Food Products

FDA’s responsibilities for overseeing the safety of imported food products  are divided among its product centers and program offices. PREDICT Uses Data and Analyzes the Data by Applying Rules That Contribute to Risk-Based Scores, but FDA Does Not Have a Documented Process for Obtaining Some Data

PREDICT uses a variety of data and FDA-created rules—conditional  statements that tell PREDICT how to react when encountering particular  information—to analyze these data and to identify high-risk imported food  shipments. Some of the domestic and foreign  sources are open sources—that is, publicly available. PREDICT Uses a Variety of Data, but FDA Does Not Have a Documented Process for Obtaining Open Source Data

PREDICT uses a variety of data to estimate the risk of imported food. Many of the data PREDICT uses come from sources within FDA. Data from Non-FDA Domestic Sources

According to FDA officials, PREDICT also uses data from sources outside  of FDA, including domestic sources, such as databases from states and  other federal agencies. Instead, FDA relies on ORA officials to informally communicate and  obtain information from officials in other FDA offices and from other  federal agencies on an ad hoc basis. The Implementation of FSMA Will Provide PREDICT with Additional Data on Imported Food

FSMA, which was enacted in 2011, contains provisions that will provide  PREDICT with additional data to analyze (i.e., from additional sources)  when estimating the risk of imported food. Of  these five, the first three are as follows:
Foreign Supplier Verification Program (FSVP): The FSVP rule  requires importers to verify that their foreign suppliers use processes  and procedures that provide the same level of public health protection  as the hazard analysis and risk-based preventive controls and other  applicable requirements of FDCA. Because some of these FSMA-related programs are still new and not yet  fully implemented, the details of how certain data generated by these  programs will be integrated into PREDICT are not finalized, but according  to FDA officials, PREDICT will use data produced by these programs and  authorities in a variety of ways. Implementation of FSVP, for example, will  identify importers that are not in compliance, and PREDICT would use  that information in assigning risk scores to products from those importers. FDA Assessed the Effectiveness of PREDICT in 2013, but Some Recommended Improvements Have Yet to Be Implemented

FDA has assessed the effectiveness of PREDICT by ongoing monitoring  of key program data and by conducting an evaluation of the tool in 2013,  and it has implemented many, but not all, of the 2013 evaluation’s  recommendations to improve PREDICT. Data provided by FDA from fiscal years 2012 to 2014 confirmed that  in general, PREDICT is fulfilling this role. FDA Conducted an Internal Evaluation of PREDICT and Developed 24 Recommendations to Improve the Tool

In May 2013, FDA completed an internal evaluation of PREDICT’s  effectiveness that examined five processes: work planning, examinations  and sampling, entry review, rules management, and communication. According to agency officials, the agency has  implemented 15, partially implemented another 6, and not implemented 3  (see fig. Federal standards for internal control state that agencies are to ensure  that the findings of audits and other reviews are promptly resolved. To  that end, agencies are to complete, within established time frames, all  actions that correct or otherwise resolve the matters brought to  management’s attention. Establishing a  practical timeline for implementing the remaining recommendations as  resources become available would provide FDA with reasonable  assurance that the improvements are made and that PREDICT remains  an effective tool for screening imports. FDA physically examines  about 1 percent of food shipment entry lines annually. Recommendations for Executive Action

To further enhance FDA’s PREDICT tool and its ability to ensure the  safety of imported food, we recommend that the Secretary of Health and  Human Services direct the Commissioner of FDA to take the following  two actions:    document the process for identifying the type of open source data  to collect, obtaining such data, and determining how PREDICT is  to use the data and    establish a timeline for implementing, as resources become  available, the remaining recommendations from FDA’s 2013  evaluation of PREDICT. Appendix I: Objectives, Scope, and Methodology

You asked us to examine how the Food and Drug Administration (FDA) is  using Predictive Risk-based Evaluation for Dynamic Import Compliance  Targeting (PREDICT) to protect the safety of the U.S. food supply. Our  objectives in this report were to examine (1) the data used by PREDICT  and how PREDICT analyzes these data to identify high-risk food  shipments for examination; (2) how implementation of the FDA Food  Safety Modernization Act (FSMA) will affect PREDICT; and (3) the extent  to which FDA has assessed the effectiveness of PREDICT and used the  results of those assessments to improve the tool. We also reviewed how the  agency identifies, obtains, and uses open source information and  compared this process to criteria for documentation found in the federal  standards for internal control.
Why GAO Did This Study

Imported food makes up a substantial and growing portion of the U.S. food supply. FDA is responsible for oversight of more than 80 percent of the U.S. food supply. However, because the volume of imported food is so high, FDA physically examines only about 1 percent of imported food annually. In 2011, FDA implemented PREDICT, a computerized tool intended to improve FDA's targeting of imports for examination by estimating the risk of imported products.
GAO was asked to review how FDA is using PREDICT to protect the U.S. food supply. This report examines (1) the data used by PREDICT and how PREDICT analyzes these data to identify high-risk food shipments for examination, (2) how implementation of FSMA will affect PREDICT, and (3) the extent to which FDA has assessed the effectiveness of PREDICT and used the assessments to improve the tool. To address these issues, GAO analyzed FDA documents, interviewed FDA officials, and analyzed data from fiscal years 2012 through 2014.

What GAO Found

The Food and Drug Administration's (FDA) Predictive Risk-based Evaluation for Dynamic Import Compliance Targeting (PREDICT) tool uses a variety of data and analyzes data by applying rules—conditional statements that tell PREDICT how to react when encountering particular information—to generate risk scores for imported food. Many of the data used by PREDICT come from internal FDA sources, such as FDA databases. PREDICT also uses data from sources outside of FDA, such as other federal agencies, states, and foreign governments. Some of the data are open source data—information that is publicly available, such as information from newspapers and websites. FDA's Office of Regulatory Affairs (ORA) relies on other FDA offices and federal agencies to provide open source data for PREDICT, but ORA does not have a documented process for identifying, obtaining, and using the data, relying instead on an ad hoc process. Federal standards for internal control call for agencies to document internal controls. Without a documented process for identifying, obtaining, and using open source data, FDA does not have reasonable assurance that ORA will consistently identify, obtain, and use such data for PREDICT.
The implementation of the FDA Food Safety Modernization Act (FSMA), enacted in 2011, will provide PREDICT with additional data for estimating the risk of imported food. FDA identified FSMA provisions likely to generate data, including the Foreign Supplier Verification Program (FSVP), which requires importers to verify that their foreign suppliers use processes and procedures that provide the same public health protection as applicable U.S. requirements. Because FSVP and other FSMA-related programs are still new and not yet fully implemented, the details of how PREDICT will use the data have not been worked out. However, according to FDA officials, the data will be useful. For example, FSVP data will identify suppliers that are not in compliance with standards, and PREDICT will use those data in assigning risk scores to imports from those suppliers.
FDA has assessed the effectiveness of PREDICT by monitoring of key data and by conducting an internal evaluation of the system, and it has implemented many, but not all, of its recommendations from the evaluation. GAO's analysis of FDA data from fiscal year 2012 through 2014 shows that in general, PREDICT is working as intended: imported food with higher-risk scores is more likely to be physically examined and to be found in violation of food safety standards or labeling requirements. In May 2013, FDA completed an evaluation of PREDICT that examined five key processes. As a result of the evaluation, FDA developed 24 recommendations for improving PREDICT and prioritized these recommendations based on feasibility and impact. According to FDA, the agency has fully implemented 15, partially implemented 6, and not implemented 3 of the recommendations. FDA officials said that the agency has not fully implemented all recommendations because of a lack of resources. However, federal standards for internal control specify that agencies are to ensure that the findings of reviews are promptly resolved. To that end, agencies are to complete, within established time frames, all actions that correct or otherwise resolve the matters brought to management's attention. Establishing a timeline for implementing the remaining recommendations as resources become available would help ensure that PREDICT continues to remain an effective tool for screening imported food.

What GAO Recommends

GAO recommends that FDA take two actions to improve the effectiveness of PREDICT: (1) document the process by which FDA is to identify, obtain, and use open source data and (2) establish a timeline for implementing the remaining recommendations from FDA's 2013 evaluation of PREDICT. FDA generally agreed with GAO's recommendations.