Among other things, the study recommended that SBA and participating  agencies improve the collection of data that track participation in the SBIR  program by businesses owned by disadvantaged individuals and women,  develop targeted outreach to such businesses that is based on an  analysis of factors that affect their participation, and improve  documentation of commercialization success. Agencies Addressed SBIR’s Purposes through Solicitations, Technical Assistance, Matching Funds Programs, and Outreach

For fiscal years 2008 through 2011, the five participating agencies we  reviewed addressed the SBIR purposes of using small business to meet  federal R&D needs and stimulating technological innovation through their  solicitations. To address the remaining program purpose—encouraging  participation in technological innovation by small businesses owned by  disadvantaged individuals and women—agencies relied mainly on  outreach activities aimed at a broader audience. To provide the assistance, the agencies  contracted with vendors and consultants who have experience in bringing  technologies to market. With the exception of NASA, the agencies  supported the technical assistance at least in part through the use of  SBIR funds. Through matching funds programs, agencies  provide additional SBIR funds to award recipients that obtain monetary  commitments above certain thresholds from outside investors. Evaluation of the effectiveness of agencies’ outreach efforts is hindered  by a lack of accurate and complete data. Comparable Data Are Not Available across Participating Agencies to Evaluate Progress in Increasing Commercialization of SBIR Technologies

SBA has not yet developed the government-use portion of its database  for collecting comparable commercialization data on SBIR technologies,  but it is taking steps to do so. In the interim, agencies have, for their own  purposes, independently gathered commercialization data that are not  comparable; the accuracy of these data is largely unknown. SBA officials told us  that they have worked with participating agencies to develop common  metrics for commercialization data, as well as a standardized data  collection instrument that will accommodate the various types of SBIR  technologies the agencies fund to meet their different missions. Other agencies, such as DOD, may continue to require  applicants and recipients to submit commercialization data directly to the  agencies, which would then upload the data into the database. While  SBA has worked with SBIR agencies to identify best practices in other  areas of SBIR program management, it has not identified best practices  for agencies to use in verifying the accuracy of commercialization data. Conclusions

DOD, DOE, NASA, NIH, and NSF have designed their SBIR solicitations  to address the program’s purposes of using small businesses to meet  federal R&D needs, stimulating technological innovation, and increasing  commercialization of innovations derived from federal R&D efforts, and  they have further addressed commercialization by providing technical  assistance or matching funds to award recipients. DOD has adopted practices  for verifying the accuracy of commercialization data it collects from prior  award recipients, but most of the participating agencies we reviewed did  not verify the accuracy of commercialization data from their prior award  recipients, and SBA has not identified best practices for participating  agencies to use in doing so. Recommendations for Executive Action

To build upon efforts to implement a government-use database for  program evaluation, we recommend that the Administrator of the Small  Business Administration work with participating SBIR agencies to take the  following two actions:  collect data on the number of applications submitted by small  businesses owned by disadvantaged individuals and women, and  identify best practices for verifying the accuracy of data related to  progress in increasing commercialization. Key contributors to this report are listed in  appendix V.

Appendix I: Scope and Methodology

In conducting this study, we reviewed Small Business Innovation  Research (SBIR) program-related activities of the Small Business  Administration (SBA) and 5 of the 11 SBIR participating agencies—the  Department of Defense (DOD), Department of Energy, National  Aeronautics and Space Administration, Department of Health and Human  Services’ National Institutes of Health (NIH), and National Science  Foundation (NSF).
Why GAO Did This Study

Federal agencies with a budget of at least $100 million for research and development (R&D) conducted by others must participate in the Small Business Innovation Research (SBIR) program. SBIR has four purposes: meet federal R&D needs; stimulate technological innovation; increase commercialization (e.g., sales) of innovations based on federal R&D; and encourage participation in innovation by small businesses owned by disadvantaged individuals and women. The Small Business Administration (SBA) oversees the efforts of participating agencies, which make awards to small businesses using SBIR funds. Congress directed SBA to develop a database with commercialization data for government use in evaluating the program. GAO was asked to determine (1) how agencies have addressed SBIR's purposes and (2) the extent of data available to evaluate progress in increasing commercialization. GAO analyzed program documents and interviewed officials at SBA and five agencies that accounted for about 96 percent of SBIR awards.

What GAO Found

For fiscal years 2008 through 2011, the participating agencies GAO reviewed--the Department of Defense (DOD), the Department of Energy (DOE), the National Aeronautics and Space Administration, the Department of Health and Human Services' National Institutes of Health, and the National Science Foundation (NSF)--addressed SBIR's purposes through solicitations for award applications, technical assistance or matching funds programs, and outreach. In particular, the agencies addressed SBIR purposes related to meeting federal R&D needs and stimulating technological innovation through their solicitations, which included research topics that were designed to meet agencies' respective R&D or mission needs. Agencies also addressed commercialization of innovations through solicitations, as well as through technical assistance for award recipients or through matching funds programs. To provide technical assistance, the agencies contracted with vendors and consultants for help in developing business plans and identifying potential customers for SBIR award recipients, among other things. Agency matching funds programs provided additional SBIR funds to award recipients that obtained commitments from outside investors. Agencies generally addressed the remaining SBIR purpose, encouraging participation by small businesses owned by disadvantaged individuals and women, through outreach activities aimed at a broader audience, such as sharing information on Web sites. However, the effectiveness of these efforts is difficult to evaluate, in part because SBA does not collect data on the number of SBIR applications submitted by such businesses, thus hindering analysis of trends in their submission of applications. Comparable data are not available across participating agencies to evaluate progress in increasing commercialization of SBIR technologies. SBA has not yet expanded an existing database to include commercialization data for program evaluation, but the agency has hired a contractor and allocated funds to develop the expanded database by August 2011. SBA has also worked with participating agencies to develop common metrics for commercialization. In the absence of the expanded database, agencies have independently gathered commercialization data for their own use that are not comparable. In collecting these data, agencies differed in the types of data collection instruments used, dates the instruments were administered, award recipient populations queried, and types of data requested. Furthermore, with the exception of DOD, agencies that GAO reviewed did not generally take steps to verify commercialization data they collected from award recipients, so the accuracy of the data is largely unknown. SBA has worked with SBIR agencies to identify best practices in other areas of program management but has not identified best practices for agencies to use in verifying the accuracy of commercialization data. Implementing the expanded database should improve the comparability of commercialization data available, but a lack of consistent practices for verifying the accuracy of these data may limit their usefulness for programwide evaluation.

What GAO Recommends

GAO recommends that SBA work with participating agencies to (1) collect data on applications from small businesses owned by disadvantaged individuals and women and (2) identify best practices for verification of commercialization data. SBA, DOE, and NSF generally agreed with these recommendations; the other agencies GAO reviewed neither agreed nor disagreed.