Data Challenges Have Limited the Implementation of the Benchmarks, and SBA and Participating Agencies Have Provided Inconsistent Information about the Consequence SBA and Participating Agencies Assessed Small Businesses against the Transition Rate Benchmark, but the Assessments Have Been Based on Inaccurate or Incomplete Data

Data challenges have limited SBA’s and the 11 participating agencies’  efforts to fully implement the benchmarks. Since 2014, SBA and the participating agencies have regularly assessed  small businesses against the Transition Rate Benchmark, which, in  general, measures the rate at which businesses move projects from  phase I to phase II. From 2014 through 2017, SBA determined that 4 to  7 small businesses did not meet the benchmark each year and placed  those businesses on a list of those ineligible to receive certain additional  awards. For example, based on our review of the award data from 2007  through 2016, we identified more than 2,700 small businesses that had  multiple records with different spellings of the same business’s name. Furthermore, we identified more than 1,400 instances in which a unique  identification number had errors, such as having an incorrect number of  digits, all zeros, or hyphens. As a result, it could be difficult to determine which small  businesses actually received more than 20 awards and should be subject  to the benchmark. SBA officials  provided us guidance on how to enter data that they said is available to  agencies, but the errors we found suggest that agencies are not fully  utilizing this guidance. SBA and participating agencies do not  know the extent to which small businesses are meeting the  Commercialization Benchmark because SBA and the agencies have  assessed businesses against the benchmark only once, in 2014, when  SBA determined that 12 businesses did not meet the benchmark. This is  in part because, according to officials from SBA and several agencies,  they cannot collect and verify the accuracy of the data needed to  implement the benchmark as written. For example, agency  officials told us that the needed data are not consistently applicable  across agencies or projects. Specifically, these officials said that an  agency may purchase the technology developed as a result of the SBIR  or STTR award, while another agency may focus on funding technologies  that will be sold on the commercial market, leading to different kinds of  data on “sales.” Additionally, officials from SBA and several of the  participating agencies told us they have been unable to collect and verify  the accuracy of the information from small businesses to assess them  against the Commercialization Benchmark. The Small Business Act and the policy directives provide agencies  flexibility in how they can implement the Commercialization Benchmark. SBA and Participating Agencies Have Provided Inconsistent Information to Small Businesses on the Consequence of Not Meeting the Benchmarks

The consequence for small businesses not meeting the benchmarks is  ineligibility to participate in phase I of the SBIR or STTR program for a  year, according to the Small Business Act. SBA and the participating agencies agreed to change how the  consequence would be implemented, starting in 2017, so that small  businesses that do not meet the benchmarks are ineligible to submit  proposals, according to SBA officials. As of November 2017, the SBIR and STTR policy directives stated that  the consequence for not meeting these benchmarks is ineligibility to  receive certain awards. Until participating agencies update their project solicitations and SBA  updates its policy directives to accurately reflect agreed-upon practices  about the consequence for small businesses that do not meet the  benchmarks, small businesses may be confused about their eligibility to  submit proposals and could invest time developing and submitting  proposals when they are not eligible to do so. Conclusions

Under the SBIR and STTR programs, federal agencies have awarded  billions of dollars to small businesses to help these businesses develop  and commercialize innovative technologies. Working together to implement the benchmark as written or  revise it so that it can be implemented could allow the agencies to fulfill  the requirement in the Small Business Act to evaluate whether small  businesses have met a minimum performance standard for  commercializing their technology. Specifically:  The Director of the Office of Investment and Innovation within SBA should  work with participating agencies to improve the reliability of its SBIR and  STTR award data (Recommendation 1).
Why GAO Did This Study

Through the SBIR and STTR programs, federal agencies have awarded about 162,000 contracts and grants totaling $46 billion to small businesses to help them develop and commercialize new technologies. Eleven federal agencies participate in the SBIR program, and 5 agencies also participate in the STTR program. Each program has three phases, which take projects from initial feasibility studies through commercialization activities. SBA oversees both programs.
In response to the 2011 reauthorization of the programs, SBA and the participating agencies developed benchmarks to measure small businesses' progress in developing and commercializing technologies. GAO was asked to review SBA's and the agencies' efforts related to these benchmarks. This report examines the extent to which SBA and the participating agencies have implemented these benchmarks, including assessing businesses against them and establishing the consequence of not meeting them. GAO analyzed award data and interviewed officials from SBA and the 11 participating agencies.

What GAO Found

Data challenges have limited the Small Business Administration's (SBA) and the 11 participating federal agencies' efforts to assess businesses against two benchmarks—the Transition Rate Benchmark and the Commercialization Benchmark—of the Small Business Innovation Research (SBIR) and Small Business Technology Transfer (STTR) programs.
Transition Rate Benchmark. Small businesses that received more than 20 awards for the first phase of the programs in the past 5 fiscal years—excluding the most recent fiscal year—must have received an average of 1 award for the second phase of the programs for every 4 first phase awards. Since 2014, SBA and the agencies participating in the programs have regularly assessed small businesses against this benchmark. From 2014 through 2017, SBA determined that 4 to 7 businesses did not meet the benchmark each year. SBA officials provided GAO guidance on how to enter data into the programs' awards database they said is available to agencies, but GAO found evidence that suggests agencies are not fully utilizing it. For example, GAO found that the database used to perform the assessments contained inaccurate and incomplete data, such as about 2,700 businesses with multiple records with different spellings of their names and more than 1,400 instances in which a unique identification number had errors, such as an incorrect number of digits, all zeros, or hyphens. Thus, it could be difficult to determine which small businesses should be subject to the benchmark.
Commercialization Benchmark. Small businesses that received more than 15 awards for the second phase of the programs in the past 10 fiscal years—excluding the most recent 2 fiscal years—must have received a certain amount of sales, investments, or patents resulting from their efforts. SBA and participating agencies have assessed small businesses against this benchmark only once, in 2014, and identified 12 businesses that did not meet the benchmark. This is, in part, due to challenges in collecting and verifying the accuracy of the data that small businesses report and that are needed to implement the benchmark, according to officials from SBA and several agencies. For example, agency officials told GAO that some needed data, such as for reported sales, are not consistently applicable across agencies or projects. The Small Business Act and policy directives provide flexibility in how the agencies can implement the benchmark. Working together to implement it as designed or revise it so that it can be implemented could allow the agencies to fulfill statutory requirements.
SBA and the participating agencies have provided inconsistent information to small businesses about the consequence of not meeting the benchmarks. SBA and the agencies agreed to change how the consequence of not meeting the benchmarks was to be implemented, starting in 2017, from ineligibility to receive certain awards to ineligibility to submit certain proposals. However, as of November 2017, some agencies had not updated this information in their project solicitations. Furthermore, SBA has not updated this information in its policy directives. Without consistent information, businesses may be confused about their eligibility to submit proposals or receive awards and could invest time developing and submitting proposals when they are not eligible to do so.

What GAO Recommends

GAO is making 11 recommendations to SBA and other agencies to take actions to improve implementation of the benchmarks, including improving the reliability of award data; implementing or revising the Commercialization Benchmark; and updating information about the consequence of not meeting the benchmarks. SBA and these agencies agreed with GAO's recommendations.