Agencies Provide Most of the Required SBIR-Award Data Elements to SBA, but Some Data Submitted are Incomplete

While federal agencies participating in the SBIR program submit a wide  range of descriptive award information to SBA annually, these agencies  are not consistently providing all of the required information. Specifically, participating  agencies are required to provide over 40 data elements for each SBIR  award they make. As a result, SBA does not have information on the  number of employees at SBIR awardee firms for about one-third of all the  awards reported by these agencies in 2004 and 2005. Agencies Generally Comply with SBA’s Formatting Guidance, but Key Data on SBIR Award Recipients May Be Inconsistent or Inaccurate

Participating agencies are providing some data that do not comply with  SBA’s formatting guidance, and while some of these inconsistencies are  corrected by SBA’s quality assurance processes, others are not. Because the data contained in the public-use section of Tech-Net will be  incorporated into the government-use section of the database, inaccurate  data in one section of the database will be replicated in the other, and  these inaccurate data will limit evaluations of the SBIR program. Instead, SBA relies on the agencies to  fully report all the required application information on the awards they  make. The formatting inconsistencies that we identified arise primarily because  the SBA reporting template used by agencies to submit required data can  be edited. SBA Is Five Years Behind Schedule in Meeting its Obligation to Implement a Government-Use SBIR Database

SBA has not met its obligation to implement a restricted government-use  database that would allow SBIR program evaluation as directed by the  2000 SBIR reauthorization act. SBA planned to meet this  requirement by expanding the existing Tech-Net database to include a  restricted government-use section that would be accessible only to  government agencies and other authorized users. However, according to SBA officials, the agency has been unable to meet  this requirement, primarily because of increased security and other  information technology project requirements, agency management  changes, and budgetary constraints. While SBA  officials expect the government-use section of the database to be  operational by October 1, 2006, they also recognize that additional  enhancements, such as improving the user-friendliness of the interface for  online submission of SBIR data by participating agencies and recipients,  will be needed after the system is made operational. Regardless of how they track commercialization success, both SBA and  agency officials generally agree that several factors complicate their  efforts to obtain this information. Agencies’ Efforts to Track SBIR Commercialization Success Vary

Of the eight agencies we reviewed, five systematically and periodically  survey SBIR recipients to gather a variety of data on program  participation, including the recipients’ commercialization experiences. Appendix I: Objectives, Scope and Methods

Our objectives for this review were to identify (1) the types of data that  participating Small Business Innovation Research (SBIR) program  agencies are reporting to the Small Business Administration (SBA) for  inclusion in the Tech-Net database, (2) the extent to which agencies  provide data for the Tech-Net database in a standard format to enable  program evaluation, (3) the extent to which SBA has met the mandate to  establish by early 2001 a government-use database that can be used for  program evaluation, and (4) the extent to which participating SBIR  agencies have developed and implemented techniques to track the  commercialization success of SBIR projects. To determine the extent to which data for the Tech-Net database are  provided in a standard format, enabling program evaluation, we compared  data provided to SBA by the eight participating agencies with data in  SBA’s Tech-Net database for fiscal years 2004 and 2005, the 2 most recent  years for which data were available. The agency data included information  about the award itself, such as the award amount, a descriptive abstract of  the project, and a unique tracking number; information about the award  recipient, such as gender and socio-economic status; and information  about the type of firm that received the award, such as number of  employees and geographic location. 5.
Why GAO Did This Study

The Small Business Innovation Research (SBIR) program was created to increase the use of small businesses to meet federal research needs and commercialize the results of this research. To monitor the program, the Small Business Administration (SBA) requires participating agencies to provide, in a standard format, specific data on all SBIR awards they make. SBA then compiles these data into a database known as Tech-Net. Congress also required SBA to create, by 2001, a restricted and more comprehensive database that would provide information for government agencies to use in evaluating the program. GAO was asked to identify the (1) types of data that agencies report to SBA for inclusion in the Tech-Net database, (2) extent to which these data are provided in a standard format, (3) extent to which SBA has established the government-use database, and (4) extent to which SBIR agencies have developed and implemented techniques to track commercialization of SBIR projects. GAO reviewed 8 of the 11 agencies participating in SBIR.

What GAO Found

Federal agencies participating in the SBIR program annually submit over 40 data elements to SBA for each award they make. These data include information on the award, such as value and a descriptive abstract; information on the recipient, such as name and gender; and information about the firm receiving the award, such as number of employees and location. Participating agencies submit most of the information required by SBA, but they are not consistently providing all required data elements, including the number of employees in the firm, and the gender and socio-economic status of the award recipient, resulting in incomplete sections of the database. Agencies stated that this happens because they do not collect all of the information that SBA wants and because SBA's requirements change regularly. Some participating agencies are not submitting SBIR award data in the standard format required by SBA, and although SBA's quality assurance processes correct most of these problems, they do not correct all of them. In 2004 and 2005, about 25 percent of the data provided by five participating agencies did not comply with SBA's format. Formatting inconsistencies occur because the template SBA has provided agencies for reporting data can be edited. According to SBA, identifying and correcting inconsistently formatted data involves considerable resources, therefore the agency has focused its quality assurance efforts only on key data elements needed to track awards; other fields, such as those containing demographic data, are generally not corrected. As a result, comprehensive program evaluations may be limited by the quality of the data in these fields. SBA officials expect this problem to be resolved by fiscal year 2007, when all data will be submitted via an Internet interface that will not allow changes in the format. SBA is 5 years behind schedule in meeting the congressional mandate to implement a restricted government-use database for the SBIR program. SBA had planned to meet this requirement by expanding its Tech-Net database to include a restricted government-use section. SBA officials attributed the delay in meeting the 2001 deadline primarily to increased security requirements needed for the database, agency management changes, and budgetary constraints. SBA officials expect the government-use section of Tech-Net to be operational by October 1, 2006, when safeguards to protect the proprietary commercialization information in the database are in place. Most agencies GAO reviewed systematically gather data on the commercialization success of SBIR-funded projects. Five of these eight agencies regularly survey all awardees to gather information on program participation, including commercial success, and one agency is about to start a similar survey. In contrast, two agencies only gather anecdotal success stories from a small sample of SBIR awardees. SBA and agency officials generally agree that despite their best efforts, obtaining commercialization information from awardees remains a major challenge.