SBA’s Lender Risk Rating System Uses Some of the Same Types of Data That Federal Financial Regulators and Selected Lenders Rely on to Conduct Off-Site Monitoring

SBA’s lender risk rating system uses some of the same types of data that  federal financial regulators and selected lenders rely on for off-site  monitoring. Like federal financial regulators and large lenders, SBA uses performance  data and prospective measures to calculate lender risk ratings. Usefulness of SBA’s Lender Risk Rating System Has Been Limited because SBA Does Not Ensure That Its Contractor Follows Sound Validation Techniques

The effectiveness of SBA’s lender risk rating system has been limited  because the agency’s contractor does not follow sound validation  practices. For  example, changes in economic conditions and industry trends can affect  model outcomes. SBA Does Not Use Its Own Data to Assess or Supplement the Contractor’s Validation of the Lender Risk Rating System

In addition to not ensuring that its contractor follows sound validation  techniques, SBA does not conduct its own analysis of data to supplement  the contractor’s validation of the lender risk rating system. However, we found that SBA data could  be useful for developing alternate measures of lender performance in  order to independently validate the lender risk rating system’s results. Further, SBA could use its own data to develop alternate measures, such  as currency rates, as performance benchmarks. SBA Does Not Use Lender Risk Ratings to Target Lenders for On-Site Review or Tailor the Scope of the Reviews

		SBA Has Used the Lender Risk Rating System to Conduct Some Off-Site Monitoring of Lenders and Their Portfolios

SBA uses its lender risk rating system to conduct off-site monitoring of  lenders and their portfolios. Lender Risk Ratings Do Not Inform the Scope of SBA’s On-Site Reviews, and Reviews Do Not Include an Assessment of Lenders’ Credit Decisions

Unlike federal financial regulators, SBA does not rely on its lender risk  ratings to help focus the scope of on-site reviews, and the reviews do not  include an assessment of the lenders’ credit decisions. The federal  financial regulators we interviewed rely on results from their off-site  monitoring systems to identify which areas of a bank’s operations they  should review more closely. They are then able to use the results to inform their off-site  monitoring systems. According to SBA officials, the file reviews focus on compliance  with SBA policy because it is not SBA’s role to evaluate lenders’ credit  decisions. Because SBA officials choose not to rely on lender risk ratings to inform  file reviews conducted during on-site reviews or assess lenders’ credit  decisions during the reviews, the agency does not have the type of  information related to the quality of the underwriting standards and  practices of lenders that is necessary to understand the risks that banks  pose to SBA’s portfolio. SBA did not require its contractor to ensure that personnel other  than the staff who developed the model validated it or to routinely  reassess the factors used in the system as part of its validations. As a result, we found that SBA conducted on-site reviews of only 3  percent of the lenders that the lender risk rating system identified as high  risk in 2008. These revised policies and procedures could require staff to (1)  use lender risk ratings to tailor the scope of file reviews performed during  on-site reviews to areas that pose the greatest risk, (2) incorporate an  assessment of lenders’ credit decisions in file reviews, and (3) use the  results of expanded file reviews to identify information, such as emerging  lending trends, that could be incorporated into its lender risk rating  system. Key contributors to this report are listed in appendix V.

Appendix I: Objectives, Scope, and Methodology

In this report, we examined (1) how the Small Business Administration’s  (SBA) risk rating system compares with the off-site monitoring tools used  by federal financial regulators and lenders and the system’s usefulness for  predicting lender performance and (2) how SBA uses the lender risk rating  system in its lender oversight activities. We interviewed officials from three federal financial  regulators—the Office of the Comptroller of the Currency (OCC), the  Board of Governors of the Federal Reserve System (the Federal Reserve),  and the Federal Deposit Insurance Corporation (FDIC)—five of the largest  7(a) lenders, and the five largest 504 lenders. We assessed SBA’s lender  risk rating system against common industry standards and our internal  control standards. Using SBA’s data, we undertook a number of evaluative steps to test the  agency’s model.
Why GAO Did This Study

The Small Business Administration (SBA) guarantees individual loans that lenders originate. The agency uses its Loan and Lender Monitoring System (L/LMS) to assess the individual risk of each loan, and SBA's contractor developed a lender risk rating system based on L/LMS data. However, questions have been raised about the extent to which SBA has used its lender risk rating system to improve its oversight of lenders. The Government Accountability Office (GAO) was asked to examine (1) how SBA's risk rating system compares with those used by federal financial regulators and lenders and the system's usefulness for predicting lender performance and (2) how SBA uses the lender risk rating system in its lender oversight activities. To meet these objectives, GAO reviewed SBA documents; interviewed officials from three federal financial regulators and 10 large SBA lenders; analyzed SBA loan data; and interviewed SBA officials.

What GAO Found

SBA's lender risk rating system uses some of the same types of information that federal financial regulators and selected large lenders use to conduct off-site monitoring, but its usefulness has been limited because SBA has not followed common industry standards when validating the system--that is, assessing the system's ability to accurately predict outcomes. Like the federal financial regulators and 10 large lenders GAO interviewed, SBA's contractor developed lender risk ratings based on loan performance data and prospective, or forward-looking, measures (such as credit scores). Using SBA data, GAO undertook a number of evaluative steps to test the lender risk rating system's predictive ability. GAO found that the system was generally successful in distinguishing between higher- and lower-risk lenders, but it better predicted the performance of larger lenders. However, the system's usefulness was limited because the contractor did not follow validation practices, such as independent and ongoing assessments of the system's processes and results, consistent with those recommended by federal financial regulators and GAO's internal control standards. For example, the agency did not require a party other than the one who developed the system to perform the validation, and SBA's contractor did not routinely reassess the factors used in the system as part of its validations. Further, SBA does not use its own data to develop alternate measures of lender performance that could be used to independently assess or supplement the risk ratings, citing resource constraints. Because SBA does not follow sound validation practices or use its own data to independently assess the risk ratings, the effectiveness of its lender risk rating system--the primary system SBA relies on to monitor and predict lender performance--may deteriorate as economic conditions and industry trends change over time. Although SBA's lender risk rating system has enabled the agency to conduct some off-site monitoring of lenders, the agency does not use the system to target lenders for on-site reviews or to inform the scope of the reviews. Unlike the Federal Deposit Insurance Corporation and the Federal Reserve, which use their off-site monitoring tools to target lenders for on-site reviews, SBA targets for review those lenders with the largest SBA-guaranteed loan portfolios. As a result of this approach, 97 percent of the lenders that SBA's risk rating system identified as high risk in 2008 were not reviewed. Further, GAO found that the scope of the on-site reviews that SBA performs is not informed by the lenders' risk ratings, and the reviews do not include an assessment of lenders' credit decisions. The federal financial regulators use the results of off-site monitoring to identify which areas of a bank's operations they should review more closely. Moreover, their reviews include an assessment of the quality of the lenders' credit decisions. Federal financial regulators are able to use review results to update their off-site monitoring systems with data on emerging lending trends. Regardless of the lender's risk rating, SBA relies on a standard on-site review form that includes an assessment of lenders' compliance with SBA policies and procedures but not an assessment of lenders' credit decisions. According to SBA officials, it is not the agency's role to assess lenders' credit decisions. Without targeting the most risky lenders for on-site reviews or gathering information related to lenders' credit decisions, SBA cannot effectively assess the risk posed by lenders or ensure that its lender risk rating system incorporates updated information on emerging lending trends.