NHTSA  collects NASS-CDS data through in-depth investigations of a sample of  police-reported motor vehicle crashes that occur in the United States. Between 1988 and 2013, NHTSA conducted  an average of about 4,700 NASS-CDS investigations each year. According to NHTSA, factors that  have contributed to this decline include the budget for NASS-CDS and  rising costs. Specific to NASS-CDS, NHTSA’s Data Modernization Project  involves the following:  redesigning the NASS-CDS sample by reviewing the data elements  that comprise the sample and the statistical methodology behind  selecting the sample; upgrading the equipment and information technology that supports  NASS-CDS to reduce redundancy, improve data quality, and enhance  the experience of NASS users; and  implementing a new sample to replace NASS-CDS. NHTSA Followed a Reasonable Process for Redesigning NASS-CDS

		NHTSA Redesigned NASS-CDS in Accordance with Applicable OMB Standards and Guidelines

We found that the process NHTSA followed to redesign NASS-CDS is  consistent with applicable government-wide standards and guidelines  issued by OMB that apply to the development of survey concepts,  methods, and design. To redesign NASS-CDS, NHTSA awarded a contract to Westat in May  2012 to assist the agency in redesigning the NASS-CDS sample. Consistent with OMB’s recommended practice to include design elements  to meet stated objectives, NHTSA tasked Westat with (1) identifying data  elements that are responsive to the current and future needs of both  NHTSA and the public and (2) developing recommendations for a new  sample design that met users’ data needs in an effective and efficient  manner while still maintaining national representativeness. Thus, in May 2014,  NHTSA announced that it planned to replace NASS-CDS with a new  system called the Crash Investigation Sampling System (CISS)—which  we discuss in detail in objective 2. Specifically, MAP-21 required NHTSA to conduct a  comprehensive review of the data elements collected as part of NASS  and report on whether there was a benefit to increasing the size of the  NASS sample. For example, the act required NHTSA to provide  Congress with information on the types of analyses that can be  conducted and the conclusions that can be drawn under the current  sample size and an expanded sample size, the number of investigations  that NHTSA should conduct as part of the sample that would allow for  optimal data analysis, NHTSA’s recommendations for improvements, and  the resources necessary to implement NHTSA’s recommendations. The act required NHTSA to  report to Congress on the results of its review, including the benefits of a  larger sample size, no later than October 1, 2013. NHTSA missed this  deadline and issued its report in January 2015, as we were completing  our review. In its report, NHTSA stated that meeting the needs of all NASS users is a  challenge and that there is no precise answer to what the optimal sample  size for NASS-CDS would be. However, NHTSA also noted that  increasing the size of the NASS-CDS sample would help meet the  evolving needs of NASS-CDS users. The CISS May Not Substantially Increase the Sample Size but May Improve Precision and Data Quality

		Although Its Sample Size May Not Substantially Increase, NHTSA Still Expects the CISS to Generate More Precise Crash and Injury Estimates

One means of determining the extent the Data Modernization Project  redesign will improve the NASS-CDS sample is to assess the potential for  CISS to meet a main technical objective of the Data Modernization  Project: achieving similar or greater levels of statistical precision for seven  important crash and injury estimates. There are many ways to design a sample to generate more statistically  precise estimates. One way NHTSA improved the expected statistical  precision of estimates from CISS was by selecting new PSUs that better  represent the current population and the number and types of crashes  nationwide. Moreover, NHTSA expects the new sample design for CISS to contain  more crashes with serious injuries and crashes involving newer vehicles  than NASS-CDS currently contains, which also should make estimates of  these crashes more statistically precise. NHTSA officials said  the new equipment they plan to provide should help reduce the time it  currently takes to conduct scene and vehicle inspections. This includes data on the use of crash-avoidance technologies in  newer vehicles as well as additional data for older vehicles, as some  users had requested. By increasing the size of the CISS  sample, NHTSA and others could likely do more to study motor-vehicle  traffic crashes in an effort to save lives and reduce the economic costs of  crashes. The Department of Transportation provided  technical comments, which we incorporated as appropriate. Appendix I: Objectives, Scope, and Methodology

This report assesses (1) the process the National Highway Traffic Safety  Administration (NHTSA) used to redesign the National Automotive  Sampling System Crashworthiness Data System (NASS-CDS) and (2)  the potential for this redesign to improve the NASS-CDS sample. These OMB documents also specify the professional principles and  practices that federal agencies should follow and the level of quality and  effort expected when initiating a new survey or redesigning an existing  survey such as NASS-CDS.
Why GAO Did This Study

In 2010, motor vehicle crashes in the United States cost almost 33,000 lives, injured 2.2 million people, and resulted in almost $900 billion in economic costs. As part of its mission to reduce these losses, NHTSA collects and analyzes data on motor vehicle crashes. One NHTSA program that collects crash data is NASS-CDS—a nationally representative sample of police-reported motor-vehicle traffic crashes; however, the NASS-CDS sample was designed in 1988, and subsequent shifts in the population and a declining sample size have necessitated an update of this sample. In 2012, NHTSA started taking steps to redesign NASS-CDS.
Congress mandated GAO to review NHTSA's progress in redesigning NASS-CDS. This report assesses the (1) process NHTSA used to redesign NASS-CDS and (2) the potential for this redesign to improve the NASS-CDS sample. To conduct this review, GAO reviewed relevant information regarding the NASS-CDS redesign and interviewed officials from NHTSA and Westat, the contractor selected to assist NHTSA in redesigning NASS-CDS. Based on comments the public submitted to NHTSA in response to a notice in the Federal Register , GAO also interviewed 21 users of this data and other interested parties regarding the improvements they would like made to NASS-CDS. The Department of Transportation reviewed a draft of this report and provided technical comments, which were incorporated as appropriate.

What GAO Found

The National Highway Traffic Safety Administration (NHTSA) followed a reasonable process for redesigning the National Automotive Sampling System Crashworthiness Data System (NASS-CDS), which is a nationally representative sample of police-reported motor-vehicle traffic crashes. The Office of Management and Budget (OMB) has standards and guidelines that specify the professional principles and practices that agencies should follow and the level of quality and effort expected when redesigning an existing survey, such as NASS-CDS. NHTSA followed a process consistent with applicable OMB standards and guidelines. For example, NHTSA consulted with NASS-CDS users to identify their requirements and expectations in redesigning NASS-CDS and tasked the contractor, Westat, with developing proposals for a new sample design to meet users' data needs in an effective and efficient manner. As of January 2015, NHTSA planned to replace NASS-CDS with a new sample, called the Crash Investigation Sampling System (CISS). However, NHTSA did not meet a congressional deadline to report on the benefits of increasing the size of the NASS-CDS sample. Specifically, the Moving Ahead for Progress in the 21st Century Act required NHTSA to report, by October 1, 2013, on whether there would be a benefit to increasing the size of the NASS sample as well as to report on the resources necessary to implement NHTSA's recommended sample size, among other things. NHTSA issued its required report in January 2015 as GAO was completing its review. In its report, NHTSA noted that increasing the size of the NASS-CDS sample would help meet the evolving needs of NASS users, but stated there was no precise answer to what an optimal sample size for NASS-CDS would be.
NHTSA expects the new sample it plans to implement as part of this redesign to generate greater statistical precision for key crash-type and injury-severity estimates than that of NASS-CDS using a similarly sized sample. One way NHTSA was able to generate more precise estimates was by selecting new sites at which to collect data. These sites, or “primary sampling units,” better represent the current population and distribution of motor vehicle crashes nationwide, representation that allows NHTSA and others to generate more precise estimates using the data. NHTSA also expects CISS to sample more crashes involving serious injuries and newer vehicles than NASS-CDS currently allows, as users had requested. NHTSA conducted about 4,700 NASS-CDS investigations annually between 1988 and 2013, and while there is no clear optimal sample size, a larger sample size could allow NHTSA to generate estimates that are even more precise or generate estimates for types of crashes that occur infrequently, estimates that could contribute to research that can affect vehicle safety. However, NHTSA's ability to increase the new CISS sample size is limited by its current and expected budget. Additional planned improvements to NASS-CDS include new technologies that allow for safer and more accurate measurements of accident scenes and vehicles involved in crashes. While NHTSA expects these new technologies to also result in some time savings, NHTSA does not expect them to allow for more investigations due to the time-intensive nature of the CISS data-collection effort.