	What Is Data Mining? Data mining involves the use of sophisticated data analysis tools to discover previously unknown, valid patterns and relationships in large data sets. Data mining has become increasingly common in both the public and private sectors. To be successful, data mining requires skilled technical and analytical specialists who can structure the analysis and interpret the output that is created. Although data mining can help reveal patterns and relationships, it does not tell the user the value or significance of these patterns. These types of determinations must be made by the user. Another limitation of data mining is that while it can identify connections between behaviors and/or variables, it does not necessarily identify a causal relationship. Industries such as banking, insurance, medicine, and retailing commonly use data mining to reduce costs, enhance research, and increase sales. In the public sector, data mining applications were initially used as a means to detect fraud and waste, but they have grown also to be used for purposes such as measuring and improving program performance. Some observers suggest that data mining should be used as a means to identify terrorist activities, such as money transfers and communications, and to identify and track individual terrorists themselves, such as through travel and immigration records. Initiatives that have attracted significant attention include the now-discontinued Terrorism Information Awareness (TIA) project conducted by the Defense Advanced Research Projects Agency (DARPA), and the now-canceled Computer-Assisted Passenger Prescreening System II (CAPPS II) that was being developed by the Transportation Security Administration (TSA). Other initiatives that have been the subject of congressional interest include the Able Danger program and data collection and analysis projects being conducted by the National Security Agency (NSA). As with other aspects of data mining, while technological capabilities are important, other factors also influence the success of a project's outcome. Data quality refers to the accuracy and completeness of the data. Mission creep refers to the use of data for purposes other than that for which the data was originally collected. As data mining efforts move forward, Congress may consider a variety of questions including, the degree to which government agencies should use and mix commercial data with government data, whether data sources are being used for purposes other than those for which they were originally designed, and the possible application of the Privacy Act to these initiatives. 1502 , information regarding the technology and data being used; information on how the technology would be used and the target dates for deployment; an assessment of the likely efficacy of the data mining technology; an assessment of the likely impact of the activity on privacy and civil liberties; a list and analysis of the laws and regulations that would apply to the data mining activity and whether these laws and regulations would need to be modified to allow the data mining activity to be implemented; information on the policies, procedures, and guidelines that would be developed and applied to protect the privacy and due process rights of individuals, and ensure that only accurate information is collected and used; and information on how individuals whose information is being used in the data mining activity will be notified of the use of their information, and, if applicable, what options will be available for individual to opt-out of the activity. CRS Report RL32536, The Multi-State Anti-Terrorism Information Exchange (MATRIX) Pilot Project , by [author name scrubbed] (pdf).
Data mining has become one of the key features of many homeland security initiatives. Often used as a means for detecting fraud, assessing risk, and product retailing, data mining involves the use of data analysis tools to discover previously unknown, valid patterns and relationships in large data sets. In the context of homeland security, data mining can be a potential means to identify terrorist activities, such as money transfers and communications, and to identify and track individual terrorists themselves, such as through travel and immigration records.
While data mining represents a significant advance in the type of analytical tools currently available, there are limitations to its capability. One limitation is that although data mining can help reveal patterns and relationships, it does not tell the user the value or significance of these patterns. These types of determinations must be made by the user. A second limitation is that while data mining can identify connections between behaviors and/or variables, it does not necessarily identify a causal relationship. Successful data mining still requires skilled technical and analytical specialists who can structure the analysis and interpret the output.
Data mining is becoming increasingly common in both the private and public sectors. Industries such as banking, insurance, medicine, and retailing commonly use data mining to reduce costs, enhance research, and increase sales. In the public sector, data mining applications initially were used as a means to detect fraud and waste, but have grown to also be used for purposes such as measuring and improving program performance. However, some of the homeland security data mining applications represent a significant expansion in the quantity and scope of data to be analyzed. Some efforts that have attracted a higher level of congressional interest include the Terrorism Information Awareness (TIA) project (now-discontinued) and the Computer-Assisted Passenger Prescreening System II (CAPPS II) project (now-canceled and replaced by Secure Flight). Other initiatives that have been the subject of congressional interest include the Multi-State Anti-Terrorism Information Exchange (MATRIX), the Able Danger program, the Automated Targeting System (ATS), and data collection and analysis projects being conducted by the National Security Agency (NSA).
As with other aspects of data mining, while technological capabilities are important, there are other implementation and oversight issues that can influence the success of a project's outcome. One issue is data quality, which refers to the accuracy and completeness of the data being analyzed. A second issue is the interoperability of the data mining software and databases being used by different agencies. A third issue is mission creep, or the use of data for purposes other than for which the data were originally collected. A fourth issue is privacy. Questions that may be considered include the degree to which government agencies should use and mix commercial data with government data, whether data sources are being used for purposes other than those for which they were originally designed, and possible application of the Privacy Act to these initiatives. It is anticipated that congressional oversight of data mining projects will grow as data mining efforts continue to evolve. This report will be updated as events warrant.