1. An information gathering system implemented in a computer system for optimizing searching comprising: a processor and memory; a data extraction tool executing in the computer system, in communication with a database, extracting website content to enable full text searching, the website content being extracted from a plurality of websites associated with business entities that are classified according to a standard industry classification system (SIC), which is a predefined taxonomy of business activities having verified information about the business entities; the database, in communication with the data extraction tool, storing the extracted website content according to a classification system that is based on the predefined taxonomy of SIC business activities; a content analyzer, in communication with the database, identifying commonly occurring keywords in the extracted website content from the websites of business entities that are similarly classified in the SIC predefined taxonomy of SIC business activities, where the commonly occurring keywords identified are used to update the classification system, the updated classification system being used to optimize searching in response to search queries; the content analyzer identifying commonly occurring keywords that are used to create a new category to update the classification system by: identifying keyword matches in the extracted website content by identifying any commonly occurring keywords or phrases in the extracted website content; and processing the matches identified by determining whether any of the keywords or phrases in the identified matches contain one or more keywords associated with any of the business activities in the SIC predefined taxonomy; and a full text indexed search engine, in communication with the database, processing a search query by matching the search query against the database, where at least a portion of the search results are clustered based on their respective SIC business activity category.