21. A method of forming compression dictionaries in a computer system that includes data sets with corresponding data catalogs that contain information identifying the format of each data set and its location, the data sets containing records of uninterrupted byte strings that will be compressed by the computer system using the compression dictionaries, the method comprising the steps of: (a) providing a library of dictionary segments that are adapted for use by the computer system to compress the data sets; (b) scanning one or more records of a data set during an initial recordation of the data set records to one or more external storage devices and generating predetermined data characteristic statistics as a result of the scanning; (c) selecting dictionary segments from the library of dictionary segments according to the generated data characteristic statistics; and (d) combining the selected dictionary segments to form a system-built dictionary that is used by the computer system to compress the remaining data records of the data set as the data set is recorded to the one or more external storage devices, wherein the generated data characteristic statistics include a frequency distribution of characters found in the scanned record, a count of occurrences for each character in the scanned record, and an entropy value for the scanned record; and the step of scanning records comprises: completing a stabilization procedure comprising repeating the steps of selecting a current first block of the record for scanning and generating a first set of the data characteristic statistics, selecting a current second block of the record, different from the first block, for scanning and generating a second set of data characteristic statistics, comparing like statistics of the first set of data characteristic statistics and the second set of data characteristic statistics, and selecting a new first block and a new second block for scanning and generating new respective sets of data characteristic statistics until the step of comparing indicates that the statistical difference between the first and second sets of statistics is below a predetermined stabilization threshold value, or until the size of the second block equals a predetermined scanning limit value, whereupon the combination of the first block and second block is stored in an interrogation buffer if the entropy value of the second set of statistics is below a predetermined entropy threshold value.
21. A method of forming compression dictionaries in a computer system that includes data sets with corresponding data catalogs that contain information identifying the format of each data set and its location, the data sets containing records of uninterrupted byte strings that will be compressed by the computer system using the compression dictionaries, the method comprising the steps of: (a) providing a library of dictionary segments that are adapted for use by the computer system to compress the data sets; (b) scanning one or more records of a data set during an initial recordation of the data set records to one or more external storage devices and generating predetermined data characteristic statistics as a result of the scanning; (c) selecting dictionary segments from the library of dictionary segments according to the generated data characteristic statistics; and (d) combining the selected dictionary segments to form a system-built dictionary that is used by the computer system to compress the remaining data records of the data set as the data set is recorded to the one or more external storage devices, wherein the generated data characteristic statistics include a frequency distribution of characters found in the scanned record, a count of occurrences for each character in the scanned record, and an entropy value for the scanned record; and the step of scanning records comprises: completing a stabilization procedure comprising repeating the steps of selecting a current first block of the record for scanning and generating a first set of the data characteristic statistics, selecting a current second block of the record, different from the first block, for scanning and generating a second set of data characteristic statistics, comparing like statistics of the first set of data characteristic statistics and the second set of data characteristic statistics, and selecting a new first block and a new second block for scanning and generating new respective sets of data characteristic statistics until the step of comparing indicates that the statistical difference between the first and second sets of statistics is below a predetermined stabilization threshold value, or until the size of the second block equals a predetermined scanning limit value, whereupon the combination of the first block and second block is stored in an interrogation buffer if the entropy value of the second set of statistics is below a predetermined entropy threshold value. 23. A method as defined in claim 21, wherein the new first block comprises the combination of the current first block and second block.