calc_distmx command

Generate a distance matrix from an input file in FASTA or FASTQ format.

The distance matrix filename is specified by the -distmxout option.

The matrix format is specified by the -format option, which can be tabbed_pairs (default), square, phylip_square or phylip_lower_triangle. See distance matrix for details for these file formats.

Clusters can be generated from a distance matrix with the cluster_aggd command.

By default, pairs are prioritized by the U-sort heuristic as used in the USEARCH algorithm. This means that pairs are considered in decreasing order of the number of unique words (U) they have in common. Since U correlates with identity, this means that pairs are considered in approximately increasing order of distance. U-sorting can be turned off using the -nousort option. U-sorting plus additional heuristics used to find HSPs can all be disabled using the -distmx_brute option, which forces all pairs of sequences to be aligned. This is guaranteed to give a complete matrix, but can be much slower for large datasets. Note that low-identity pairs generally have little effect on clustering or tree topology, so the additional "accuracy" of a brute force calculation often has little biological value.

The -sparsemx_minid option gives the minimum identity which should be written to a matrix in tabbed_pairs format.

An identity threshold for terminating the calculation can be specified using the termid option, which is in the range 0.0 to 1.0, where 1.0 means identical sequences (100% sequence id). This is a speed optimization that saves time by skipping alignments of low-identity pairs. If a pair is encountered with fractional identity < termid, the calculation is stopped. Because U-sorted order does not correlate perfectly with identity, you should set termid somewhat lower than the minimum identity that you care about. For example, if you want all pairs with >80% id to appear in the matrix, then you might set -termid 0.7. Tests on small datasets can be used to tune -termid to a reasonable value. By default, termid is set to 0 and the calculation continues for all pairs that have at least one word in common. The word length is set by the ‑wordlength option.

Examples

usearch -calc_distmx seqs.fa -distmxout mx.txt -sparsemx_minid 0.8 -termid 0.7

usearch -calc_distmx seqs.fa -distmxout dist.phy -format phylip_lower_triangle
{"ft_lang_label":"__label__en","ft_lang_prob":0.8456496,"math_prob":0.90863526,"size":2391,"snap":"2020-45-2020-50","text_gpt3_token_len":535,"char_repetition_ratio":0.11813992,"word_repetition_ratio":0.010781671,"special_character_ratio":0.202844,"punctuation_ratio":0.107551485,"nsfw_num_words":0,"has_unicode_error":false,"math_prob_llama3":0.95067585,"pos_list":[0],"im_url_duplicate_count":[null],"WARC_HEADER":"{\"WARC-Type\":\"response\",\"WARC-Date\":\"2020-11-29T08:10:14Z\",\"WARC-Record-ID\":\"<urn:uuid:48968fe3-151a-440d-8dc2-983b5e5a7d15>\",\"Content-Length\":\"5652\",\"Content-Type\":\"application/http; msgtype=response\",\"WARC-Warcinfo-ID\":\"<urn:uuid:106c52b0-dbc5-4660-a082-7e14c6f7d905>\",\"WARC-Concurrent-To\":\"<urn:uuid:76ad7b79-1d2d-4baa-ae2b-b82c93a3f23a>\",\"WARC-IP-Address\":\"199.195.116.69\",\"WARC-Target-URI\":\"http://www.drive5.com/usearch/manual8.1/cmd_calc_distmx.html\",\"WARC-Payload-Digest\":\"sha1:YQVCVB5UIHR7D4BJUMM4WSUL4YHGVB4C\",\"WARC-Block-Digest\":\"sha1:BGRAQW65LWUWLGLNPEX4E6RVPWT2TSQV\",\"WARC-Identified-Payload-Type\":\"text/html\",\"warc_filename\":\"/cc_download/warc_2020/CC-MAIN-2020-50/CC-MAIN-2020-50_segments_1606141197278.54_warc_CC-MAIN-20201129063812-20201129093812-00456.warc.gz\"}"}