I'd like to learn the differences between 3 common formats such as FASTA, FASTQ and SAM. How they are different? Are there any benefits of using one over another?Based on Wikipedia pages, I can't tell the differences between them.
What is the difference between FASTA, FASTQ, and SAM file formats?
file formats;fasta;fastq;sam
Lets start with what they have in common: All three formats storesequence data, andsequence metadata.Furthermore, all three formats are text-based.However, beyond that all three formats are different and serve different purposes.Lets start with the simplest format:FASTAFASTA stores a variable number of sequence records, and for each record it stores the sequence itself, and a sequence ID. Each record starts with a header line whose first character is >, followed by the sequence ID. The next lines of a record contain the actual sequence.The Wikipedia artice gives several examples for peptide sequences, but since FASTQ and SAM are used exclusively (?) for nucleotide sequences, heres a nucleotide example:>Mus_musculus_tRNA-Ala-AGC-1-1 (chr13.trna34-AlaAGC)GGGGGTGTAGCTCAGTGGTAGAGCGCGTGCTTAGCATGCACGAGGcCCTGGGTTCGATCCCCAGCACCTCCA>Mus_musculus_tRNA-Ala-AGC-10-1 (chr13.trna457-AlaAGC)GGGGGATTAGCTCAAATGGTAGAGCGCTCGCTTAGCATGCAAGAGGtAGTGGGATCGATGCCCACATCCTCCAThe ID can be in any arbitrary format, although several conventions exist.In the context of nucleotide sequences, FASTA is mostly used to store reference data; that is, data extracted from a curated database; the above is adapted from GtRNAdb (a database of tRNA sequences).FASTQFASTQ was conceived to solve a specific problem of FASTA files: when sequencing, the confidence in a given base call (that is, the identity of a nucleotide) varies. This is expressed in the Phred quality score. FASTA had no standardised way of encoding this. By contrast, a FASTQ record contains a sequence of quality scores for each nucleotide.A FASTQ record has the following format:A line starting with @, containing the sequence ID.One or more lines that contain the sequence.A new line starting with the character +, and being either empty or repeating the sequence ID.One or more lines that contain the quality scores.Heres an example of a FASTQ file with two records:@071112_SLXA-EAS1_s_7:5:1:817:345GGGTGATGGCCGCTGCCGATGGCGTCAAATCCCACC+IIIIIIIIIIIIIIIIIIIIIIIIIIIIII9IG9IC@071112_SLXA-EAS1_s_7:5:1:801:338GTTCAGGGATACGACGTTTGTATTTTAAGAATCTGA+IIIIIIIIIIIIIIIIIIIIIIIIIIIIIIII6IBIFASTQ files are mostly used to store short-read data from high-throughput sequencing experiments. As a consequence, the sequence and quality scores are usually put into a single line each, and indeed many tools assume that each record in a FASTQ file is exactly four lines long, even though this isnt guaranteed.As for FASTA, the format of the sequence ID isnt standardised, but different producers of FASTQ use fixed notations that follow strict conventions.SAMSAM files are so complex that a complete description [PDF] takes 15 pages. So heres the short version.The original purpose of SAM files is to store mapping information for sequences from high-throughput sequencing. As a consequence, a SAM record needs to store more than just the sequence and its quality, it also needs to store information about where and how a sequence maps into the reference.Unlike the previous formats, SAM is tab-based, and each record, consisting of either 11 or 12 fields, fills exactly one line. Heres an example (tabs replaced by fixed-width spacing):r001  99  chr1  7 30  17M         =  37  39  TTAGATAAAGGATACTG   IIIIIIIIIIIIIIIIIr002  0   chrX  9 30  3S6M1P1I4M  *  0   0   AAAAGATAAGGATA      IIIIIIIIII6IBI    NM:i:1For a description of the individual fields, refer to the documentation. The relevant bit is this: SAM can express exactly the same information as FASTQ, plus, as mentioned, the mapping information. However, SAM is also used to store read data without mapping information.In addition to sequence records, SAM files can also contain a header, which stores information about the reference that the sequences were mapped to, and the tool used to create the SAM file. Header information precede the sequence records, and consist of lines starting with @.SAM itself is almost never used as a storage format; instead, files are stored in BAM format, which is a compact binary representation of SAM. It stores the same information, just more efficiently, and in conjunction with a search index, allows fast retrieval of individual records from the middle of the file (= fast random access). BAM files are also much more compact than compressed FASTQ or FASTA files.The above implies a hierarchy in what the formats can store: FASTA  FASTQ  SAM.In a typical high-throughput analysis workflow, you will encounter all three file types:FASTA to store the reference genome/transcriptome that the sequence fragments will be mapped to.FASTQ to store the sequence fragments before mapping.SAM/BAM to store the sequence fragments after mapping.