sort a fasta file containing the Oxford Nanopore Technologies (ONT) header by sequencing start_time ascending I have basecalled ONT reads and converted them to multifasta. The multifasta contains the original ONT headers in this format:

3ebb56cd-3671-4abd-b1ac-0c759bt068d0 runid=eb6214851489c8e00eb0dcbd00d737f7dddxxxx read=1078 ch=422 start_time=2017-11-21T20:35:34Z

By checking the sequencing output as fasta, the sequences are not ordered by start_time.

&gt;f546fab3-39b8-4efb-9bc7-cf6ab19cf6dd runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=363 start_time=2017-11-21T20:36:34Z
TTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATG
&gt;fa9f83ee-ec58-419b-aff7-e40604cb7a34 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=183 start_time=2017-11-21T20:35:09Z
TGGTATGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATG 
&gt;371a075a-6701-4187-affc-7aa267c41850 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=315 start_time=2017-11-21T20:35:57Z
CTCGTTTCAGGTATGCTTCGTTTCAGTTGCCCTCAGGAGCAGATTGTTTATGGGTTACTTTCCTGTCGCTCTATCTTCAC 


I learned how to extract header IDs and time_start from ONT fastq files by using this answer. 

I'd like to have a similar approach but with fasta. 

Yet, I would like to create a new fasta file where the sequences are written in an ascending order as they were sequenced (start_time) as follows:

&gt;fa9f83ee-ec58-419b-aff7-e40604cb7a34 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=183 start_time=2017-11-21T20:35:09Z
TGGTATGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATGTGGTATG 
&gt;371a075a-6701-4187-affc-7aa267c41850 runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=315 start_time=2017-11-21T20:35:57Z
CTCGTTTCAGGTATGCTTCGTTTCAGTTGCCCTCAGGAGCAGATTGTTTATGGGTTACTTTCCTGTCGCTCTATCTTCAC
&gt;f546fab3-39b8-4efb-9bc7-cf6ab19cf6dd runid=eb6214851489c8e00eb0dcbd00d73214851489c8 read=100 ch=363 start_time=2017-11-21T20:36:34Z
TTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATGTTGGTATG
Edit: After doing some reading, if it can be assumed that all the timestamps are ISO8601 and of the same representation of ISO861 (there are a few), then they can be sorted lexicographically, directly, without converting to epoch-time integers. So I have edited this answer to add an answer without conversion steps:



Here's one approach that uses the timestamps directly:

(1) Extract the start times from the headers and sort them to another file, in ascending order:

$ awk '(/^&gt;/){ match($0, /start_time=(.*)/, a); print a[1]; }' my.fa | sort &gt; starts_iso8601.sorted.txt


(2) Use that file of ordered ISO8601-formatted starts to write out FASTA records in your sort order:

$ while IFS='' read -r line || [[ -n "$line" ]]; do awk -v line=$line -v RS="&gt;" '{ m = match($0,line); if (m) { printf("&gt;%s", $0); } }' my.fa ; done &lt; starts_iso8601.sorted.txt &gt; my_ordered.fa


Quite a bit cleaner and faster. 

You might separate the timestamp extraction and sort steps into two commands and inspect the intermediate result, just to ensure that you have valid strings. Or use the approach below to convert the timestamps, which will catch bad input.



Here's a second way that may add unnecessary work. As an advantage, however, it adds some error checking by converting the ISO8601-formatted timestamp. 

If the string you pull out of the header is not a valid timestamp, the first conversion step (ISO8601-to-epoch) will fail and you will then be able to catch invalid input pretty quickly, which can add confidence that your output is correct.

(1) Extract the start times from the headers:

$ awk '(/^&gt;/){ match($0, /start_time=(.*)/, a); print a[1]; }' my.fa &gt; starts_iso8601.txt


(2) Convert the ISO8601-formatted times to time-since-epoch times:

$ while IFS='' read -r line || [[ -n "$line" ]]; do date -d"$line" +%s; done &lt; starts_iso8601.txt &gt; starts_epoch.txt


(3) Sort the epoch times — which are just integers — to another file, in ascending order:

$ sort -n starts_epoch.txt &gt; starts_epoch.sorted.txt


(4) Convert sorted epoch starts back to ISO8601 format:

$ while IFS='' read -r line || [[ -n "$line" ]]; do date -d"@$line" -u +"%Y-%m-%dT%H:%M:%SZ"; done &lt; starts_epoch.sorted.txt &gt; starts_iso8601.sorted.txt


(5) Use that file of ordered starts to write out FASTA records in your sort order:

$ while IFS='' read -r line || [[ -n "$line" ]]; do awk -v line=$line -v RS="&gt;" '{ m = match($0,line); if (m) { printf("&gt;%s", $0); } }' my.fa ; done &lt; starts_iso8601.sorted.txt &gt; my_ordered.fa


Note: Use gawk, gdate, gsort in place of awk, date and sort if you're using OS X, Homebrew, and GNU core utilities.

I'm showing the steps separately to explain the general procedure. 

Many or all of these steps can be glued together in a Unix pipeline to save the time, memory and disk space costs of writing intermediate files. 

Also, I'm not doing much error checking here. You might look at cases where headers do not contain start_time, or times are not all in ISO8601, or other cases not anticipated here.