Getting the frequency of letters in each position I have a text file like this example:

&gt;chr12:86512-86521
CGGCCAAAG
&gt;chr16:96990-96999
CTTTCATTT
&gt;chr16:97016-97025
TTTTGATTA
&gt;chr16:97068-97077
ATTTAGGGA


This file is divided into different parts, and every part has 2 lines. The line which starts with &gt; is ID and the 2nd line is a sequence of letters and the letters are A, T, C or G and also the length of each sequence is 9 so, for every sequence of letters there are 9 positions. I want to get the frequency of the 4 mentioned letters in every position (we have 9 positions).

Here is the expected output for the small example:

one = {'T': 1, 'A': 1, 'C': 2, 'G': 0}
two = {'T': 3, 'A': 0, 'C': 0, 'G': 1}
three = {'T': 3, 'A': 0, 'C': 0, 'G': 1}
four = {'T': 3, 'A': 0, 'C': 1, 'G': 0}
five = {'T': 0, 'A': 1, 'C': 2, 'G': 1}
six  = {'T': 0, 'A': 3, 'C': 0, 'G': 1}
seven = {'T': 2, 'A': 1, 'C': 0, 'G': 1}
eight = {'T': 2, 'A': 1, 'C': 0, 'G': 1}
nine = {'T': 1, 'A': 2, 'C': 0, 'G': 1}


I am doing that in Python using the following command. This command has 3 steps. Steps 1 and 2 work fine, but would you help me to improve step 3, which made this pipeline so slow for big files?

Step 1: to parse the file into a comma-separated file

def fasta_to_textfile(filename, outfile): 
    with open(filename) as f, open(outfile, 'w') as outfile:
        header = sequence = None
        out = csv.writer(outfile, delimiter=',')
        for line in f:
            if line.startswith('&gt;'):
                if header:
                    entry = header + [''.join(sequence)]
                    out.writerow(entry)
                header = line.strip('&gt;\n').split('|')
                sequence = []
            else:
                sequence.append(line.strip())
        if header:
            entry = header + [''.join(sequence)]
            out.writerow(entry)


Step 2: comma-separated file to a Python dictionary

def file_to_dict(filename):
    f = open(filename, 'r')
    answer = {}
    for line in f:
        k, v = line.strip().split(',')
        answer[k.strip()] = v.strip()
    return answer


To print functions from steps 1 and 2:

a = fasta_to_textfile('infile.txt', 'out.txt')
d = file_to_dict('out.txt')


Step 3: to get the frequency

one=[]
two=[]
three=[]
four=[]
five=[]
six=[]
seven=[]
eight=[]
nine=[]
mylist = d.values()
for seq in mylist:
    one.append(seq[0])
    two.append(seq[1])
    se.append(seq[2])
    four.append(seq[3])
    five.append(seq[4])
    six.append(seq[5])
    seven.append(seq[6])
    eight.append(seq[7])
    nine.append(seq[8])

from collections import Counter
one=Counter(one)    
two=Counter(two)
three=Counter(three)
four=Counter(four)
five=Counter(five)
That's a crazy amount of code for such a simple task.  There's no need to generate any intermediate files.  There's no need to create nine separate variables for the nine columns.

To process an input file (whether from sys.stdin or as a named argument on the command line), use fileinput.  That way, you don't have to hard-code 'infile.txt'.  Then, simply ignore the lines that start with &gt; and strip off the newlines.

To work on columns rather than rows, use zip().

Suggested solution

These seven lines could replace your entire code.

import fileinput
from collections import Counter

def nucleotide_lines(f):
    for line in f:
        if not line.startswith('&gt;'):
            yield line.rstrip()

print([Counter(col) for col in zip(*nucleotide_lines(fileinput.input()))])