Python performance on reading files with extremely long lines
Question: I've got a file with around 6MB of data. All of the data are written in a
single line. Why is the following command taking more than 15 minutes to
finish? Is it normal?

    
    
    infile = open('file.txt')
    outfile = open('out.txt', 'w')
    for line in infile.readlines():
        outfile.write(line);
    

**Details:**

I'm using Python 2.7.

Output from **wc** :

  * newline count: 2
  * word count: 3475246 
  * byte count: 6951140 

**Evaluation 1:**

_Reference_ code using in file (suggestted by Ahsanul Haque and Daewon Lee):

    
    
    for line in infile:
        output.write(line):
    

_Time: 959.487 secs._


Answer: Try the following code snippet. The `readlines()` loads all data on memory,
which seems to cause a long time.

    
    
    infile = open('file.txt', 'r')
    outfile = open('out.txt', 'w')
    for line in infile:
        outfile.write(line)
    

With my python 3.5 (64bit) on Windows 10 OS, the following code snippet
finished within a few seconds.

    
    
    import time
    
    start = time.time()
    
    with open("huge_text.txt", "w") as fout:
        for i in range(1737623):
            fout.write("ABCD ")
        fout.write('\n')   
        for i in range(1737623):
            fout.write("EFGH ")
        fout.write('\n')
    # end of with
    
    infile = open('huge_text.txt', 'r')
    outfile = open('out.txt', 'w')
    for line in infile:
        outfile.write(line)
    
    outfile.close()
    infile.close()
    
    end = time.time()
    print("Time elapsed: ", end - start)
    
    """
    <Output>
    Time elapsed:  1.557690143585205
    """
    

