Printing a list to a file then reassembling said list This is a GCSE practice problem, to practice for the actual coursework. Computing's my passion and I'd like to impress the examiners (note: any tips would be appreciated)

Essentially, this is simply splitting by lines into an array, printing said array into a file, then reassembling it by looping through one list by indexing the other.

uniqueword =[]
for i in range(0,len(sentencelist)):
    if not sentencelist[i] in uniqueword:#&lt;--- creates list of unique words
        uniqueword.append(sentencelist[i])
    else:
        i+=1
print(uniqueword)
positionlist = []
for i in range(0,len(sentencelist)):
    positionlist.append(uniqueword.index(sentencelist[i]))#&lt;-- finds the index of the unique words in the sentence and appends them to positionlist
print(positionlist)
wordvals = []
end=False
im = 0
while True:
    try:
        wordvals.append(uniqueword[im])
        im+=1
    except:
        break
print(wordvals)
text_file = open("sentencemanip.txt", "w")#prints by line on new lines
for item in positionlist:
  text_file.write("%s\n" % item)
text_file.close()
text_file = open("sentencemanip1.txt", "w")
for item in wordvals:
  text_file.write("%s\n" % item)#whee
text_file.close()
print('the sentence has been printed to the text files sentencemanip.txt and sentencemanip1.txt')


import sys
sentencelist = []
def splitsent(text):

    # split the text
    words = text.split()

    # for each word in the line:
    for word in words:
        sentencelist.append(word)


and    

    import sys
    pl = []
    wv= []
    with open('sentencemanip.txt') as f:
        pl = f.read().splitlines() 
    f.close()
    with open('sentencemanip1.txt') as f: 
    wv=f.read().splitlines()
    f.close()
    print(pl)
    print(wv)
    for i in pl: print(wv[int(i)],'',end = '')#&lt;- here


The question is, is there a better/more efficient/faster way to accomplish this task? Am I making any elementary errors in variable assignment? Would while or for loops be better?

Any help is appreciated on my continuing quest to become not awful at python!
Forewords

Let's break down the problem once more:


you have a text and want to extract individual words out of it;
with these words, you want to build a list of unique ones.
you replace each word by its position in the list of uniques;
and write these positions and the list of uniques into two files.


After that, an other part can read both files and rebuild the original text.

Now let's get back to the code and put everything into a function to organize things a bit and allow for better reusability and testing. One function for the text to files: def split_sentence(text, filename_indexes, filename_uniques): and one for the files to text: def build_sentence(filename_indexes, filename_uniques):. You can see that by giving meaningful names to parameters, we can get a better grasp at what they are used for.

Text to file

Let's iterate over each part of the algorithm and see what can be improved. Remember, we are defining:

def split_sentence(text, filename_indexes, filename_uniques):


Improving

Extract individual words from a text


sentencelist = []

# split the text
words = text.split()

# for each word in the line:
for word in words:
    sentencelist.append(word)



Each "thing" that you can place after in in a for loop (such as words here) is an iterable. And building a list by appending elements from an iterable into a for loop is a waste of time: the list constructor already does the exact same thing. So, at the very least, you should do words = text.split(); sentencelist = list(words).

But here, words is not only an iterable, it is also already a list. So you don't need to copy it over. The only thing you need is:

    sentence_list = text.split()


Build a list of unique words


uniqueword =[]
for i in range(0,len(sentencelist)):
    if not sentencelist[i] in uniqueword:#&lt;--- creates list of unique words
        uniqueword.append(sentencelist[i])
    else:
        i+=1



So many things done wrong here:


The else: i+=1 part is useless as i, being the iteration variable, is overwritten at each loop iteration. And since you perform no extra operation with it after incrementing it, you can drop these two lines.
range(0, len(sentencelist)): as 0 is the default start of a range, it is common practice to not specify it.
for i in range(len(sentencelist)): ... sentencelist[i] ...: this is a code smell. Unless you have a truly good reason to do that, it is faster, more Pythonic and more readable to iterate over the elements directly rather than using their indices. So use for element in sentencelist: ... element ... instead. If you really need the index, you can still use enumerate.
uniqueword = []; for element in sentencelist: if ...: uniqueword.append(element): again this is unefficient and rather un-Pythonic. Use a list-comprehension instead. Except you can't here because you need the partially built list to check for existence of the current element. But:


All in all, we could discuss about ways to improve this for loop, but that would be missing the point: you don't need it. There is a data structure in Python that is meant for holding unique elements: set. Whenever you try to insert an elemnt in a set, if the element is already present then nothing happens. Since you will need a list for its indexes latter on, you will need to convert it back:

    unique_words = list(set(sentence_list))


Compute the position of each word in the list of uniques


positionlist = []
for i in range(0,len(sentencelist)):
    positionlist.append(uniqueword.index(sentencelist[i]))#&lt;-- finds the index of the unique words in the sentence and appends them to positionlist



I already talked about it a bit above, but you should convert this loop into a list-comprehension; iterating over the elements rather than the indices:

    positions = [unique_words.index(word) for word in sentence_list]


You then have this code:


wordvals = []
end=False
im = 0
while True:
    try:
        wordvals.append(uniqueword[im])
        im+=1
    except:
        break



That I don't understand. You are copying uniqueword into wordvals. For one, this can be done using wordvals = list(uniqueword). For two, this is useless as you already have the list (uniqueword) and you don't modify any of these two lists. So just stick to using uniqueword and avoid unnecessary computation.

Write to files

First off, you should get into the habit of using with when you use open. You do it in your second snippet but not in the first. Second, closeing the file is done automaticaly by with so you don't need to do it manually aftewards.

The "%s\n" % item syntax is the old-style formatting syntax which is superseeded by the format function and the str.format method of strings. And the upcoming "f-strings" in Python 3.6 will make it even more easy to use. So write '{}\n'.format(item) instead for now and f'{item}\n' in Python 3.6. This has the added advantage that, if item is a tuple, your code won't crash.

But, in Python 3 (which I guess you are using, judging by your print calls), you can use the print function to write into files and use its full potential of features: automatic conversion of anything into a string and newlines at the end of each call:

    with open(filename_indexes, 'w') as f:
        for position in positions:
            print(position, file=f)
    with open(filename_uniques, 'w') as f:
        for word in unique_words:
            print(word, file=f)


Improving (again)

Now that we reduced the visual clutter of the function down to 9 lines, let's focus on the complexity of the algorithm.

Splitting the text and getting the unique words out of it can't really be reduced. But when you build the list of indices, you perform an \$O(n)\$ lookup into unique_words for each word in sentence_list: you end up with an overall complexity of \$O(n^2)\$. You can do better.

You don't really need the indices of the words. All you need is a unique value associated to each unique word. So you will need a data structure to hold such association and a way to generate enough unique values. You already know how to generate n unique values: range(n). And to associate one of them to each word, you will need a dictionary. Also the zip function will help in the process of making pairs:

unique_words = set(sentence_list)
lookup_table = dict(zip(unique_words, range(len(unique_words))))


That way, you can replace unique_words.index(word) which is \$O(n)\$ by lookup_table[word] which is \$O(1)\$. Unfortunately, this means that you will have to sort the words in lookup_table by their associated value if you want to be able to reconstruct the text properly. Luckily, sorted, the standard function to sort elements of a container, can help us doing so. It accepts a key parameter which is a function that will be called with each value that should return its "index" in the resulting sorted sequence. Here we’ll make use of the get method on dictionaries that return the element associated to the given value. In the case of lookup table, it will return the unique number associated to each word:

for word in sorted(lookup_table, key=lookup_table.get):
    print(word, file=f)


The whole code for the text to file transformation can thus become:

def split_sentence(text, filename_indexes, filename_uniques):
    words = text.split()
    uniques = set(words)
    lookup_table = dict(zip(uniques, range(len(uniques))))
    positions = [lookup_table[word] for word in words]

    with open(filename_indexes, 'w') as f:
        for position in positions:
            print(position, file=f)
    with open(filename_uniques, 'w') as f:
        for word in sorted(lookup_table, key=lookup_table.get):
            print(word, file=f)


File to text transformation

First, the line wv=f.read().splitlines() is underindented and cannot possibly work as the file will be closed by the time you try to read it.

Second, f.read().splitlines() is neat as it removes newlines characters at the end of each line but it has the disadvantage of copying the file in memory twice: once when reading it, and once when building the list of lines. You can achieve pretty much the same using a list-comprehension and half the amount of memory. List-comprehensions also have the advantage that you can perform any operation while building the list, such as converting a string into a number:

def build_sentence(filename_indexes, filename_uniques):
    with open(filename_indexes) as f:
        positions = [int(line) for line in f]
    with open(filename_uniques) as f:
        words = [line.strip() for line in f]

    for index in positions:
        print(words[index], '', end='')


But you can go further and transform the number into a word as well; granted that you already have your list of words:

def build_sentence(filename_indexes, filename_uniques):
    with open(filename_uniques) as f:
        words = [line.strip() for line in f]
    with open(filename_indexes) as f:
        sentence = [words[int(line)] for line in f]

    for word in sentence:
        print(word, '', end='')


Lastly, you shouldn't compute a value and print it in the same function: return the value, if the caller want it printed, it will print it. But if the responsibility of your functions is to compute and return a value only, it is much more reusable than if they also tried to print it.

So you will need to find a way to build a string out of each word in sentence by inserting a space between each of them. You could try to do it in a loop, starting with an empty list and concatenating words and spaces. But again, that would be terribly inneficient and completely un-Pythonic. Instead use str.join(iterable) that will do the exact same thing: concatenate every elements in iterable and inserting str between each. Here you would need ' '.join(sentence).

But you can do even better by not creating the sentence list in the first place and let join combine the words and the spaces as the lines are read from the file:

def build_sentence(filename_indexes, filename_uniques):
    with open(filename_uniques) as f:
        words = [line.strip() for line in f]
    with open(filename_indexes) as f:
        return ' '.join(words[int(line)] for line in f)