Nested if-else within a for-loop using np.concatenate to create batches for a neural network I am having a for-loop in python 3.5. Inside the loop, I have multiple if-else statements in which I append some information. I am using this piece of code to create batches for a neural network built in Tensorflow.

Here is my code:

for generated_lemma, classified_lemma, token_hidden, token_hidden_generator in zip(
        tokens_generated,
        tokens_classified,
        token_dnn_output,
        token_generator_output
):
    if generated_lemma.replace('#', '|') not in self.lemmas_word2vec_model:
        if classified_lemma.replace('#', '|') not in self.lemmas_word2vec_model:
            if classified_lemma == '&lt;UNK&gt;':
                x.append(
                    np.concatenate([
                        self.embedding_unknown_lemma,
                        self.embedding_unknown_token,
                        token_hidden,
                        token_hidden_generator
                    ]
                    )
                )
            else:
                x.append(
                    np.concatenate([
                        self.embedding_unknown_lemma,
                        self.embedding_unknown_lemma,
                        token_hidden,
                        token_hidden_generator
                    ]
                    )
                )
        else:
            x.append(
                np.concatenate([
                    self.embedding_unknown_lemma,
                    self.lemmas_word2vec_model[classified_lemma.replace('#', '|')],
                    token_hidden,
                    token_hidden_generator
                ]
                )
            )
    elif classified_lemma.replace('#', '|') not in self.lemmas_word2vec_model:
        if classified_lemma == '&lt;UNK&gt;':
            x.append(
                np.concatenate([
                    self.lemmas_word2vec_model[generated_lemma.replace('#', '|')],
                    self.embedding_unknown_token,
                    token_hidden,
                    token_hidden_generator
                ]
                )
            )
        else:
            x.append(
                np.concatenate([
                    self.lemmas_word2vec_model[generated_lemma.replace('#', '|')],
                    self.embedding_unknown_lemma,
                    token_hidden,
                    token_hidden_generator
                ]
                )
            )
    else:
        x.append(
            np.concatenate([
                self.lemmas_word2vec_model[generated_lemma.replace('#', '|')],
                self.lemmas_word2vec_model[classified_lemma.replace('#', '|')],
                token_hidden,
                token_hidden_generator
            ]
            )
        )


My code works fine, although it is a bit slow. Hence, I would like to know if there was a more "pythonic" way to achieve the same thing?

My aim is to improve both style and performance.
Your code is coupling independent ideas, this is making your code at least twice as long as you need it to be, as you have to duplicate your if statements.
And so I refactored your code in the following way:


Move x.append(np.concatenate(...)) out of the if statements. This is as you're needlessly duplicating this across them all.
Make the array passed to (1) outside the ifs, this is as you always add token_hidden and token_hidden_generator.
Replace once. generated_lemma = generated_lemma.replace('#', '|') and you don't have to call replace again.
Move all the generated_lemma code into a single if-else. Leading to:

generated_lemma = generated_lemma.replace('#', '|')
if generated_lemma in self.lemmas_word2vec_model:
    gen_lemma = self.lemmas_word2vec_model[generated_lemma]
else:
    gen_lemma = self.embedding_unknown_lemma

Cleanup the rest of the ifs.


This got me:

for generated_lemma, classified_lemma, token_hidden, token_hidden_generator in zip(
    tokens_generated,
    tokens_classified,
    token_dnn_output,
    token_generator_output
):
    generated_lemma = generated_lemma.replace('#', '|')
    if generated_lemma in self.lemmas_word2vec_model:
        gen_lemma = self.lemmas_word2vec_model[generated_lemma]
    else:
        gen_lemma = self.embedding_unknown_lemma

    classified_lemma = classified_lemma.replace('#', '|')
    if classified_lemma in self.lemmas_word2vec_model:
        class_lemma = self.lemmas_word2vec_model[classified_lemma]
    elif classified_lemma == '&lt;UNK&gt;':
        class_lemma = self.embedding_unknown_token
    else:
        class_lemma = self.embedding_unknown_lemma

    x.append(np.concatenate([
        gen_lemma,
        class_lemma,
        token_hidden,
        token_hidden_generator
    ]))


This may be faster, due to (3), however, I doubt it'd have any serious speed-ups. I don't really know any of the data types, and so can't comment on whether you need np.concatenate, or a faster way to add to x.