Machine Translation using seq2seq model I am trying to perform Hindi To English translation using s2s model, following - https://chunml.github.io/ChunML.github.io/project/Sequence-To-Sequence/

I have used https://github.com/karimkhanp/Seq2Seq/tree/master/data dataset which contains 37726 sentences for training.

Training took around 44 hours on my 8GB machine. I considered 3 layers, 10 epoch, 20000 vocap size

Other parameters

ap.add_argument('-max_len', type=int, default=200)
ap.add_argument('-vocab_size', type=int, default=20000)
ap.add_argument('-batch_size', type=int, default=100)
ap.add_argument('-layer_num', type=int, default=3)
ap.add_argument('-hidden_dim', type=int, default=1000)
ap.add_argument('-nb_epoch', type=int, default=10)
ap.add_argument('-mode', default='train') 


But now when I test using hindi sentence, it give UNK for all words. Though I take same sentence for testing as in training even it says UNK in results.

Test sentences:

डेली हिन्दी न्यूज - बुंदेलखंड का प्रथम अन्तरजालीय स्थल
मैं भारत से प्यार करता हूँ
ज़बूर जो कि दाउद को प्रदान की गयी
प्रशासनिक विभाजन
वे इस देश के प्रथम UNK -LRB- अफ्रीकी UNK -RRB-
नेपाली विदेश
वेल्श खिलाड़ी इंग्लैंड के लिए खेलने के लिए पात्र हैं
उनके बड़े भाई अजीत तेंडुलकर ने उन्हें खेलने के लिये प्रोत्साहित किया था ।
फिर एक मिनट के बाद किताब छत की ओर उछालकर उन्होंने कहा - चलो ।


Result :

of of of
of of of
the of of of

lrb lrb rrb rrb

the the of of of
the the of of of of
the the the of of of of


I could not understand the issue behind this result. I have used some sentences which are same as training sentences. Atleast answer should be correct for them.

Did I do anything wrong? I really appreciate any help

Answer - 

if MODE == 'train':
    k_start = 1
    t1 = time()
    # If any trained weight was found, then load them into the model
    if len(saved_weights) != 0:
        print('[INFO] Saved weights found, loading...')
        epoch = saved_weights[saved_weights.rfind('_')+1:saved_weights.rfind('.')]
        model.load_weights(saved_weights)
        k_start = int(epoch) + 1

    i_end = 0
    for k in range(k_start, NB_EPOCH+1):
        # Shuffling the training data every epoch to avoid local minima
        indices = np.arange(len(X))
        np.random.shuffle(indices)
        X = X[indices]
        y = y[indices]

        # Training 1000 sequences at a time
        for i in range(0, len(X), 1000):
            if i + 1000 &gt;= len(X):
                i_end = len(X)
            else:
                i_end = i + 1000
            y_sequences = process_data(y[i:i_end], y_max_len, y_word_to_ix)

            print('[INFO] Training model: epoch {}th {}/{} samples'.format(k, i, len(X)))
            model.fit(X[i:i_end], y_sequences, batch_size=BATCH_SIZE, nb_epoch=1, verbose=2)
        model.save_weights('checkpoint_epoch_{}.hdf5'.format(k))
    print("Time taken to train the data in hour=&gt;", (time()-t1)/3600)

# Performing test if we chose test mode
else:
    # Only performing test if there is any saved weights
    if len(saved_weights) == 0:
        print("The network hasn't been trained! Program will exit...")
        sys.exit()
    else:
        # import pdb
        # pdb.set_trace()
        X_test = load_test_data('test', X_word_to_ix, MAX_LEN)
        X_test = pad_sequences(X_test, maxlen=X_max_len, dtype='int32')
        model.load_weights(saved_weights)
        # print(X_test)
        # print(model.predict(X_test))
        predictions = np.argmax(model.predict(X_test), axis=2)
        # print(predictions)
        # print(y_ix_to_word)
        sequences = []
        for prediction in predictions:
            sequence = ' '.join([y_ix_to_word[index] for index in prediction if index &gt; 0])
            print(sequence)
            sequences.append(sequence)
Clearly, the optimization of your loss function led to it only predicting the most common words in English. The probability of other words are obviously small. 
I am no expert, but I would check three things. 1. How long does it take for the loss to converge? With the output you show it is likely it does not take a long time. 2. Maybe try to filter out so you have different words in the sentences. 3. Check if it can help to optimize another loss function which would have a bias towards more diverse sentences.
I hope this can give you a few ideas at least.