Siamese network return same loss value while training I have ~9k training samples, the same pair vectors are labeled as 0 and not same pair samples are labeled as 1.

I trained it for 100 epochs for the first 3 epochs the loss value is fluctuating, then it returns the same loss value until the end.

Base network architecture:

def create_base_network():

    a = 'tanh'
    model = Sequential()
    model.add(Dense(self.INPUT_DIM, input_shape=(self.INPUT_DIM, ), activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(900, activation=a))
    model.add(Dense(1000, activation=a))
    model.add(Dense(5000, activation=a))
    model.add(Dense(1000, activation=a))
    model.add(Dense(900, activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(600, activation=a))
    model.add(Dense(self.INPUT_DIM, activation=a))
    return model


Euclidean distance:

K.sqrt(K.maximum(K.sum(K.square(x - y), axis=0, keepdims=True), K.epsilon()))


Initilization model:

    base_network = self._create_base_network()
    input_a = Input(shape=(self.INPUT_DIM,))
    input_b = Input(shape=(self.INPUT_DIM,))

    processed_a = base_network(input_a)
    processed_b = base_network(input_b)

    distance = Lambda(self._euclidean_distance, output_shape=self._dist_output_shape)([processed_a, processed_b])
    model = Model(inputs=[input_a, input_b], outputs=distance)


Compilation of model:

model.compile(loss='mse',
          metrics=['mse'],
          optimizer=optimizers.Adam()
          )


Training the model:

model.fit([self.train_d['vec1'], self.train_d['vec2']], self.train_d['label'], batch_size=128, epochs=self.args.epochs, shuffle=True)


Log statements:

Epoch 1/100
9544/9544 [==============================] - 6s 594us/step - loss: 0.4556 - mean_squared_error: 0.4556
Epoch 2/100
9544/9544 [==============================] - 4s 470us/step - loss: 1.0693 - mean_squared_error: 1.0693
Epoch 3/100
9544/9544 [==============================] - 4s 464us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 4/100
9544/9544 [==============================] - 4s 465us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 5/100
9544/9544 [==============================] - 4s 461us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 99/100
9544/9544 [==============================] - 4s 470us/step - loss: 0.7328 - mean_squared_error: 0.7328
Epoch 100/100
9544/9544 [==============================] - 4s 470us/step - loss: 0.7328 - mean_squared_error: 0.7328


Can someone help me to find out what's the issue here?

Thanks in Advance.
Finally, I found a solution to this problem. I completely modified the architecture, Then I reduced the learning rate to 0.00005. Then the loss value of the model falls continuously.

Latest Architecture:

a = 'tanh'
model = Sequential()
model.add(Dense(self.INPUT_DIM, input_shape=(self.INPUT_DIM, ), 
activation=a))
model.add(Dense(600, activation=a))
model.add(Dense(self.INPUT_DIM, activation=a))