Keras: Very high loss for Autoencoder I am trying to implement an autoencoder for prediction of multiple labels using Keras. This is a snippet:
input = Input(shape=(768,))
hidden1 = Dense(512, activation='relu')(input)
compressed = Dense(256, activation='relu', activity_regularizer=l1(10e-6))(hidden1) 
hidden2 = Dense(512, activation='relu')(compressed)
output = Dense(768, activation='sigmoid')(hidden2) # sigmoid is used because output of autoencoder is a set of probabilities

model = Model(input, output)
model.compile(optimizer='adam', loss='categorical_crossentropy') # categorical_crossentropy is used because it's prediction of multiple labels
history = model.fit(x_train, x_train, epochs=100, batch_size=50, validation_split=0.2)

I ran this in Jupyter Notebook (CPU) and I am getting loss and validation loss as:
loss: 193.8085 - val_loss: 439.7132
but when I ran it in Google Colab (GPU), I am getting very high loss and validation loss:
loss: 28383285849773932.0000 - val_loss: 26927464965996544.0000.
What could be the reason for this behavior?
You should not use the categorical cross-entropy loss, but the binary cross-entropy.
The categorical cross-entropy is meant for categorical probability distribution. In your scenario, this means that only one label would be active at the same time. If multiple labels can be active at the same time, then they are independent, and you need to compute the binary cross-entropy.