Why does my loss value start at approximately -10,000 and my accuracy not improve? I'm developing a multi-label classifier using the Keras library, but I am stuck with a relatively low accuracy of about 2% and my loss value per epoch is around -10,000 with little change between epoch. I'm relatively new to Keras but need to use it for my university work, and I am wondering what is going wrong.

My model currently looks as follows:

model = Sequential()
model.add(Dense(1024, activation='relu', input_shape=X.shape[1:]))
model.add(Dropout(0.2))
model.add(Dense(512, activation='relu'))
model.add(Dropout(0.2))
model.add(Dense(3, activation='sigmoid'))

sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
model.compile(loss='binary_crossentropy', optimizer=sgd, metrics=['accuracy'])

model.fit(X, y, batch_size=20, epochs=100)


In this case, the input_shape[1:] is 4096, a flattened feature array and the multi-label I'm trying to classify has a first value that ranges from 1 - 163, the second value ranges from 1 - 2000, and the third value ranges from 2007 - 2014.

The issue that I am having is while running my model I get a result that does not change during each epoch as is shown below:



I don't know what I'm doing wrong, I'll include how the features and the labels look below in respective order: 

[['0.0' ' 0.0' ' 0.0' ..., ' 5.7333' ' 0.0' ' 2.58643']
 ['0.0' ' 3.12623' ' 0.0' ..., ' 0.0' ' 0.0' ' 2.93147']
 ['0.0' ' 0.0' ' 0.0' ..., ' 7.49419' ' 0.0' ' 1.55746']
 ['0.0' ' 0.0' ' 0.0' ..., ' 0.0' ' 0.0' ' 0.3666']
 ['0.0' ' 4.67996' ' 0.0' ..., ' 0.0' ' 0.0' ' 0.0']]

[['78' '1' '2010']
 ['78' '1' '2010']
 ['78' '1' '2014']
 ['78' '1' '2013']
 ['78' '1' '2012']]


I'm at a loss of what to try and as I have mentioned, I'm new to Keras and deep learning so any help will be greatly appreciated.
You are using the sigmoid activation function on the output layer that squashes the values between 0 and 1, but you are setting the targets outside of that range. I suggest scaling your outputs between 0 and 1 for each output unit. By the way, using a library like conx that is built on Keras would give you a warning about this very issue. For more, see: http://conx.readthedocs.io/en/latest/Getting%20Started%20with%20conx.html#A-Simple-Network