Image features (produced by VGG19) do not properly train an ANN in Keras I've used a VGG16 network to extract features from an image dataset, creating a features dataset using the following code:

model = VGG16(weights='imagenet', include_top=False)
x = 0

for folder in os.listdir(os.fsencode('training_cnn_images/')):
    folder = str(folder)[2:-1]
    directory = os.fsencode('training_cnn_images/' + folder)
    dn = str(directory)[2:-1]
    for file in os.listdir(directory):
        fn = str(file)[2:-1]
        img = image.load_img(dn + '/' + fn, target_size=(224, 224))
        img_data = image.img_to_array(img)
        img_data = np.expand_dims(img_data, axis=0)
        img_data = preprocess_input(img_data)
        vgg16_feature = model.predict(img_data)
        with open(('training_cnn_features' + dn[19:] + '/' + fn)[:-4] + '.pkl', 'wb') as f:
            pkl.dump(vgg16_feature, f)
        x += 1
        printProgress(x, 1895, 'Generating Features:')


I simply feed an image to VGG16, and the resultant output is used as the extracted features.
Now that I have a .pkl dataset containing the features for all the images of the dataset, I wanted to create an ANN that takes these features as an input and classifies the images (technically their features) into two categories.

So, I wrote this code to do just that:

inney = [None] * 315
outey = [None] * 315

x = 0
for file in os.listdir(os.fsencode('training_cnn_features/n02085620-Chihuahua/')):
    fn = str(file)[2:-1]
    with open('training_cnn_features/n02085620-Chihuahua/' + fn, 'rb') as f:
       inney[x] = pkl.load(f).flatten()
       outey[x] = np.array([[0, 1]])
    x += 1
for file in os.listdir(os.fsencode('training_cnn_features/n02085782-Japanese_spaniel/')):
    fn = str(file)[2:-1]
    with open('training_cnn_features/n02085782-Japanese_spaniel/' + fn, 'rb') as f:
       inney[x] = pkl.load(f).flatten()
       outey[x] = np.array([[1, 0]])
    x += 1

inney = np.asarray(inney)
outey = np.asarray(outey)

model = Sequential()
model.add(Dense(1024, input_shape=(25088,), activation='relu'))
model.add(Dense(2, activation='softmax'))
model.compile(loss = "mean_squared_error", optimizer = optimizers.Adamax(lr=0.002, beta_1=0.9, beta_2=0.999, epsilon=None, decay=0.0))
model.fit(inney, outey, epochs=10, batch_size=4, verbose=1)
model.save('test.h5')


I had also created a test dataset by partitioning the original dataset into a training and test set. When I attempted to test the network, the network always seems to output [0, 1] or [*some very small value*, 1]. When I try with sigmoid, it flips, i.e, it becomes [1, 0].

A similar issue arises (one output is always one or very close to it while the other is always zero or very close to it) when I use inceptionV3.

Nothing seems to work. I've attempted adding the extra layers directly to the inception or VGG16 network, and freezing the non-classifier layers, but I face the same issue over and over again. The neural network just won't classify.

I've attempted using both sigmoid and softmax, but face the same issue in both cases.

Help would be very much appreciated.
Thanks.
After doing a bunch of searching and trial &amp; error, I found out that using the sigmoid activation function for all the layers and using a binary_crossentropy loss fixed the issue. The newly added hidden layers were too few and too small for the ReLU function to work.