Large amount of Sigmoid outputs are ones and zeros I have Keras neural network for binary classification with final layer having one output with Sigmoid activation. I have noticed that large amount of output numbers are strictly one or zero (rather than between 0 and 1 as expected). What could be the reason for this? 

At first I thought maybe network is 100% sure about accuracy of these numbers, but noticed that some of these predictions are actually incorrect.

Edit:

Model:

model = tf.keras.models.Sequential()

model.add(tf.keras.layers.Dense(32, activation = tf.nn.relu, input_shape=(X_train.shape[1],)))

model.add(tf.keras.layers.Dense(64, activation = tf.nn.relu))

model.add(tf.keras.layers.Dense(32, activation = tf.nn.relu))

model.add(tf.keras.layers.Dense(1, activation = tf.nn.sigmoid))

model.compile(optimizer = tf.keras.optimizers.Adam(), loss = 'binary_crossentropy', metrics = ['accuracy', roc_auc])

model.fit(X_train, y_train, epochs=10, validation_data=(X_test, y_test))

y_pred = model.predict(X_test)

yhat = []

for pred in y_pred:
    if pred &gt;= 0.7:
        yhat.append(1)
    else:
        yhat.append(0)


Data consists of 8 columns (features) and 270 000 rows (9th column is 'y' column). Out of these 270,000 rows only 9% contained labels of class 1 (rest were zero), so I downsampled the data (just removed bunch of data with label 0), trained the model and then did the prediction on full data. I modified how ones and zeros were determined by Sigmoid though, I changed threshold from 0.5 to 0.7 (which was the ROC score I got on downsampled data)
Apparently the problem was that I forgot to normalize data. As I found out, sigmoid turns any input greater than 10 into 1 and inputs less than -10 into 0. Normalization solves this problem (as normalized values are between 0 and 1)