Improving a basic neural network with only 3 input columns I am just getting started with neural networks, and I was hoping someone could shed some light on an issue I keep facing with a model I am working on. I have a dataset that looks like this:
df:

x1     x2    x3      y1
70     3     0.25    0.4
80     4     0.5     1.0
70     4     1.25    0.5
...    ...    ...     ...

...with a shape of (400, 4)
My objective is to train a deep learning model to predict the y1 value using the x1, x2, and x3 inputs.
Here is my code so far adapted from this source:
import pandas
from keras.models import Sequential
from keras.layers import Dense
from keras.wrappers.scikit_learn import KerasRegressor
from sklearn.model_selection import cross_val_score
from sklearn.model_selection import KFold
from sklearn.preprocessing import StandardScaler
from sklearn.pipeline import Pipeline

# define base model
def baseline_model():
    # create model
    model = Sequential()
    model.add(Dense(3, input_dim=3, kernel_initializer='normal', activation='sigmoid'))
    model.add(Dense(1, kernel_initializer='normal'))
    # Compile model
    model.compile(loss='mean_absolute_percentage_error', optimizer='sgd')
    return model

kfold = KFold(n_splits=10)

estimator = KerasRegressor(build_fn=baseline_model, epochs=100, batch_size=5, verbose=0)
results = cross_val_score(estimator, X_train, y_train, cv=kfold)
print(&quot;Results: %.2f (%.2f) MSE&quot; % (results.mean(), results.std()))

However, my results are a little unusual:
Results: -122.97 (87.82) MSE
y_pred = model.predict(X_test)
y_pred

array([[0.9994005],
[0.9994005],
[0.9994005],
[0.9994005], ...)
My main questions here are why are the final values always the same, in this case &quot;0.9994005&quot;, and what can I do to improve this model? What the main things you look for in deep learning models that direct you in the right path?
There is no point in looking at the prediction at this point. Your model has not learnt the data yet.
Suggestions -

When copy from a source. Try copying everything first. Run it and then start tweaking for learning Or to map with your own need. You have changed a few things as compared to what Jason has in his post

Use the wisdom of researchers. So, you can keep a few things as default -
     1. Optimizer - Adam
     2. Activation - Relu for Hidden layer
     3. Loss - MSE(Regression), Cross-entropy(Classification). Beware of Cros-entropy variance.
     4. Activation for Output layer - Linear(Regression), Sigmoid/Softmax(Classification)
     3. Have Batch normalization layer when you have a deep network.

Try to know why behind every point that I have mentioned. It's will be an easy search on the Internet.

A good place to start can be - Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow, 2nd Edition, by Aurélien Géron
Further reading -
     Practical Recommendations for Gradient-Based Training of Deep Architectures, by Yoshua Bengio
    Andrej Karpathy blog
    Raúl Gómez blog