I'm working on a disease classification dataset which has 25 features including the class attribute. It is a binary classification problem. The dataset has total 300 training instances.

I trained a feed-forward neural network with the following architecture:

```
model = Sequential()

model.add(Dense(100, input_dim=train_x.shape[1], activation='relu'))
model.add(Dense(50, activation='relu'))
model.add(Dense(25, activation='relu'))
model.add(Dense(1, activation='sigmoid'))
model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy'])

model.fit(train_x.values, train_y.values, epochs=50, batch_size=40, validation_split=.2, verbose=0)

```

The architecture has 3 hidden layers with 100, 50 and 25 hidden neurons in the layers.

After training, the model shows the following scores on the test set which includes 100 test instances.

```
100/100 [==============================] - 0s 94us/step
Test loss score: 0.02940008039586246
Test accuracy: 100.0

```

So, I've got 100% accuracy with this architecture.

The number of hidden layers and hidden neurons per layer was taken arbitrarily.

However, I have found something in an [article](https://www.heatonresearch.com/2017/06/01/hidden-layers.html) of Jeff Heaton that: 

[![enter image description here](https://i.stack.imgur.com/tHAz5.png)](https://i.stack.imgur.com/tHAz5.png)

and also found some rules about choosing the number of hidden neurons per layer: 

*1. The number of hidden neurons should be between the size of the input layer and the size of the output layer.*

*2. The number of hidden neurons should be 2/3 the size of the input layer, plus the size of the output layer.*

*3. The number of hidden neurons should be less than twice the size of the input layer.*

These three rules provide a starting point for you to consider.

Now my question is: 

1. Is my model architecture acceptable?
2. According to Heaton, we can have more than 2 hidden layers for sort of automatic feature engineering. So is using 3 layers okay or not?
3. Here, I've used 100, 50 and 25 neurons in the hidden layers arbitrarily. The output layer contains only 1 neuron as it is a binary classification. But according to the thumb rule, *the number of hidden neurons should be between the size of the input layer and the size of the output layer.* So, according to this rule, the number of hidden neurons for this dataset should be between 1-24 as there are 24 input features and the 1 output feature. So, using 100, 50, 25 hidden neurons is wrong?
4. For this dataset of 300 instances with 24 features and a class attribute of a binary class, what should be the number of hidden neurons? Is there any other empirical rule that I can follow?
5. Using 100, 50 and 25 hidden neurons yields a good result. So should I decrease the number of neurons? Why?