Is Overfitting always bad? I have a data set of total 8000 sound samples. These are the results of my multi layer neural network, binary classifier:

Precision: [0.95 0.96]
Recall: [0.96 0.95]
F-Score: [0.95 0.95]
S: [1217. 1254.]
Accuracy training 1.0
Accuracy Test 0.95


I am happy with the test accuracy, precision and recall. But the 1.0 accuracy bugs me since its overfitting to the training-set. Is this a bad thing even the test-set accuracy is satisfactory?

Here below I load the car sound features(1) and other types of sounds features(0) from disk and assign the labels. I double checked and I dont add labels as a feature while I process and extract features from audio. sets are cut to be equal to 4100 samples each

car_features =np.load('car_features_final.npy')
car_labels =np.ones(len(car_features),dtype=int)
shuffle(car_features, random_state=12)

other_features = np.load('other_features.npy')
other_features=list(islice(shuffle(other_features, random_state=12),4100))
other_labels = np.zeros(len(other_features),dtype=int)
shuffle(other_features, random_state=12)

all_features = np.append(car_features, other_features, axis=0)
all_labels = np.append(car_labels,other_labels, axis=0)

X_train, X_test, y_train, y_test = train_test_split(all_features, all_labels, test_size=0.30, random_state=42)

clf = MLPClassifier(activation='relu', solver='adam', alpha= 0.1, hidden_layer_sizes=(300, 300, 300, 100), random_state=1, max_iter=500)
clf.fit(X_train, y_train)

y_predicted_test = clf.predict(X_test)
y_predicted_train = clf.predict(X_train)
p,r,f,s = precision_recall_fscore_support(y_test, y_predicted_test)
It is overfitting if you have an accuracy on training of 100%, but the test accuracy would be 5%. That's overfitting.

In your case, there is a good match between training and testing accuracy. To check for any sampling bias, do a K-fold cross validation with the same hyper-parameters.