Overfitting after first epoch I am using convolutional neural networks (via Keras) as my model for facial expression recognition (55 subjects). My data set is quite hard and around 450k with 7 classes. I have balanced my training set per subject and per class label.

I implemented a very simple CNN architecture (with real-time data augmentation):

model = Sequential()
model.add(Convolution2D(32, 3, 3, border_mode=borderMode, init=initialization,  input_shape=(48, 48, 3)))
model.add(BatchNormalization())
model.add(PReLU())
model.add(MaxPooling2D(pool_size=(2, 2)))

model.add(Flatten())
model.add(Dense(256))
model.add(BatchNormalization())
model.add(PReLU())
model.add(Dropout(0.5))

model.add(Dense(nb_output))
model.add(Activation('softmax'))


After first epoch, my training loss decreases constantly while validation loss increases. Could overfitting happen that soon? Or is there a problem with my data being confusing? Should I also balance my testing set?
It is very unlikely for such huge dataset(450k ) to overfit after just one epoch.

Try to run the code for 20-30 epochs and see if you see any decrease in the validation set error (increase in validation accuracy.)

Try to change your learning rate if (unless you are using adaptive learning parameter)