Overfit from the loss curve
I'll go through your question one by one:




  My initial assumption was that the point came at around epoch 28 since the validation error almost remains constant and then increases ever so slightly. However, I still wanted to know if this is fine or the model is indeed overfitting.


Commond knowledge from DS/ML handbooks says you're right. You should keep training until validation Loss doesn't start going up. Please keep in mind that some overfitting is inevitable (a good model is not one that eliminates overfitting, that would be impossible, but a model that is able to keep it at bay).




  Another concern I have is that the training and validation curves are very very close to each other. Is this an expected behavior?


You should keep training even after validation loss gets higher than training loss, and stop only when validation loss starts growing. Ideally, you should have a very slight overfitting, with validation set just above training, and its slope is flat. That means: I can't train my model more than that without making things worse.