How to decrease the gap between training and validation error? I have been executing an open-source Text-to-speech system Ossian. It uses feed forward DNNs for it's acoustic modeling. The error graph I've got after running the acoustic model looks like this:

Here are some relevant information:


Size of Data: 7 hours of speech data (4000 sentences) 
Some hyper-parameters: 


batch_size       : 128 
training_epochs  : 15 
L2_regularization: 0.003 



Can anyone point me to the directions to improve this model? I'm assuming it is suffering from over-fitting problem? What should I do to avoid this? Increasing data? Or changing batch-size/epochs/regularization parameters? Thanks in advance.
As Learning is a mess points out, more information would help make useful suggestions. What is the size of the model (in terms of the trainable parameters)? Also, how much training data do you have?
However, you can try these: 
a. Dropouts - this a regularization method, it is a kind of proxy for ensemble learning. Not sure if you are using this.
b. Smaller Batches: 8/16/32 - Empirically it has been found that smaller batches find better minimums in the sense of generalization on the test/validation sets. See https://arxiv.org/pdf/1609.04836.pdf. 
c. Holding the number of parameters the same, deeper models (more layers) have been found to generalize better. See https://arxiv.org/abs/1706.10239.