Will my validation loss eventually go down?
When training neural network, you need to train many epochs in order for the model to learn enough to generalize. What is happening now the model is changing weights to fit the data that it observes (lower training loss) but does not generalize (as evidenced by high variance on validation data).
There are a couple of options:

Downsampling your data. Take only the most meaningful data to train.
Scale up training. Either use graphics processing units (GPUs) or distributed training.
Run training for longer.