Handle Unbalanced data
You have not specified that what neural network you are using but as comments, you should try to fit your data first. You have to try to find a model that learns your training data. For this purpose you don't have to increase the number of data, at least not at this stage. You should try to find a good model which suits your data. For this purpose you have to change the hyper-parameters of your neural network, e.g. number of layers or number of neurons in layers. You can take a look at here and here which the former can help you and the latter helps you understand the features learned by CNNs in case you are using them. 

For using F1 score in Keras I've not seen but you can implement it and pass it to compile method, take a look at here.