**Question**

What are good reference articles/blogs/tutorials to learn how to intepret learning curves for deep convolutional neural networks?

**Background**
I am trying to apply convolutional neural networks (CNN) for vessel segmentation (specifically to determine whether or not the center pixel of an image patch is on a vessel) using caffe. 

I have about 225000 training images (~50% positive) and 225000 (~50% positive) testing/validation images.

My input images are of size 65 x 65. I have four convolutional layers (48x6x6, 48x5x5, 48x4x4, 48x2x2) each followed by 2x2 max-pooling layers , one fully connected layer of 50 neurons, and a final scoring layer with 2 neurons. My training batch size is 256 and my testing batch size is 100. 

I am using a stochastic gradient descent optimizer (SGD) and an inverse decay learning rate policy. Below are my [caffe solver](http://caffe.berkeleyvision.org/tutorial/solver.html) parameters:

* **type**: "SGD"
* **base\_lr**: 0.01
* **lr\_policy**: "inv"
* **gamma**: 0.1
* **power**: 0.75
* **momentum**: 0.9
* **weight\_decay**: 0.0005

Below is the learning curve i am getting:

[![enter image description here](https://i.stack.imgur.com/ZNI8Y.png)](https://i.stack.imgur.com/ZNI8Y.png)

I am using the cross entropy classification loss or multinomial logistic loss (see [here](http://caffe.berkeleyvision.org/doxygen/classcaffe_1_1SoftmaxWithLossLayer.html)). 

I would like to hear how people interpret this learning curve and what parameters they would change to try and improve the test accuracy. 

* The training loss is decreasing but decreasing very slowly. Could it mean that my learning rate is low?
* On the contrary i see that the test loss decreases quickly at first and then slows down. Could this mean that my learning rate was high and it got stuck in the local minimum?
* Also the test accuracy has stabilized and stopped increasing too soon. Could this mean that i have to try and increase my model capacity or decrease my regularization?

In general, what would help me and also probably others is if someone could point out a reference article/book/blog-post that delves deeply into the interpretation of such learning curves with many example cases.

I found this [blog post](http://lamda.nju.edu.cn/weixs/project/CNNTricks/CNNTricks.html) which was very helpful but there is not much about the interpretation of learning curves (atleast not to my satisfaction).