title: Implementing Deep Learning architecture with a given dataset
body: <p>I am quite new in caffe and in deep learning. I am just trying to implement deep learning architecture. 
  <a href="https://i.stack.imgur.com/B7Cmq.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/B7Cmq.png" alt="Architecture"></a></p>
  
  <p>This is the architecture that i am trying to implement . The architecture and Parse27k dataset was created and constructed by <strong><em>Computer Vision Group at the Visual Computing Institute, RWTH Aachen University</em></strong> .</p>
  
  <p>Below you can see  my  models that i need to improve : </p>
  
  <p><em>Train_val.prototxt</em></p>
  
  <pre><code>name: "Parse27"
  layer {
    name: "data"
    type: "HDF5Data"
    top: "crops"
    top: "labels"
    include {
      phase: TRAIN
    }
  
    hdf5_data_param {
      source: "/home/nail/caffe/caffe/examples/hdf5_classification/data/train.txt"
      batch_size: 256
    }
  }
  layer {
    name: "data"
    type: "HDF5Data"
    top: "crops"
    top: "labels"
    include {
      phase: TEST
    }
    hdf5_data_param {
      source: "/home/nail/caffe/caffe/examples/hdf5_classification/data/test.txt"
      batch_size: 256
    }
  }
  layer {
    name: "conv1"
    type: "Convolution"
    bottom: "crops"
    top: "conv1"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    convolution_param {
      num_output: 96
      kernel_size: 11
      stride: 4
      weight_filler {
        type: "gaussian"
        std: 0.01
      }
      bias_filler {
        type: "constant"
        value: 0
      }
    }
  }
  layer {
    name: "relu1"
    type: "ReLU"
    bottom: "conv1"
    top: "conv1"
  }
  layer {
    name: "pool1"
    type: "Pooling"
    bottom: "conv1"
    top: "pool1"
    pooling_param {
      pool: MAX
      kernel_size: 3
      stride: 2
    }
  }
  layer {
    name: "norm1"
    type: "LRN"
    bottom: "pool1"
    top: "norm1"
    lrn_param {
      local_size: 5
      alpha: 0.0001
      beta: 0.75
    }
  }
  layer {
    name: "conv2"
    type: "Convolution"
    bottom: "norm1"
    top: "conv2"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    convolution_param {
      num_output: 256
      pad: 2
      kernel_size: 5
      group: 2
      weight_filler {
        type: "gaussian"
        std: 0.01
      }
      bias_filler {
        type: "constant"
        value: 1
      }
    }
  }
  layer {
    name: "relu2"
    type: "ReLU"
    bottom: "conv2"
    top: "conv2"
  }
  layer {
    name: "pool2"
    type: "Pooling"
    bottom: "conv2"
    top: "pool2"
    pooling_param {
      pool: MAX
      kernel_size: 3
      stride: 2
    }
  }
  layer {
    name: "norm2"
    type: "LRN"
    bottom: "pool2"
    top: "norm2"
    lrn_param {
      local_size: 5
      alpha: 0.0001
      beta: 0.75
    }
  }
  layer {
    name: "conv3"
    type: "Convolution"
    bottom: "norm2"
    top: "conv3"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    convolution_param {
      num_output: 384
      pad: 1
      kernel_size: 3
      weight_filler {
        type: "gaussian"
        std: 0.01
      }
      bias_filler {
        type: "constant"
        value: 0
      }
    }
  }
  layer {
    name: "relu3"
    type: "ReLU"
    bottom: "conv3"
    top: "conv3"
  }
  layer {
    name: "conv4"
    type: "Convolution"
    bottom: "conv3"
    top: "conv4"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    convolution_param {
      num_output: 384
      pad: 1
      kernel_size: 3
      group: 2
      weight_filler {
        type: "gaussian"
        std: 0.01
      }
      bias_filler {
        type: "constant"
        value: 1
      }
    }
  }
  layer {
    name: "relu4"
    type: "ReLU"
    bottom: "conv4"
    top: "conv4"
  }
  layer {
    name: "conv5"
    type: "Convolution"
    bottom: "conv4"
    top: "conv5"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    convolution_param {
      num_output: 256
      pad: 1
      kernel_size: 3
      group: 2
      weight_filler {
        type: "gaussian"
        std: 0.01
      }
      bias_filler {
        type: "constant"
        value: 1
      }
    }
  }
  layer {
    name: "relu5"
    type: "ReLU"
    bottom: "conv5"
    top: "conv5"
  }
  layer {
    name: "pool5"
    type: "Pooling"
    bottom: "conv5"
    top: "pool5"
    pooling_param {
      pool: MAX
      kernel_size: 3
      stride: 2
    }
  }
  layer {
    name: "fc6"
    type: "InnerProduct"
    bottom: "pool5"
    top: "fc6"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    inner_product_param {
      num_output: 4096
      weight_filler {
        type: "gaussian"
        std: 0.005
      }
      bias_filler {
        type: "constant"
        value: 1
      }
    }
  }
  layer {
    name: "relu6"
    type: "ReLU"
    bottom: "fc6"
    top: "fc6"
  }
  layer {
    name: "drop6"
    type: "Dropout"
    bottom: "fc6"
    top: "fc6"
    dropout_param {
      dropout_ratio: 0.5
    }
  }
  layer {
    name: "fc7"
    type: "InnerProduct"
    bottom: "fc6"
    top: "fc7"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    inner_product_param {
      num_output: 4096
      weight_filler {
        type: "gaussian"
        std: 0.005
      }
      bias_filler {
        type: "constant"
        value: 1
      }
    }
  }
  layer {
    name: "relu7"
    type: "ReLU"
    bottom: "fc7"
    top: "fc7"
  }
  layer {
    name: "drop7"
    type: "Dropout"
    bottom: "fc7"
    top: "fc7"
    dropout_param {
      dropout_ratio: 0.5
    }
  }
  layer {
    name: "fc8"
    type: "InnerProduct"
    bottom: "fc7"
    top: "fc8"
    param {
      lr_mult: 1
      decay_mult: 1
    }
    param {
      lr_mult: 2
      decay_mult: 0
    }
    inner_product_param {
      num_output: 1000
      weight_filler {
        type: "gaussian"
        std: 0.01
      }
      bias_filler {
        type: "constant"
        value: 0
      }
    }
  }
  layer {
    name: "accuracy"
    type: "Accuracy"
    bottom: "fc8"
    bottom: "labels"
    top: "accuracy"
    include {
      phase: TEST
    }
  }
  layer {
    name: "loss"
    type: "SoftmaxWithLoss"
    bottom: "fc8"
    bottom: "labels"  
    top: "loss"
  }
  </code></pre>
  
  <p><em>Solver.prototxt</em></p>
  
  <pre><code>net: "models/Parse27/train_val.prototxt"
  test_iter: 1000
  test_interval: 1000
  base_lr: 0.01
  lr_policy: "step"
  gamma: 0.1
  stepsize: 100000
  display: 20
  max_iter: 450000
  momentum: 0.9
  weight_decay: 0.0005
  snapshot: 10000
  snapshot_prefix: "models/Parse27/Parse27_train"
  solver_mode: GPU
  </code></pre>
  
  <p>I have 2 main hardships in implementing this architecture. </p>
  
  <ol>
  <li><p>As seen above, my model does not include custom-loss layer. My model is almost caffeNet architecture. But i should replace my last layer which is inside the red box with custom loss layer (green box).</p></li>
  <li><p>My train dataset has following structure.</p></li>
  </ol>
  
  <blockquote>
  <pre><code>crops       Dataset {27482, 3, 128, 192}
  labels      Dataset {27482, 12}
  mean        Dataset {3, 128, 192}
  pids        Dataset {27482}
  </code></pre>
  </blockquote>
  
  <p>As seen here number of rows(examples) in crops and in labels are same 27482. However i have 12 columns in my label datasets. And my model  works when there is only 1 label. How i can make it to train for all labels? </p>
  
  <p>My model in the Train_val.prototxt is looking like this for now: </p>
  
  <p><a href="https://i.stack.imgur.com/7ZpgC.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/7ZpgC.png" alt="enter image description here"></a></p>
  
  <p>Any kind of help or suggestion will be highly appreciated. </p>