title: Implement Siamese Network in Keras Issue
body: <p>I was trying to implement a Siamese Network with Keras, I have two input data,</p>
  
  <p>X1 shape: (10000, 52)
  X2 shape: (10000, 600)</p>
  
  <p>Each sample in X1 is assumed similar to sample in X2. For example: X1[0] is similar to X2[0]. Thus, I used the "cosine similarity" when two inputs are merged. I was trying to use the following code:</p>
  
  <pre><code>def contrastive_loss(y_true, y_pred):
  '''Contrastive loss from Hadsell-et-al.'06
  http://yann.lecun.com/exdb/publis/pdf/hadsell-chopra-lecun-06.pdf
  '''
  
  margin = 1
  return K.mean(y_true * K.square(y_pred) + (1 - y_true) * K.square(K.maximum(margin - y_pred, 0)))
  
  left_branch = Sequential()
  left_branch.add(Embedding(1000, 32, input_length=52))
  left_branch.add(LSTM(64))
  left_branch.add(Dense(256))
  
  right_branch = Sequential()
  right_branch.add(Embedding(1000, 32, input_length=600))
  right_branch.add(LSTM(64))
  right_branch.add(Dense(256))
  
  merged = Merge([left_branch, right_branch], mode='cos',dot_axes=1)
  
  final_model = Sequential()
  final_model.add(merged)
  final_model.summary()
  
  x1 = np.random.randint(1000, size=(10000, 52))
  x2 = np.random.randint(1000, size=(10000, 600))
  y = np.ones((10000,),dtype=np.int)
  
  final_model.compile(optimizer='adam', loss=contrastive_loss)
  
  final_model.fit([x1, x2], y, nb_epoch=2, batch_size=32)
  
  getoutput_t = K.function([merged.layers[0].layers[0].input], [merged.layers[0].layers[2].output])
  
  getoutput_d = K.function([merged.layers[1].layers[0].input], [merged.layers[1].layers[2].output])
  
  t1 = getoutput_t([x1])[0]
  
  t2 = getoutput_d([x2])[0]
  </code></pre>
  
  <p>My issue is: after training, t1[0] was very similar to t1[1], but very different from t2[0]. I checked the output of t1 and t2, the generated output were strange:</p>
  
  <p>t1:</p>
  
  <p>array([[ -3.48182112e-01, 6.57344190e-03, -6.75882818e-03,
  4.11706511e-03, -2.41146213e-03, -1.14028137e-02,
  2.49683809e+00, -9.19755269e-03, -2.37327255e-02....]</p>
  
  <p>[ -3.48182231e-01, 6.57369522e-03, -6.75880583e-03,
  4.11719829e-03, -2.41167075e-03, -1.14026833e-02,
  2.49683785e+00, -9.19752941e-03, -2.37324722e-02,...]
  .......]])</p>
  
  <p>The t1[0] was almost same to t1[1], same issue also was found in t2.</p>
  
  <p>Thus, I have to concerns:</p>
  
  <p>Is there any issue in my Siamese model? (e.g. loss function or layer design)</p>
  
  <p>I don't have negative sampling so far, thus, the target values are "1", is this the issue?</p>