title: How to speed up the calculation of cross entropy loss in tf.nn.softmax_cross_entropy_with_logits()
body: <p>I would like to make a multi-label classification model (each example has multiple labels and the number of labels for each example are not fixed). For instance, example1 may have class labels "X", "Y", while example2 have class labels "X", "Y", and "Z". My goal is calculate the cross entropy loss for this multi-label classification model.</p>
  
  <p>My first solution is to manually create the dense one-hot representation of target class and calculate the loss. However, this solution is slow when my vocabulary is of size O(10K). I wonder whether there is a more efficient way to do this?</p>
  
  <p>[Update to provide the relevant code]</p>
  
  <pre><code>## During the data input phrase
  def input_fn():
      ... 
      ## target_ids is a sparseTensor
      target_ids = lookup_table.lookup(target_label_strings)
  
      ## change the dense_shape 
      st2 = tf.SparseTensor(indices=target_ids.indices,
                            values=target_ids.values,
                            dense_shape=[batch_size,vocab_size])
  
      ## Convert to dense Tensor
      st2_ordered = tf.sparse_reorder(st2)
      dt = tf.sparse_tensor_to_dense(st2_ordered)
  
      ## Row normalization
      dt_float = tf.cast(dt, tf.float32)
      dt_float = tf.add(dt_float, tf.constant(1e-10))
  
      dt_row_norm = tf.reduce_sum(dt_float, axis=1)
      target["target_ids"] = dt_float / tf.reshape(dt_row_norm, (-1,1))
  
      return feature_map, target
  
  ## Model training
  def get_loss_fn(self, target, weights, mode):
      ...
      ## the self.final_logit is the final output layer
      loss = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(
        labels=target["target_ids"], logits=self.final_logit))
      ...
  </code></pre>
  
  <p>Thanks.</p>