title: Word position prediction with keras
body: <p>I am facing the following problem. I need to create a model to predict the product groups from product title. For each word in a sentence I need to predict the position of a word marked as a product. My dataset looks like this:</p>
  <p>Input: product title (max 10 words), eg.</p>
  <ul>
  <li>“red adidas sneakers” (this is when there is one product in the product title) or</li>
  <li>“red adidas sneakers + white t-shirt” (this is an example when there are two products in title)</li>
  </ul>
  <p>There may be more products in the title</p>
  <p>Output: the position of product for each word in the title.</p>
  <ul>
  <li>E. g. for title:  “red adidas sneakers” word “”sneakers” indicates that’s a product. Other words are sneaker’s features. It’s on 3rd position do so the output would be a vector [3,3,3,0,0,0,0,0,0,0].  So I am mapping features to the product (word to position of product)</li>
  <li>For title with 2 products:  “red adidas sneakers + white t-shirt” the output would look like this: [3,3,3,0,6,6,0,0,0,0]. mapping goes like this: (red=3, adidas=3, sneakers=3, +=0, white=6, t-shirt=6)</li>
  </ul>
  <p>It seems like a category prediction problem but it is not. Two similar titles make different outputs: e.g. “adiadas sneakers red t-shirt white” gets [2,2,2,4,4,0,0,0,0,0] and if differs from [3,3,3,0,6,6,0,0,0,0]. The same word “red” gets a different prediction (2 and 3).</p>
  <p>Therefore I have two questions.</p>
  <ol>
  <li>Is there a known model that solves this kind of problem?</li>
  </ol>
  <p>I came with an idea inspired by the attention mechanism to create a model with 2 encoders from the same title. Use some CNN architecture to embrace the word context.  And then make dot product of two encoded inputs and predict word positions.</p>
  <pre><code>embeddings = tf.keras.layers.Embedding(max_tokens, dimension)
  input1_embeddings = token_embedding(embeddings)
  input2_embeddings = token_embedding(embeddings)
  
  # CNN layer.
  cnn_layer = tf.keras.layers.Conv1D(
      filters=100,
      kernel_size=3,
      # Use 'same' padding so outputs have the same shape as inputs.
      padding='same')
  encoded1 = cnnlayer(input1_embeddings)
  encoded2 = cnnlayer(input2_embeddings)
  l = tf.keras.layers.Dot(axes=(1, 2))([encoded1, encoded2])
  output = Dense(10, activation=”softmax”)(l)
  </code></pre>
  <p>Is this a reasonable way to do it?</p>