<p dir="auto">I am using Keras to predict multilables, that is, one record might have multiple classes. For ex, if there are 10 target classes, a record can have more than one target class. In this scenario whats the best way to transform target. I am using categorical_crossentropy as my objective. I tried both - 1) having multiple 1s and 0s within 10 output vectors &amp; 2) having only 1 in one of the 10 output vectors &amp; rest 0s. In this way I would duplicate each record with multiple outputs.<br>
I read Keras link <a href="https://keras.io/objectives/" rel="nofollow">https://keras.io/objectives/</a> where its specifically written -<br>
"Note: when using the categorical_crossentropy objective, your targets should be in categorical format (e.g. if you have 10 classes, the target for each sample should be a 10-dimensional vector that is all-zeros expect for a 1 at the index corresponding to the class of the sample). In order to convert integer targets into categorical targets, you can use the Keras utility to_categorical"<br>
If this is true, still I don't get any error with with 1st approach and even the predictions are better. I want to understand whats happening inside and how does it tackle this kind of situations</p>
<p dir="auto">The issue appears because of composing the layers: <code class="notranslate">TimeDistributed(L)</code></p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import keras
from keras import backend as K
from keras.models import Model, load_model
from keras.layers import Input, Dense, Concatenate, Flatten
from keras.layers.wrappers import TimeDistributed
a = Input(shape=(784,5))
b = Input(shape=(784,5))

L = Dense(units=10)

u = TimeDistributed(L)(a)  # u = L(a) works correctly
v = L(b)

x = Concatenate() ([u, v])

model = Model(inputs=[a, b], outputs=x)

print model.count_params() # 120
model.summary() # 60

model.save('tmp.model')

###################################

loaded_model = load_model('tmp.model')

print loaded_model.count_params() # 120
loaded_model.summary() # 120"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">keras</span>
<span class="pl-k">from</span> <span class="pl-s1">keras</span> <span class="pl-k">import</span> <span class="pl-s1">backend</span> <span class="pl-k">as</span> <span class="pl-v">K</span>
<span class="pl-k">from</span> <span class="pl-s1">keras</span>.<span class="pl-s1">models</span> <span class="pl-k">import</span> <span class="pl-v">Model</span>, <span class="pl-s1">load_model</span>
<span class="pl-k">from</span> <span class="pl-s1">keras</span>.<span class="pl-s1">layers</span> <span class="pl-k">import</span> <span class="pl-v">Input</span>, <span class="pl-v">Dense</span>, <span class="pl-v">Concatenate</span>, <span class="pl-v">Flatten</span>
<span class="pl-k">from</span> <span class="pl-s1">keras</span>.<span class="pl-s1">layers</span>.<span class="pl-s1">wrappers</span> <span class="pl-k">import</span> <span class="pl-v">TimeDistributed</span>
<span class="pl-s1">a</span> <span class="pl-c1">=</span> <span class="pl-v">Input</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span>(<span class="pl-c1">784</span>,<span class="pl-c1">5</span>))
<span class="pl-s1">b</span> <span class="pl-c1">=</span> <span class="pl-v">Input</span>(<span class="pl-s1">shape</span><span class="pl-c1">=</span>(<span class="pl-c1">784</span>,<span class="pl-c1">5</span>))

<span class="pl-v">L</span> <span class="pl-c1">=</span> <span class="pl-v">Dense</span>(<span class="pl-s1">units</span><span class="pl-c1">=</span><span class="pl-c1">10</span>)

<span class="pl-s1">u</span> <span class="pl-c1">=</span> <span class="pl-v">TimeDistributed</span>(<span class="pl-v">L</span>)(<span class="pl-s1">a</span>)  <span class="pl-c"># u = L(a) works correctly</span>
<span class="pl-s1">v</span> <span class="pl-c1">=</span> <span class="pl-v">L</span>(<span class="pl-s1">b</span>)

<span class="pl-s1">x</span> <span class="pl-c1">=</span> <span class="pl-v">Concatenate</span>() ([<span class="pl-s1">u</span>, <span class="pl-s1">v</span>])

<span class="pl-s1">model</span> <span class="pl-c1">=</span> <span class="pl-v">Model</span>(<span class="pl-s1">inputs</span><span class="pl-c1">=</span>[<span class="pl-s1">a</span>, <span class="pl-s1">b</span>], <span class="pl-s1">outputs</span><span class="pl-c1">=</span><span class="pl-s1">x</span>)

<span class="pl-k">print</span> <span class="pl-s1">model</span>.<span class="pl-en">count_params</span>() <span class="pl-c"># 120</span>
<span class="pl-s1">model</span>.<span class="pl-en">summary</span>() <span class="pl-c"># 60</span>

<span class="pl-s1">model</span>.<span class="pl-en">save</span>(<span class="pl-s">'tmp.model'</span>)

<span class="pl-c">###################################</span>

<span class="pl-s1">loaded_model</span> <span class="pl-c1">=</span> <span class="pl-en">load_model</span>(<span class="pl-s">'tmp.model'</span>)

<span class="pl-k">print</span> <span class="pl-s1">loaded_model</span>.<span class="pl-en">count_params</span>() <span class="pl-c"># 120</span>
<span class="pl-s1">loaded_model</span>.<span class="pl-en">summary</span>() <span class="pl-c"># 120</span></pre></div>
<p dir="auto">Note that <code class="notranslate">count_params()</code> always gives incorrect numbers. Is that another bug?</p>