title: PyTorch: Sizes of tensors must match on 2 input neural network
body: <p>I am attempting to recreate a 2 input neural network from this article: <a href="https://towardsdatascience.com/moving-from-keras-to-pytorch-f0d4fff4ce79" rel="nofollow noreferrer">https://towardsdatascience.com/moving-from-keras-to-pytorch-f0d4fff4ce79</a> <br/></p>
  <p>I have copied the network described in the post and adjusted it so that it fits my data. The first input is from GloVe Word embeddings while the other is numerical features about the text data.</p>
  <pre><code>class Net(nn.Module):
      def __init__(self,hidden_size,lin_size, embedding_matrix=embedding_weights):
          super(Alex_NeuralNet_Meta, self).__init__()
  
          # Initialize some parameters for your model
          self.hidden_size = hidden_size
          drp = 0.1
  
          # Layer 1: Embeddings.
          self.embedding = nn.Embedding(size_of_vocabulary, pretrained_embedding_dim)
          self.embedding.weight = nn.Parameter(torch.tensor(embedding_matrix, dtype=torch.float32))
          self.embedding.weight.requires_grad = False
  
          # Layer 2: Dropout1D(0.1)
          self.embedding_dropout = nn.Dropout2d(0.1)
  
          # Layer 3: Bidirectional CuDNNLSTM
          self.lstm = nn.LSTM(pretrained_embedding_dim, hidden_size, bidirectional=True, batch_first=True)
  
          # Layer 4: Bidirectional CuDNNGRU
          self.gru = nn.GRU(hidden_size*2, hidden_size, bidirectional=True, batch_first=True)
  
          # Layer 7: A dense layer
          self.linear = nn.Linear(hidden_size*6 + X2_train.shape[1], lin_size)
          self.relu = nn.ReLU()
  
          # Layer 8: A dropout layer 
          self.dropout = nn.Dropout(drp)
  
          # Layer 9: Output dense layer with one output for our Binary Classification problem.
          self.out = nn.Linear(lin_size, 1)
  
      def forward(self, x):
          
          '''
          here x[0] represents the first element of the input that is going to be passed. 
          We are going to pass a tuple where first one contains the sequences(x[0])
          and the second one is a additional feature vector(x[1])
          '''
          h_embedding = self.embedding(x[0].long())
          h_embedding = torch.squeeze(self.embedding_dropout(torch.unsqueeze(h_embedding, 0)))
          #print(&quot;emb&quot;, h_embedding.size())
          h_lstm, _ = self.lstm(h_embedding)
  #         print(&quot;lst&quot;,h_lstm.size())
          h_gru, hh_gru = self.gru(h_lstm)
          hh_gru = hh_gru.view(-1, 2*self.hidden_size )
          print(&quot;gru&quot;, h_gru.size())
          print(&quot;h_gru&quot;, hh_gru.size())
  
          # Layer 5: is defined dynamically as an operation on tensors.
          avg_pool = torch.mean(h_gru, 1)
          max_pool, _ = torch.max(h_gru, 1)
          print(&quot;avg_pool&quot;, avg_pool.size())
          print(&quot;max_pool&quot;, max_pool.size())
  
          # the extra features you want to give to the model
          f = torch.tensor(x[1], dtype=torch.float).cuda()
          print(&quot;f&quot;, f.size())
  
          # Layer 6: A concatenation of the last state, maximum pool, average pool and 
          # additional features
          conc = torch.cat(( hh_gru, avg_pool, max_pool, f), 1)
          #print(&quot;conc&quot;, conc.size())
  
          # passing conc through linear and relu ops
          conc = self.relu(self.linear(conc))
          conc = self.dropout(conc)
          out = self.out(conc)
          # return the final output
          return out
  </code></pre>
  <p>And during runtime I get an error on the concatenation line:</p>
  <pre><code>RuntimeError: Sizes of tensors must match except in dimension 0. Got 33164 and 20 (The offending index is 0)
  </code></pre>
  <p>From the dimensions of the outputs, I can see where the problem lies but I am not sure how I can fix it</p>
  <br/>
  <p>The data inputs to the network is:</p>
  <pre><code>torch.Size([20, 150])
  torch.Size([33164, 40])
  </code></pre>
  <p>The sizes of each layer output is:</p>
  <pre><code>gru torch.Size([20, 150, 80])
  h_gru torch.Size([20, 80])
  avg_pool torch.Size([20, 80])
  max_pool torch.Size([20, 80])
  f torch.Size([33164, 40])
  </code></pre>
  <p>For the example above the batch size is 20, hidden_size is 40, the number of rows in numerical data features is 33164 and its feature size is 40.</p>
  <p>Thanks for any help in advance</p>