title: Diffrent node number in mini_batch
body: <p>I am fairly new to graph neural networks and I am training a GNN model using self attention and I have a few questions.</p>
  <p>The question is my node count and node_num differs in each batch such that in the first batch I have:</p>
  <p>Batch(batch=[1181], edge_attr=[1975, 3], edge_index=[2, 1975], x=[1181, 300])</p>
  <p>in the second batch I have:
  batch=[1134], edge_attr=[1635, 3], edge_index=[2, 1635], x=[1134, 300]</p>
  <p>There were 1181 nodes in batch1, whereas 1134 nodes in batch2</p>
  <p>When I tried to calculate self attention between nodes, I encountered the following problem</p>
  <p>Here's how self attention works</p>
  <p><a href="https://i.stack.imgur.com/9P1tK.png" rel="nofollow noreferrer">enter image description here</a></p>
  <p>the Q, W, K calculate as follows:
  <a href="https://i.stack.imgur.com/63W4c.png" rel="nofollow noreferrer">enter image description here</a></p>
  <p>the dimension of wq, wk, wv is</p>
  <p>self.w_1 = Param(torch.Tensor(self.nodes_num, self.nodes_num))</p>
  <p>So the problem I have is this</p>
  <p>in batch1 , the the dimension of wq, wk, wv is self.w_q = Param(torch.Tensor(1181, 1181))
  in batch2 , the the dimension of wq, wk, wv is self.w_q = Param(torch.Tensor(1134, 1134))
  Dimensions vary with the number of nodes, causing w_q to be constantly redefined</p>
  <p>Is this equivalent to using only one batch of samples for the model?</p>
  <p>If so, how can I solve the problem?</p>