<p dir="auto">Is it possible to add option batch_first = True for MultiheadAttention and  transformer modules like it is done for RNN  ?<br>
Batch  x Sequence x Embedding<br>
I find it more understandable and intuitive when batch_first is true</p>
<h2 dir="auto">🚀 Feature</h2>
<p dir="auto">Like recurrent nets (nn.RNN, nn.GRU, nn.LSTM etc) nn.Transformer is missing <strong>batch_first</strong> defualt parameter</p>
<h2 dir="auto">Motivation</h2>
<p dir="auto">torchtext BuckterIteror also has batch_first default parameter hence if nn.Transformer would have batch_first, it will save the dimension permutation</p>
<h2 dir="auto">Pitch</h2>
<p dir="auto">A conditional statement inside the nn.Transformer's forward method to check whether batch_first is True or not we can permute inside the nn.Transformer module if required</p>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/albanD/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/albanD">@albanD</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mruberry/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mruberry">@mruberry</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zhangguanheng66/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zhangguanheng66">@zhangguanheng66</a></p>