<p dir="auto">Feature:<br>
Make all Tensors have an optional list of dimension names like ['B','C','H','W']<br>
Then the broadcasting in binary operations like<br>
a + b<br>
can be done according to the names (if they both have named dimensions). For example, if Tensor a has a list ['B','C','H','W'] and b has list ['C'] , then the expected result is efficiently equivalent to<br>
a + b.view([1,-1,1,1])</p>
<p dir="auto">This will make the broadcasting safe and very convenient to use.<br>
Currently, because broadcasting is implicit, if something goes wrong it is a luck if a dimension mismatch happens somewhen later on or there is an out of memory error thrown.</p>
<p dir="auto">I think it could further ease the writing of a generic code that has to work with different number of spatial dimensions (e.g. fully connected layers and conv layers), debug problems, etc. Maybe the named dimensions method be also adopted in functions to allow for the syntax like<br>
log_softmax(axis='C')</p>
<p dir="auto">Why the usual axis number is not sufficient? The axes are hardwired to the dimensionality. For example, the affine weight in BatchNorm is 1D, and its dimension 0 is by default not matching to the dimension 0 in e.g. the input data. So there going to be lots of these .view() commands and if you expect the input with or without spatial dimensions it is going to be more pain.</p>
<p dir="auto">Now consider the code using named dimensions. It will pretty much work if you change the data layout (how it is linearized in the memory). Or if you increase the number of dimensions. For example, you want to extend to a siamese (or 4-stream) network processing, for which it is convenient to use 5D Tensors with dimensions  'instance', 'stream', 'channels','width','height'. Suddenly 'channels' becomes dimension number 2 instead of 1, but if dimensions are matched by names there is no problem.</p>
<p dir="auto">It would be great if there was a way to name tensor dimensions and use the names in all of the tensor methods that take dim as an argument.</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="images = torch.Tensor(data, dims=('batch', 'channel', 'height', 'width'))
more_images = torch.Tensor(data, dims=('batch', 'channel', 'height', 'width'))

torch.cat((images, more_images), dim='batch')"><pre class="notranslate"><span class="pl-s1">images</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-v">Tensor</span>(<span class="pl-s1">data</span>, <span class="pl-s1">dims</span><span class="pl-c1">=</span>(<span class="pl-s">'batch'</span>, <span class="pl-s">'channel'</span>, <span class="pl-s">'height'</span>, <span class="pl-s">'width'</span>))
<span class="pl-s1">more_images</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-v">Tensor</span>(<span class="pl-s1">data</span>, <span class="pl-s1">dims</span><span class="pl-c1">=</span>(<span class="pl-s">'batch'</span>, <span class="pl-s">'channel'</span>, <span class="pl-s">'height'</span>, <span class="pl-s">'width'</span>))

<span class="pl-s1">torch</span>.<span class="pl-en">cat</span>((<span class="pl-s1">images</span>, <span class="pl-s1">more_images</span>), <span class="pl-s1">dim</span><span class="pl-c1">=</span><span class="pl-s">'batch'</span>)</pre></div>
<p dir="auto">This could obviously be expanded to support axis objects that could support a basic algebra to allow using dimensions in functions like <code class="notranslate">view()</code>.</p>
<p dir="auto">Another somewhat related feature would be a way to add type annotations as proposed in <a href="https://docs.google.com/document/d/1vpMse4c6DrWH5rq2tQSx3qwP_m_0lyn-Ij4WHqQqRHY/edit#heading=h.rkj7d39awayl" rel="nofollow">this google doc</a>:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="Batch = DimensionVar('Batch')
Row = DimensionVar('Row')
Column = DimensionVar('Column')
Channel = DimensionVar('Channel', size=3)

RGBImage = NDArray[Batch, Row, Column, Channel]
GrayscaleImage = NDArray[Batch, Row, Column]

def rgb_to_grayscale(array: RGBImage) -&gt; GrayscaleImage:
  return numpy.mean(array, axis=RGBImage.axis(Channel))"><pre class="notranslate"><span class="pl-v">Batch</span> <span class="pl-c1">=</span> <span class="pl-v">DimensionVar</span>(<span class="pl-s">'Batch'</span>)
<span class="pl-v">Row</span> <span class="pl-c1">=</span> <span class="pl-v">DimensionVar</span>(<span class="pl-s">'Row'</span>)
<span class="pl-v">Column</span> <span class="pl-c1">=</span> <span class="pl-v">DimensionVar</span>(<span class="pl-s">'Column'</span>)
<span class="pl-v">Channel</span> <span class="pl-c1">=</span> <span class="pl-v">DimensionVar</span>(<span class="pl-s">'Channel'</span>, <span class="pl-s1">size</span><span class="pl-c1">=</span><span class="pl-c1">3</span>)

<span class="pl-v">RGBImage</span> <span class="pl-c1">=</span> <span class="pl-v">NDArray</span>[<span class="pl-v">Batch</span>, <span class="pl-v">Row</span>, <span class="pl-v">Column</span>, <span class="pl-v">Channel</span>]
<span class="pl-v">GrayscaleImage</span> <span class="pl-c1">=</span> <span class="pl-v">NDArray</span>[<span class="pl-v">Batch</span>, <span class="pl-v">Row</span>, <span class="pl-v">Column</span>]

<span class="pl-k">def</span> <span class="pl-en">rgb_to_grayscale</span>(<span class="pl-s1">array</span>: <span class="pl-v">RGBImage</span>) <span class="pl-c1">-&gt;</span> <span class="pl-v">GrayscaleImage</span>:
  <span class="pl-k">return</span> <span class="pl-s1">numpy</span>.<span class="pl-en">mean</span>(<span class="pl-s1">array</span>, <span class="pl-s1">axis</span><span class="pl-c1">=</span><span class="pl-v">RGBImage</span>.<span class="pl-en">axis</span>(<span class="pl-v">Channel</span>))</pre></div>
<ul dir="auto">
<li><a href="https://github.com/BIDS/datarray">Datarray: numpy with axis names</a></li>
<li><a href="https://mail.python.org/pipermail/numpy-discussion/2017-November/077429.html" rel="nofollow">python mailing list discussion</a></li>
<li><a href="https://docs.google.com/document/d/1vpMse4c6DrWH5rq2tQSx3qwP_m_0lyn-Ij4WHqQqRHY/edit#heading=h.rkj7d39awayl" rel="nofollow">Google doc proposing array shape annotations</a></li>
<li><a href="https://www.python.org/dev/peps/pep-0472/" rel="nofollow">Python PEP 472 proposing named indexing</a></li>
</ul>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gchanan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gchanan">@gchanan</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a></p>