The expanded size of the tensor (64) must match the existing size (66) at non-singleton dimension 2
When I run the following (unchanged) code in stable PyTorch 1.9, I get the following error. How can I diagnose and fix it?
(fashcomp) [jalal@goku fashion-compatibility]$ python main.py --test --l2_embed --resume runs/nondisjoint_l2norm/model_best.pth.tar --datadir ../../../data/fashion/
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torchvision/transforms/transforms.py:310: UserWarning: The use of the transforms.Scale transform is deprecated, please use transforms.Resize instead.
  warnings.warn(&quot;The use of the transforms.Scale transform is deprecated, &quot; +
=&gt; loading checkpoint 'runs/nondisjoint_l2norm/model_best.pth.tar'
=&gt; loaded checkpoint 'runs/nondisjoint_l2norm/model_best.pth.tar' (epoch 5)
/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/functional.py:718: UserWarning: Named tensors and all their associated APIs are an experimental feature and subject to change. Please do not use them for anything important until they are released as stable. (Triggered internally at  /pytorch/c10/core/TensorImpl.h:1156.)
  return torch.max_pool2d(input, kernel_size, stride, padding, dilation, ceil_mode)
Traceback (most recent call last):
  File &quot;main.py&quot;, line 312, in &lt;module&gt;
    main()    
  File &quot;main.py&quot;, line 149, in main
    test_acc = test(test_loader, tnet)
  File &quot;main.py&quot;, line 244, in test
    embeddings.append(tnet.embeddingnet(images).data)
  File &quot;/scratch3/venv/fashcomp/lib/python3.8/site-packages/torch/nn/modules/module.py&quot;, line 1051, in _call_impl
    return forward_call(*input, **kwargs)
  File &quot;/scratch3/research/code/fashion/fashion-compatibility/type_specific_network.py&quot;, line 119, in forward
    masked_embedding = masked_embedding / norm.expand_as(masked_embedding)
RuntimeError: The expanded size of the tensor (64) must match the existing size (66) at non-singleton dimension 2.  Target sizes: [256, 66, 64].  Tensor sizes: [256, 66]

Related issue: https://github.com/mvasil/fashion-compatibility/pull/13

The problem is that norm is missing that extra dimension: masked_embedding is [256, 66, 64] and norm is [256, 66].  You can fix that by adding this extra dimension ([256, 66, 1]) to norm this way:
masked_embedding = masked_embedding / norm.unsqueeze(-1).expand_as(masked_embedding)

OR, by modifying the call (this one) that generated norm:
norm = torch.norm(masked_embedding, p=2, dim=2, keepdim=True) + 1e-10

