<h2 dir="auto"><g-emoji class="g-emoji" alias="bug" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f41b.png">🐛</g-emoji> Bug</h2>
<p dir="auto">Inconsistent results of torch.argmax with tensors that have duplicated values</p>

<h2 dir="auto">To Reproduce</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cpu = torch.LongTensor([[2,3,0,3,3], [1,3,1,3,1]])
cpu.argmax(dim=1) # tensor([4, 3])
cuda = torch.cuda.LongTensor([[2,3,0,3,3], [1,3,1,3,1]])
cuda.argmax(dim=1) #tensor([4, 1], device='cuda:0')"><pre class="notranslate"><code class="notranslate">cpu = torch.LongTensor([[2,3,0,3,3], [1,3,1,3,1]])
cpu.argmax(dim=1) # tensor([4, 3])
cuda = torch.cuda.LongTensor([[2,3,0,3,3], [1,3,1,3,1]])
cuda.argmax(dim=1) #tensor([4, 1], device='cuda:0')
</code></pre></div>
<p dir="auto">When I did the same on another machine(pytorch 0.4.1, CUDA 9.0)</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="cpu = torch.LongTensor([[2,3,0,3,0], [1,3,1,3,1]])
cpu.argmax(dim=1) # tensor([4, 1]). 
cuda = torch.cuda.LongTensor([[2,3,0,3,0], [1,3,1,3,1]])
cuda.argmax(dim=1) # tensor([1, 1, device='cuda:0')"><pre class="notranslate"><code class="notranslate">cpu = torch.LongTensor([[2,3,0,3,0], [1,3,1,3,1]])
cpu.argmax(dim=1) # tensor([4, 1]). 
cuda = torch.cuda.LongTensor([[2,3,0,3,0], [1,3,1,3,1]])
cuda.argmax(dim=1) # tensor([1, 1, device='cuda:0')
</code></pre></div>

<h2 dir="auto">Expected behavior</h2>
<p dir="auto">torch.argmax returns the same value for tensors that have same data in it, regardless of devices(or other factors).</p>

<h2 dir="auto">Environment</h2>
<p dir="auto">PyTorch version: 0.4.1.post2<br>
Is debug build: No<br>
CUDA used to build PyTorch: 9.0.176</p>
<p dir="auto">OS: Ubuntu 16.04.5 LTS<br>
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.10) 5.4.0 20160609<br>
CMake version: version 3.5.1</p>
<p dir="auto">Python version: 3.6<br>
Is CUDA available: Yes<br>
CUDA runtime version: 9.0.176<br>
GPU models and configuration:<br>
GPU 0: GeForce GTX 1080 Ti<br>
GPU 1: GeForce GTX 1080 Ti</p>
<p dir="auto">Nvidia driver version: 410.79<br>
cuDNN version: /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7.3.1</p>
<p dir="auto">Versions of relevant libraries:<br>
[pip3] msgpack-numpy==0.4.4.1<br>
[pip3] numpy==1.15.0<br>
[pip3] torch==0.4.1.post2<br>
[pip3] torchtext==0.3.1<br>
[pip3] torchvision==0.2.1<br>
[conda] blas                      1.0                         mkl<br>
[conda] mkl                       2018.0.3                      1<br>
[conda] mkl_fft                   1.0.4            py36h4414c95_1<br>
[conda] mkl_random                1.0.1            py36h4414c95_1<br>
[conda] mxnet-mkl                 1.3.0.post0               <br>
[conda] pytorch                   0.4.1           py36_py35_py27__9.0.176_7.1.2_2    pytorch<br>
[conda] torchtext                 0.3.1                     <br>
[conda] torchvision               0.2.1                    py36_0</p>
<h2 dir="auto">Additional context</h2>
<p dir="auto">It would be helpful if  the documentation describes if torch.argmax returns the last or the first index if a tensor holds duplicated values.</p>
<p dir="auto">Currently there doesn't seem to be a way to skip tracing of certain functions. Encountered this error when trying to symbolically trace open source BERT model to quantize it.</p>
<p dir="auto">Original error is from<br>
<a href="https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L238">https://github.com/huggingface/transformers/blob/master/src/transformers/modeling_bert.py#L238</a></p>
<p dir="auto">Unit test to repro -</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="import torch
from torch.fx import symbolic_trace
import torch.nn as nn

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def transpose_inp(self, x):
        new_shape = x.size()[:-1] + (5, 5)
        x = x.view(*new_shape)
        return x.permute(0, 2, 1, 3)

    def forward(self, x):
        tmp = self.transpose_inp(x)
        return tmp

m = symbolic_trace(M())"><pre class="notranslate"><code class="notranslate">import torch
from torch.fx import symbolic_trace
import torch.nn as nn

class M(torch.nn.Module):
    def __init__(self):
        super().__init__()

    def transpose_inp(self, x):
        new_shape = x.size()[:-1] + (5, 5)
        x = x.view(*new_shape)
        return x.permute(0, 2, 1, 3)

    def forward(self, x):
        tmp = self.transpose_inp(x)
        return tmp

m = symbolic_trace(M())
</code></pre></div>
<p dir="auto">Throws the following error</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="m = symbolic_trace(M())
  File &quot;/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/symbolic_trace.py&quot;, line 219, in symbolic_trace
    graph.output(delegate.create_arg(fn(*args)))
  File &quot;scripts/test_fx.py&quot;, line 49, in forward
    tmp = self.transpose_inp(x)
  File &quot;scripts/test_fx.py&quot;, line 44, in transpose_inp
    x = x.view(*new_shape)
  File &quot;/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/proxy.py&quot;, line 62, in __iter__
    self._no_arg_unpack()
  File &quot;/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/proxy.py&quot;, line 70, in _no_arg_unpack
    raise TraceError('Proxy object cannot be unpacked as function argument')
torch.fx.proxy.TraceError: Proxy object cannot be unpacked as function argument"><pre class="notranslate"><code class="notranslate">m = symbolic_trace(M())
  File "/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/symbolic_trace.py", line 219, in symbolic_trace
    graph.output(delegate.create_arg(fn(*args)))
  File "scripts/test_fx.py", line 49, in forward
    tmp = self.transpose_inp(x)
  File "scripts/test_fx.py", line 44, in transpose_inp
    x = x.view(*new_shape)
  File "/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/proxy.py", line 62, in __iter__
    self._no_arg_unpack()
  File "/home/supriyar/local/miniconda3/envs/pytorch/lib/python3.7/site-packages/torch/fx/proxy.py", line 70, in _no_arg_unpack
    raise TraceError('Proxy object cannot be unpacked as function argument')
torch.fx.proxy.TraceError: Proxy object cannot be unpacked as function argument
</code></pre></div>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gmagogsfm/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gmagogsfm">@gmagogsfm</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jerryzh168/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jerryzh168">@jerryzh168</a>, <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/vkuzo/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/vkuzo">@vkuzo</a></p>