<p dir="auto">This error happens when trying to read in the weights file using an older version of Pytorch. I assume this is why you say Pytorch needs to be built from source. However I've tried that all evening and I can't find a way to navigate all of the nvcc/gcc/cuda incompatibilities to get it to compile. Many errors, all of which are common when I google, with lots of workarounds, but all of them only partially work. Fundamentally it seems like some sort of std::tuple issue with CUDA/nvcc which Nvidia acknowledges but say they won't fix until the next CUDA release.</p>
<p dir="auto">Is there any chance you could cave your weight file out into an older PyTorch format? Then I could just install python-pytorch-cuda-0.3.1-2 for my version of linux and be up and running in moments. ELF itself compiled fine, and it runs with python-pytorch-cuda-0.3.1-2... it just can't read the weights file. <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="304606324" data-permission-text="Title is private" data-url="https://github.com/pytorch/pytorch/issues/5729" data-hovercard-type="issue" data-hovercard-url="/pytorch/pytorch/issues/5729/hovercard" href="https://github.com/pytorch/pytorch/issues/5729">#5729</a> states it's because of the newer file format.</p>
<p dir="auto">Thanks!</p>
<p dir="auto">I did try going to <a href="https://pytorch.org/" rel="nofollow">https://pytorch.org/</a> and choosing Linux / pip / Python 3.6 / CUDA 9.2 which gives me these commands:</p>
<p dir="auto">pip3 install <a href="http://download.pytorch.org/whl/cu91/torch-0.4.0-cp36-cp36m-linux_x86_64.whl" rel="nofollow">http://download.pytorch.org/whl/cu91/torch-0.4.0-cp36-cp36m-linux_x86_64.whl</a><br>
pip3 install torchvision</p>
<p dir="auto">Which doesn't give me the error above, but I do instead get:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="RuntimeError: Error(s) in loading state_dict for Model_PolicyValue:
Unexpected key(s) in state_dict: &quot;init_conv.1.num_batches_tracked&quot;, &quot;pi_final_conv.1.num_batches_tracked&quot;, &quot;value_final_conv.1.num_batches_tracked&quot;, &quot;resnet.resnet.0.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.0.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.1.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.1.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.2.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.2.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.3.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.3.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.4.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.4.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.5.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.5.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.6.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.6.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.7.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.7.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.8.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.8.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.9.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.9.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.10.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.10.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.11.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.11.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.12.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.12.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.13.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.13.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.14.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.14.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.15.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.15.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.16.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.16.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.17.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.17.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.18.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.18.conv_upper.1.num_batches_tracked&quot;, &quot;resnet.resnet.19.conv_lower.1.num_batches_tracked&quot;, &quot;resnet.resnet.19.conv_upper.1.num_batches_tracked&quot;."><pre class="notranslate"><code class="notranslate">RuntimeError: Error(s) in loading state_dict for Model_PolicyValue:
Unexpected key(s) in state_dict: "init_conv.1.num_batches_tracked", "pi_final_conv.1.num_batches_tracked", "value_final_conv.1.num_batches_tracked", "resnet.resnet.0.conv_lower.1.num_batches_tracked", "resnet.resnet.0.conv_upper.1.num_batches_tracked", "resnet.resnet.1.conv_lower.1.num_batches_tracked", "resnet.resnet.1.conv_upper.1.num_batches_tracked", "resnet.resnet.2.conv_lower.1.num_batches_tracked", "resnet.resnet.2.conv_upper.1.num_batches_tracked", "resnet.resnet.3.conv_lower.1.num_batches_tracked", "resnet.resnet.3.conv_upper.1.num_batches_tracked", "resnet.resnet.4.conv_lower.1.num_batches_tracked", "resnet.resnet.4.conv_upper.1.num_batches_tracked", "resnet.resnet.5.conv_lower.1.num_batches_tracked", "resnet.resnet.5.conv_upper.1.num_batches_tracked", "resnet.resnet.6.conv_lower.1.num_batches_tracked", "resnet.resnet.6.conv_upper.1.num_batches_tracked", "resnet.resnet.7.conv_lower.1.num_batches_tracked", "resnet.resnet.7.conv_upper.1.num_batches_tracked", "resnet.resnet.8.conv_lower.1.num_batches_tracked", "resnet.resnet.8.conv_upper.1.num_batches_tracked", "resnet.resnet.9.conv_lower.1.num_batches_tracked", "resnet.resnet.9.conv_upper.1.num_batches_tracked", "resnet.resnet.10.conv_lower.1.num_batches_tracked", "resnet.resnet.10.conv_upper.1.num_batches_tracked", "resnet.resnet.11.conv_lower.1.num_batches_tracked", "resnet.resnet.11.conv_upper.1.num_batches_tracked", "resnet.resnet.12.conv_lower.1.num_batches_tracked", "resnet.resnet.12.conv_upper.1.num_batches_tracked", "resnet.resnet.13.conv_lower.1.num_batches_tracked", "resnet.resnet.13.conv_upper.1.num_batches_tracked", "resnet.resnet.14.conv_lower.1.num_batches_tracked", "resnet.resnet.14.conv_upper.1.num_batches_tracked", "resnet.resnet.15.conv_lower.1.num_batches_tracked", "resnet.resnet.15.conv_upper.1.num_batches_tracked", "resnet.resnet.16.conv_lower.1.num_batches_tracked", "resnet.resnet.16.conv_upper.1.num_batches_tracked", "resnet.resnet.17.conv_lower.1.num_batches_tracked", "resnet.resnet.17.conv_upper.1.num_batches_tracked", "resnet.resnet.18.conv_lower.1.num_batches_tracked", "resnet.resnet.18.conv_upper.1.num_batches_tracked", "resnet.resnet.19.conv_lower.1.num_batches_tracked", "resnet.resnet.19.conv_upper.1.num_batches_tracked".
</code></pre></div>
<h2 dir="auto">PyTorch GitHub Issues Guidelines</h2>
<p dir="auto">We like to limit our issues to bug reports and feature requests. If you have a question or would like help and support, please visit our forums: <a href="https://discuss.pytorch.org/" rel="nofollow">https://discuss.pytorch.org/</a></p>
<p dir="auto">If you are submitting a feature request, please preface the title with [feature request].</p>
<p dir="auto">When submitting a bug report, please include the following information (where relevant):</p>
<ul dir="auto">
<li>PyTorch or Caffe2: <code class="notranslate">PyTorch</code></li>
<li>OS: <code class="notranslate">Ubuntu 16.04</code></li>
<li>PyTorch version:  save using <code class="notranslate">0.4.0a0+67bbf58</code>, load using <code class="notranslate">0.2.0</code></li>
<li>How you installed PyTorch (conda, pip, source): <code class="notranslate">source</code></li>
<li>Python version: <code class="notranslate">python2.7</code></li>
<li>CUDA/cuDNN version: <code class="notranslate">8.0/6.1</code></li>
<li>GPU models and configuration: <code class="notranslate">TitanXp</code></li>
<li>GCC version (if compiling from source): <code class="notranslate">gcc (Ubuntu 5.4.0-6ubuntu1~16.04.9) 5.4.0 20160609</code></li>
<li>CMake version:  <code class="notranslate">cmake version 3.5.1</code></li>
<li>Build command you used (if compiling from source): <code class="notranslate">python setup.py install --user</code></li>
<li>Versions of any other relevant libraries:</li>
</ul>
<p dir="auto">In addition, including the following information will also be very helpful for us to diagnose the problem:</p>
<ul dir="auto">
<li>A script to reproduce the bug. Please try to provide as minimal of a test case as possible.</li>
<li>Error messages and/or stack traces of the bug</li>
<li>Context around what you are trying to do</li>
</ul>
<ul dir="auto">
<li>Create a tensor and save it in <code class="notranslate">0.4.0a0+67bbf58</code>:</li>
</ul>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch as tc
print(tc.__version__) # 0.2.0
t = tc.ones(3)
tc.save(t, './t.pth')"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span> <span class="pl-k">as</span> <span class="pl-s1">tc</span>
<span class="pl-en">print</span>(<span class="pl-s1">tc</span>.<span class="pl-s1">__version__</span>) <span class="pl-c"># 0.2.0</span>
<span class="pl-s1">t</span> <span class="pl-c1">=</span> <span class="pl-s1">tc</span>.<span class="pl-en">ones</span>(<span class="pl-c1">3</span>)
<span class="pl-s1">tc</span>.<span class="pl-en">save</span>(<span class="pl-s1">t</span>, <span class="pl-s">'./t.pth'</span>)</pre></div>
<ul dir="auto">
<li>Load it in <code class="notranslate">0.2.0</code>:</li>
</ul>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch as tc
print(tc.__version__) # 0.4.0a0+67bbf58
t = tc.load('./t.pth') # error"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span> <span class="pl-k">as</span> <span class="pl-s1">tc</span>
<span class="pl-en">print</span>(<span class="pl-s1">tc</span>.<span class="pl-s1">__version__</span>) <span class="pl-c"># 0.4.0a0+67bbf58</span>
<span class="pl-s1">t</span> <span class="pl-c1">=</span> <span class="pl-s1">tc</span>.<span class="pl-en">load</span>(<span class="pl-s">'./t.pth'</span>) <span class="pl-c"># error</span></pre></div>
<ul dir="auto">
<li>ERROR:</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
&lt;ipython-input-2-5932959b71e5&gt; in &lt;module&gt;()
----&gt; 1 x = tc.load('./t.pth')

/home/ylxie/.local/lib/python2.7/site-packages/torch/serialization.pyc in load(f, map_location, pickle_module)
    229         f = open(f, 'rb')
    230     try:
--&gt; 231         return _load(f, map_location, pickle_module)
    232     finally:
    233         if new_fd:

/home/ylxie/.local/lib/python2.7/site-packages/torch/serialization.pyc in _load(f, map_location, pickle_module)
    377     unpickler = pickle_module.Unpickler(f)
    378     unpickler.persistent_load = persistent_load
--&gt; 379     result = unpickler.load()
    380 
    381     deserialized_storage_keys = pickle_module.load(f)

AttributeError: 'module' object has no attribute '_rebuild_tensor_v2'"><pre class="notranslate"><code class="notranslate">---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
&lt;ipython-input-2-5932959b71e5&gt; in &lt;module&gt;()
----&gt; 1 x = tc.load('./t.pth')

/home/ylxie/.local/lib/python2.7/site-packages/torch/serialization.pyc in load(f, map_location, pickle_module)
    229         f = open(f, 'rb')
    230     try:
--&gt; 231         return _load(f, map_location, pickle_module)
    232     finally:
    233         if new_fd:

/home/ylxie/.local/lib/python2.7/site-packages/torch/serialization.pyc in _load(f, map_location, pickle_module)
    377     unpickler = pickle_module.Unpickler(f)
    378     unpickler.persistent_load = persistent_load
--&gt; 379     result = unpickler.load()
    380 
    381     deserialized_storage_keys = pickle_module.load(f)

AttributeError: 'module' object has no attribute '_rebuild_tensor_v2'
</code></pre></div>
<p dir="auto">How could I solve this? this is an abstract version of my issue,  the <code class="notranslate">t</code> is actually a <code class="notranslate">state_dict</code>. Is there any way I can specify a saving protocol for the convenience of loading models at an older version.</p>