<h2 dir="auto">🐛 Bug</h2>
<p dir="auto">When running ctc on the GPU, torch.nn.functional.ctc_loss expects different types for the "targets" input, depending on whether the selected backed is cudnn or the native implementation.</p>
<p dir="auto">However, the user does not know in advance whether the cuDNN implementation will be used by PyTorch or not. This is very inconvenient since the program may crash without any clear mechanism to prevent it.</p>
<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Python script:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch

# Generate data.
logits = torch.normal(mean=torch.zeros((20, 10, 5), dtype=torch.float32))
logprobs = torch.nn.functional.log_softmax(logits, dim=-1)
targets = torch.randint(1, 10, size=(10, 4), dtype=torch.int32)
target_lengths = torch.randint(1, 5, size=(10,), dtype=torch.int32)

# All examples have the same input length, so that cuDNN can be used.
input_lengths = 20 * torch.ones((10,), dtype=torch.int32)
# Reshape targets, so that cuDNN can be used
targets = torch.cat(tuple(targets[i, :target_lengths[i]] for i in range(10)))

# CPU: OK
print(torch.nn.functional.ctc_loss(logprobs, targets, input_lengths, target_lengths))

# CUDA, PyTorch native implementation: OK
torch.backends.cudnn.enabled = False
print(torch.nn.functional.ctc_loss(logprobs.to('cuda'), targets.to('cuda'), input_lengths, target_lengths))

# CUDA, cuDNN implementation: CRASHES (cuDNN expects targets in CPU).
torch.backends.cudnn.enabled = True
print(torch.nn.functional.ctc_loss(logprobs.to('cuda'), targets.to('cuda'), input_lengths, target_lengths))"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span>

<span class="pl-c"># Generate data.</span>
<span class="pl-s1">logits</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">normal</span>(<span class="pl-s1">mean</span><span class="pl-c1">=</span><span class="pl-s1">torch</span>.<span class="pl-en">zeros</span>((<span class="pl-c1">20</span>, <span class="pl-c1">10</span>, <span class="pl-c1">5</span>), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">torch</span>.<span class="pl-s1">float32</span>))
<span class="pl-s1">logprobs</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-s1">nn</span>.<span class="pl-s1">functional</span>.<span class="pl-en">log_softmax</span>(<span class="pl-s1">logits</span>, <span class="pl-s1">dim</span><span class="pl-c1">=</span><span class="pl-c1">-</span><span class="pl-c1">1</span>)
<span class="pl-s1">targets</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">randint</span>(<span class="pl-c1">1</span>, <span class="pl-c1">10</span>, <span class="pl-s1">size</span><span class="pl-c1">=</span>(<span class="pl-c1">10</span>, <span class="pl-c1">4</span>), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">torch</span>.<span class="pl-s1">int32</span>)
<span class="pl-s1">target_lengths</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">randint</span>(<span class="pl-c1">1</span>, <span class="pl-c1">5</span>, <span class="pl-s1">size</span><span class="pl-c1">=</span>(<span class="pl-c1">10</span>,), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">torch</span>.<span class="pl-s1">int32</span>)

<span class="pl-c"># All examples have the same input length, so that cuDNN can be used.</span>
<span class="pl-s1">input_lengths</span> <span class="pl-c1">=</span> <span class="pl-c1">20</span> <span class="pl-c1">*</span> <span class="pl-s1">torch</span>.<span class="pl-en">ones</span>((<span class="pl-c1">10</span>,), <span class="pl-s1">dtype</span><span class="pl-c1">=</span><span class="pl-s1">torch</span>.<span class="pl-s1">int32</span>)
<span class="pl-c"># Reshape targets, so that cuDNN can be used</span>
<span class="pl-s1">targets</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">cat</span>(<span class="pl-en">tuple</span>(<span class="pl-s1">targets</span>[<span class="pl-s1">i</span>, :<span class="pl-s1">target_lengths</span>[<span class="pl-s1">i</span>]] <span class="pl-k">for</span> <span class="pl-s1">i</span> <span class="pl-c1">in</span> <span class="pl-en">range</span>(<span class="pl-c1">10</span>)))

<span class="pl-c"># CPU: OK</span>
<span class="pl-en">print</span>(<span class="pl-s1">torch</span>.<span class="pl-s1">nn</span>.<span class="pl-s1">functional</span>.<span class="pl-en">ctc_loss</span>(<span class="pl-s1">logprobs</span>, <span class="pl-s1">targets</span>, <span class="pl-s1">input_lengths</span>, <span class="pl-s1">target_lengths</span>))

<span class="pl-c"># CUDA, PyTorch native implementation: OK</span>
<span class="pl-s1">torch</span>.<span class="pl-s1">backends</span>.<span class="pl-s1">cudnn</span>.<span class="pl-s1">enabled</span> <span class="pl-c1">=</span> <span class="pl-c1">False</span>
<span class="pl-en">print</span>(<span class="pl-s1">torch</span>.<span class="pl-s1">nn</span>.<span class="pl-s1">functional</span>.<span class="pl-en">ctc_loss</span>(<span class="pl-s1">logprobs</span>.<span class="pl-en">to</span>(<span class="pl-s">'cuda'</span>), <span class="pl-s1">targets</span>.<span class="pl-en">to</span>(<span class="pl-s">'cuda'</span>), <span class="pl-s1">input_lengths</span>, <span class="pl-s1">target_lengths</span>))

<span class="pl-c"># CUDA, cuDNN implementation: CRASHES (cuDNN expects targets in CPU).</span>
<span class="pl-s1">torch</span>.<span class="pl-s1">backends</span>.<span class="pl-s1">cudnn</span>.<span class="pl-s1">enabled</span> <span class="pl-c1">=</span> <span class="pl-c1">True</span>
<span class="pl-en">print</span>(<span class="pl-s1">torch</span>.<span class="pl-s1">nn</span>.<span class="pl-s1">functional</span>.<span class="pl-en">ctc_loss</span>(<span class="pl-s1">logprobs</span>.<span class="pl-en">to</span>(<span class="pl-s">'cuda'</span>), <span class="pl-s1">targets</span>.<span class="pl-en">to</span>(<span class="pl-s">'cuda'</span>), <span class="pl-s1">input_lengths</span>, <span class="pl-s1">target_lengths</span>))</pre></div>
<p dir="auto">Output:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="$ python minimal_ctc.py 
tensor(17.2880)
tensor(17.2880, device='cuda:0')
Traceback (most recent call last):
  File &quot;minimal_ctc.py&quot;, line 24, in &lt;module&gt;
    print(torch.nn.functional.ctc_loss(logprobs.to('cuda'), targets.to('cuda'), input_lengths, target_lengths))
  File &quot;/home/joapuipe/.virtualenvs/pytorch1.1-py3.6/lib/python3.6/site-packages/torch/nn/functional.py&quot;, line 1813, in ctc_loss
    zero_infinity)
RuntimeError: Expected tensor to have CPU Backend, but got tensor with CUDA Backend (while checking arguments for cudnn_ctc_loss)"><pre class="notranslate"><code class="notranslate">$ python minimal_ctc.py 
tensor(17.2880)
tensor(17.2880, device='cuda:0')
Traceback (most recent call last):
  File "minimal_ctc.py", line 24, in &lt;module&gt;
    print(torch.nn.functional.ctc_loss(logprobs.to('cuda'), targets.to('cuda'), input_lengths, target_lengths))
  File "/home/joapuipe/.virtualenvs/pytorch1.1-py3.6/lib/python3.6/site-packages/torch/nn/functional.py", line 1813, in ctc_loss
    zero_infinity)
RuntimeError: Expected tensor to have CPU Backend, but got tensor with CUDA Backend (while checking arguments for cudnn_ctc_loss)
</code></pre></div>
<h2 dir="auto">Expected behavior</h2>
<p dir="auto">First, the ctc_loss API should specify in the doc which device it expects the inputs.<br>
Secondly, I think it should be consistent regardless of the backend implementation used (e.g. copy to the expected device if necessary).</p>
<p dir="auto">If the dev team does not like implicit copies, I suggest a function like: <code class="notranslate">torch.backends.cudnn.will_ctc_loss_run_on_cudnn(logprobs, targets, inputs_lengths, targets_length, blank)</code> so that the user can check whether or not cudnn will be used for the given input (i.e the same check as </p><div class="Box Box--condensed my-2">
  <div class="Box-header f6">
    <p class="mb-0 text-bold">
      <a href="https://github.com/pytorch/pytorch/blob/1d705b4b075e32540293d1717b582442a66ffcce/aten/src/ATen/native/LossCTC.cpp#L344">pytorch/aten/src/ATen/native/LossCTC.cpp</a>
    </p>
    <p class="mb-0 color-fg-muted">
         Line 344
      in
      <a data-pjax="true" class="commit-tease-sha" href="/pytorch/pytorch/commit/1d705b4b075e32540293d1717b582442a66ffcce">1d705b4</a>
    </p>
  </div>
  <div itemprop="text" class="Box-body p-0 blob-wrapper blob-wrapper-embedded data">
    <table class="highlight tab-size mb-0 js-file-line-container" data-tab-size="8" data-paste-markdown-skip="">

        <tbody><tr class="border-0">
          <td id="L344" class="blob-num border-0 px-3 py-0 color-bg-default" data-line-number="344"></td>
          <td id="LC344" class="blob-code border-0 px-3 py-0 color-bg-default blob-code-inner js-file-line"> <span class="pl-k">bool</span> use_cudnn = </td>
        </tr>
    </tbody></table>
  </div>
</div>
).<p></p>
<h2 dir="auto">Environment</h2>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Collecting environment information...
PyTorch version: 1.1.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.6 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
CMake version: version 3.6.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: 
GPU 0: GeForce GTX 780
GPU 1: GeForce GTX 1080

Nvidia driver version: 415.27
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] nnutils-pytorch==0.4.0
[pip3] numpy==1.16.4
[pip3] torch==1.1.0
[pip3] torch-baidu-ctc==0.3.0
[pip3] torchvision==0.3.0
[conda] Could not collect"><pre class="notranslate"><code class="notranslate">Collecting environment information...
PyTorch version: 1.1.0
Is debug build: No
CUDA used to build PyTorch: 9.0.176

OS: Ubuntu 16.04.6 LTS
GCC version: (Ubuntu 5.4.0-6ubuntu1~16.04.11) 5.4.0 20160609
CMake version: version 3.6.1

Python version: 3.6
Is CUDA available: Yes
CUDA runtime version: 10.0.130
GPU models and configuration: 
GPU 0: GeForce GTX 780
GPU 1: GeForce GTX 1080

Nvidia driver version: 415.27
cuDNN version: Could not collect

Versions of relevant libraries:
[pip3] nnutils-pytorch==0.4.0
[pip3] numpy==1.16.4
[pip3] torch==1.1.0
[pip3] torch-baidu-ctc==0.3.0
[pip3] torchvision==0.3.0
[conda] Could not collect
</code></pre></div>
<p dir="auto">cc <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ezyang/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ezyang">@ezyang</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gchanan/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gchanan">@gchanan</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/zou3519/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/zou3519">@zou3519</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/jerryzh168/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/jerryzh168">@jerryzh168</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/ssnl/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/ssnl">@ssnl</a></p>
<h2 dir="auto">🐛 Bug</h2>

<h2 dir="auto">To Reproduce</h2>
<p dir="auto">Steps to reproduce the behavior:</p>
<p dir="auto">1.I i updated pytorch version and ctc，use pytorch_nightly, but in my train ,nn.CTCloss() is still zero,so,i would like to ask if the version pytorch(nightly) has been solved this problem<br>
1.<br>
1.</p>

<h2 dir="auto">Expected behavior</h2>

<h2 dir="auto">Environment</h2>
<p dir="auto">Please copy and paste the output from our<br>
<a href="https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py" rel="nofollow">environment collection script</a><br>
(or fill out the checklist below manually).</p>
<p dir="auto">You can get the script and run it with:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py"><pre class="notranslate"><code class="notranslate">wget https://raw.githubusercontent.com/pytorch/pytorch/master/torch/utils/collect_env.py
# For security purposes, please check the contents of collect_env.py before running it.
python collect_env.py
</code></pre></div>
<ul dir="auto">
<li>PyTorch Version (e.g., 1.0):</li>
<li>OS (e.g., Linux):</li>
<li>How you installed PyTorch (<code class="notranslate">conda</code>, <code class="notranslate">pip</code>, source):</li>
<li>Build command you used (if compiling from source):</li>
<li>Python version:</li>
<li>CUDA/cuDNN version:</li>
<li>GPU models and configuration:</li>
<li>Any other relevant information:</li>
</ul>
<h2 dir="auto">Additional context</h2>