<h2 dir="auto">Issue description</h2>
<p dir="auto">It is nice that pytorch now has <code class="notranslate">dtype</code> attribute for tensors and a default dtype so that we can easily switch between different precisions. However, as I understand it:</p>
<ol dir="auto">
<li>if we want to define e.g. a small value <code class="notranslate">eps</code> that is the smallest representable positive value of double, right now we have to switch to numpy and use sth like <code class="notranslate">np.finfo(np.float64).tiny</code>.</li>
<li>if we want this value to depend on the current default dtype, we have to call <code class="notranslate">torch.get_default_dtype()</code>, convert it somehow to numpy dtype, then use <code class="notranslate">np.finfo</code>. This must be performed every time we need this number unless we are sure the default dtype is not changed in between.</li>
</ol>
<p dir="auto">It would thus be very nice if we have sth like <code class="notranslate">torch.finfo</code> similar to <code class="notranslate">np.finfo</code> that additionally takes the current default pytorch dtype as the default argument so as to have a hassle-free and efficient way of solving the above problems.</p>
<p dir="auto">If you have two gpu cards, and you want train your model only on the second card.<br>
Then you might write the following code</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="device = torch.device('cuda', 1)
net.cuda(device)"><pre class="notranslate"><code class="notranslate">device = torch.device('cuda', 1)
net.cuda(device)
</code></pre></div>
<p dir="auto">or simply</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="net.cuda(1)"><pre class="notranslate"><code class="notranslate">net.cuda(1)
</code></pre></div>
<p dir="auto">But you can find the first gpu card was also filled with tensors.<br>
This will not happen in v0.3.</p>
<p dir="auto">Currently, my solution to this problem would be</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="with torch.cuda.device(1):
    net.cuda()"><pre class="notranslate"><code class="notranslate">with torch.cuda.device(1):
    net.cuda()
</code></pre></div>
<p dir="auto">Hope this bug will be solved in the future.</p>