<h2 dir="auto">Environment info</h2>
<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 3.5.1</li>
<li>Platform: Colab Notebook</li>
<li>Python version: 3.6.9</li>
<li>PyTorch version (GPU?): 1.7.0</li>
<li>Using GPU in script?: Yes</li>
<li>Using distributed or parallel set-up in script?: No</li>
</ul>
<h3 dir="auto">Who can help</h3>
<p dir="auto">tokenizers: <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/mfuntowicz/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/mfuntowicz">@mfuntowicz</a><br>
--&gt;</p>
<h2 dir="auto">Information</h2>
<p dir="auto">Model I am using : DPRReaderTokenizer</p>
<p dir="auto">The problem arises when using:<br>
DPRReaderTokenizer, instead of returning as many tensor as passages, he returns only one (On the documentation it must return (n_passages, sequence_length), but it returns (1, sequence_length) on basic examples.</p>
<p dir="auto">The tasks I am working on is:</p>
<ul dir="auto">
<li>Tokenization with DPRReaderTokenizer on multiple passages (texts)</li>
</ul>
<h2 dir="auto">To reproduce</h2>
<p dir="auto">from transformers import AlbertTokenizer, AlbertForQuestionAnswering, DPRReader, DPRReaderTokenizer, AutoTokenizer<br>
import torch<br>
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")</p>
<p dir="auto">tokenizer_DPR = DPRReaderTokenizer.from_pretrained('facebook/dpr-reader-single-nq-base')<br>
model_DPR = DPRReader.from_pretrained('facebook/dpr-reader-single-nq-base', return_dict=True).cuda()</p>
<p dir="auto">encoded_inputs = tokenizer_DPR(<br>
questions=["What is Transformers?"],<br>
titles=['Attention is all you need', 'One famous library'],<br>
texts=['Attention is a new mechanism designed to improve the performance of the seq2seq models', 'One of the most<br>
famous NLP library is called Transformers' ],<br>
padding=True,<br>
return_tensors='pt'<br>
)</p>
<p dir="auto">encoded_inputs<br>
{'input_ids': tensor([[  101,  2054,  2003, 19081,  1029,   102,  3086,  2003,  2035,  2017,<br>
2342,   102,  3086,  2003,  1037,  2047,  7337,  2881,  2000,  5335,<br>
1996,  2836,  1997,  1996,  7367,  4160,  2475,  3366,  4160,  4275]]), 'attention_mask': tensor([[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,<br>
1, 1, 1, 1, 1, 1]])}</p>
<h2 dir="auto">Remarks</h2>
<p dir="auto">The expected outputs is two tensors ... and I got only one .. <g-emoji class="g-emoji" alias="-1" fallback-src="https://github.githubassets.com/images/icons/emoji/unicode/1f44e.png">👎</g-emoji></p>
<h2 dir="auto"><strong>First good issue</strong></h2>
<p dir="auto">A current error is that a user forwards a batched tensor of <code class="notranslate">input_ids</code> that include a padding token, e.g. <code class="notranslate">input_ids = torch.tensor([["hello", "this", "is", "a", "long", "string"], ["hello", "&lt;pad&gt;", "&lt;pad&gt;", "&lt;pad&gt;", "&lt;pad&gt;"]]</code></p>
<p dir="auto">In this case, the <code class="notranslate">attention_mask</code> should be provided as well. Otherwise the output hidden_states will be incorrectly computed. This is quite a common silent error IMO.</p>
<p dir="auto">With <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/LysandreJik/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/LysandreJik">@LysandreJik</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sgugger/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sgugger">@sgugger</a>, we have decided to not <strong>automatically</strong> create the <code class="notranslate">attention_mask</code> that masks out the padding tokens in this case because of the reasons explains here: <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1121908183" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/15479" data-hovercard-type="issue" data-hovercard-url="/huggingface/transformers/issues/15479/hovercard?comment_id=1066639938&amp;comment_type=issue_comment" href="https://github.com/huggingface/transformers/issues/15479#issuecomment-1066639938">#15479 (comment)</a> . However as pointed out in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="1121908183" data-permission-text="Title is private" data-url="https://github.com/huggingface/transformers/issues/15479" data-hovercard-type="issue" data-hovercard-url="/huggingface/transformers/issues/15479/hovercard" href="https://github.com/huggingface/transformers/issues/15479">#15479</a>, we should IMO at least displa a warning since this error happens a lot IMO.</p>
<p dir="auto">As a first good issue, one could add such a warning to the BertModel in a first case which would go something like:</p>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="if attention_mask is not None and (input_ids == pad_token_id).any():
    logger.warn(&quot;display nice warning here....&quot;)"><pre class="notranslate"><span class="pl-k">if</span> <span class="pl-s1">attention_mask</span> <span class="pl-c1">is</span> <span class="pl-c1">not</span> <span class="pl-c1">None</span> <span class="pl-c1">and</span> (<span class="pl-s1">input_ids</span> <span class="pl-c1">==</span> <span class="pl-s1">pad_token_id</span>).<span class="pl-en">any</span>():
    <span class="pl-s1">logger</span>.<span class="pl-en">warn</span>(<span class="pl-s">"display nice warning here...."</span>)</pre></div>
<p dir="auto">What do you think <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/sgugger/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/sgugger">@sgugger</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/LysandreJik/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/LysandreJik">@LysandreJik</a> ?</p>