<h3 dir="auto">System Info</h3>
<p dir="auto">working:</p>
<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 4.26.1</li>
<li>Platform: Linux-6.0.12-x86_64-with-glibc2.10</li>
<li>Python version: 3.8.16</li>
<li>Huggingface_hub version: 0.12.0</li>
<li>PyTorch version (GPU?): 1.13.1+cu117 (True)</li>
<li>Tensorflow version (GPU?): not installed (NA)</li>
<li>Flax version (CPU?/GPU?/TPU?): not installed (NA)</li>
<li>Jax version: not installed</li>
<li>JaxLib version: not installed</li>
<li>Using GPU in script?: yes</li>
<li>Using distributed or parallel set-up in script?: no</li>
</ul>
<p dir="auto">broken:</p>
<ul dir="auto">
<li><code class="notranslate">transformers</code> version: 4.27.0.dev0</li>
<li>Platform: Linux-6.0.12-x86_64-with-glibc2.10</li>
<li>Python version: 3.8.16</li>
<li>Huggingface_hub version: 0.12.0</li>
<li>PyTorch version (GPU?): 1.13.1+cu117 (True)</li>
<li>Tensorflow version (GPU?): not installed (NA)</li>
<li>Flax version (CPU?/GPU?/TPU?): not installed (NA)</li>
<li>Jax version: not installed</li>
<li>JaxLib version: not installed</li>
<li>Using GPU in script?: yes</li>
<li>Using distributed or parallel set-up in script?: no</li>
</ul>
<h3 dir="auto">Who can help?</h3>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/gante/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/gante">@gante</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/NielsRogge/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/NielsRogge">@NielsRogge</a></p>
<h3 dir="auto">Information</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> The official example scripts</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> My own modified scripts</li>
</ul>
<h3 dir="auto">Tasks</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> An officially supported task in the <code class="notranslate">examples</code> folder (such as GLUE/SQuAD, ...)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> My own task or dataset (give details below)</li>
</ul>
<h3 dir="auto">Reproduction</h3>
<ol dir="auto">
<li>Start with clean env setup via <a href="https://github.com/salesforce/LAVIS/blob/main/requirements.txt">https://github.com/salesforce/LAVIS/blob/main/requirements.txt</a> (transformers-4.26.1)</li>
<li>Run <code class="notranslate">python test_simple.py</code>, model is correctly loaded and prints a caption</li>
<li><code class="notranslate">pip install --upgrade git+https://github.com/huggingface/transformers</code> (I wanted the new shiny blip2 conversion script so I can conver my finetuned model into HF format)</li>
<li><code class="notranslate">Resolved https://github.com/huggingface/transformers to commit 8b3db33a763ccef828fca89bac7e6cbff314f131</code></li>
<li>Run <code class="notranslate">python test_simple.py</code></li>
<li><code class="notranslate">RuntimeError: Sizes of tensors must match except in dimension 1. Expected size 25 but got size 5 for tensor number 1 in the list.</code></li>
</ol>
<div class="highlight highlight-source-python notranslate position-relative overflow-auto" dir="auto" data-snippet-clipboard-copy-content="import torch
from lavis.models import load_model_and_preprocess
import torch
from PIL import Image
import requests

device = torch.device(&quot;cuda&quot; if torch.cuda.is_available() else &quot;cpu&quot;)
model, vis_processors, _ = load_model_and_preprocess(name=&quot;blip2_opt&quot;, model_type=&quot;caption_coco_opt6.7b&quot;, is_eval=True, device=device)

url = &quot;...&quot;
raw_image = Image.open(requests.get(url, stream=True).raw).convert(&quot;RGB&quot;)
image = vis_processors[&quot;eval&quot;](raw_image).unsqueeze(0).to(device)
data = model.generate({&quot;image&quot;: image})
print(data)"><pre class="notranslate"><span class="pl-k">import</span> <span class="pl-s1">torch</span>
<span class="pl-k">from</span> <span class="pl-s1">lavis</span>.<span class="pl-s1">models</span> <span class="pl-k">import</span> <span class="pl-s1">load_model_and_preprocess</span>
<span class="pl-k">import</span> <span class="pl-s1">torch</span>
<span class="pl-k">from</span> <span class="pl-v">PIL</span> <span class="pl-k">import</span> <span class="pl-v">Image</span>
<span class="pl-k">import</span> <span class="pl-s1">requests</span>

<span class="pl-s1">device</span> <span class="pl-c1">=</span> <span class="pl-s1">torch</span>.<span class="pl-en">device</span>(<span class="pl-s">"cuda"</span> <span class="pl-k">if</span> <span class="pl-s1">torch</span>.<span class="pl-s1">cuda</span>.<span class="pl-en">is_available</span>() <span class="pl-k">else</span> <span class="pl-s">"cpu"</span>)
<span class="pl-s1">model</span>, <span class="pl-s1">vis_processors</span>, <span class="pl-s1">_</span> <span class="pl-c1">=</span> <span class="pl-en">load_model_and_preprocess</span>(<span class="pl-s1">name</span><span class="pl-c1">=</span><span class="pl-s">"blip2_opt"</span>, <span class="pl-s1">model_type</span><span class="pl-c1">=</span><span class="pl-s">"caption_coco_opt6.7b"</span>, <span class="pl-s1">is_eval</span><span class="pl-c1">=</span><span class="pl-c1">True</span>, <span class="pl-s1">device</span><span class="pl-c1">=</span><span class="pl-s1">device</span>)

<span class="pl-s1">url</span> <span class="pl-c1">=</span> <span class="pl-s">"..."</span>
<span class="pl-s1">raw_image</span> <span class="pl-c1">=</span> <span class="pl-v">Image</span>.<span class="pl-en">open</span>(<span class="pl-s1">requests</span>.<span class="pl-en">get</span>(<span class="pl-s1">url</span>, <span class="pl-s1">stream</span><span class="pl-c1">=</span><span class="pl-c1">True</span>).<span class="pl-s1">raw</span>).<span class="pl-en">convert</span>(<span class="pl-s">"RGB"</span>)
<span class="pl-s1">image</span> <span class="pl-c1">=</span> <span class="pl-s1">vis_processors</span>[<span class="pl-s">"eval"</span>](<span class="pl-s1">raw_image</span>).<span class="pl-en">unsqueeze</span>(<span class="pl-c1">0</span>).<span class="pl-en">to</span>(<span class="pl-s1">device</span>)
<span class="pl-s1">data</span> <span class="pl-c1">=</span> <span class="pl-s1">model</span>.<span class="pl-en">generate</span>({<span class="pl-s">"image"</span>: <span class="pl-s1">image</span>})
<span class="pl-en">print</span>(<span class="pl-s1">data</span>)</pre></div>
<h3 dir="auto">Expected behavior</h3>
<p dir="auto">Can use BLIP2 with latest HF</p>
<h3 dir="auto">System Info</h3>
<p dir="auto"><code class="notranslate">transformers</code> version: 4.27.0.dev0</p>
<h3 dir="auto">Who can help?</h3>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/NielsRogge/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/NielsRogge">@NielsRogge</a></p>
<h3 dir="auto">Information</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> The official example scripts</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> My own modified scripts</li>
</ul>
<h3 dir="auto">Tasks</h3>
<ul class="contains-task-list">
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox"> An officially supported task in the <code class="notranslate">examples</code> folder (such as GLUE/SQuAD, ...)</li>
<li class="task-list-item"><input type="checkbox" id="" disabled="" class="task-list-item-checkbox" checked=""> My own task or dataset (give details below)</li>
</ul>
<h3 dir="auto">Reproduction</h3>
<p dir="auto">The following error happens when running BLIP-2 model's generate() with num_beams&gt;1 and input batch_size&gt;1</p>
<p dir="auto"><a target="_blank" rel="noopener noreferrer nofollow" href="https://user-images.githubusercontent.com/13638455/218443029-8bc45f00-4884-4785-95a9-228aa08d1266.png"><img width="892" alt="Screenshot 2023-02-13 at 6 53 28 pm" src="https://user-images.githubusercontent.com/13638455/218443029-8bc45f00-4884-4785-95a9-228aa08d1266.png" style="max-width: 100%;"></a></p>
<h3 dir="auto">Expected behavior</h3>
<p dir="auto">The model should be able to do batch generate.</p>