RuntimeError: cu_seqlens_q must have dtype int32
#59
by
ginnyyk
- opened
I've got a runtime error for inference.
generated_ids = model.generate(**inputs, max_new_tokens=128)
Can anyone help with it? Thank you.
pip install transformers==4.46.3
This worked for me