Running issue
(llm) C:\Users\nxf76091\Downloads\work\llm\deepseek>python run_model.py
tokenizer_config.json: 100%|█████████████████████████████████████████████████████████████████████████████████████████| 6.72k/6.72k [00:00<00:00, 6.72MB/s]
tokenizer.json: 100%|████████████████████████████████████████████████████████████████████████████████████████████████| 11.4M/11.4M [00:10<00:00, 1.13MB/s]
special_tokens_map.json: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 485/485 [00:00<?, ?B/s]
Traceback (most recent call last):
File "C:\Users\nxf76091\Downloads\work\llm\deepseek\run_model.py", line 86, in
generated = generate_text(model_path, prompt, tokenizer, max_gen_tokens, total_sequence, window, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\nxf76091\Downloads\work\llm\deepseek\run_model.py", line 45, in generate_text
output = rt_session.run(output_names, inputs_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\software\anaconda\envs\llm\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 266, in run
return self._sess.run(output_names, input_feed, run_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: past_key_values.
3.key for the following indices
index: 3 Got: 64 Expected: 128
Please fix either the inputs/outputs or the model.
When I ran this project according to the guide, I encountered the above problem? Can you give me some advice?
Hi,
Thank you, there was a slight mistake in the guide, try it again now!
I have reinstalled the corresponding onnx==1.16.1 and onnxruntime==1.17.1 versions according to the guide, but the following error still occurs during operation. Do you have any suggestions?
Test log:
(llm) C:\Users\nxf76091\Downloads\work\huggingface>python run_model.py
Traceback (most recent call last):
File "C:\Users\nxf76091\Downloads\work\huggingface\run_model.py", line 86, in
generated = generate_text(model_path, prompt, tokenizer, max_gen_tokens, total_sequence, window, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\nxf76091\Downloads\work\huggingface\run_model.py", line 45, in generate_text
output = rt_session.run(output_names, inputs_dict)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\software\anaconda\envs\llm\Lib\site-packages\onnxruntime\capi\onnxruntime_inference_collection.py", line 220, in run
return self._sess.run(output_names, input_feed, run_options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
onnxruntime.capi.onnxruntime_pybind11_state.InvalidArgument: [ONNXRuntimeError] : 2 : INVALID_ARGUMENT : Got invalid dimensions for input: past_key_values.
16.value for the following indices
index: 1 Got: 0 Expected: 2
Please fix either the inputs/outputs or the model.
FIxed! Try again and sorry!
Thanks again.
Thank you for your reply. When the program is running, I found that the memory occupied has reached 12G. Why is it so large? Is this normal? Do you have any suggestions?