Update README.md

README.md

@@ -56,18 +56,6 @@ We also provide the source code and the model weight for the original demo, allo
 	python demo.py -c echo840/Monkey 
 	```
 
-In order to generate more detailed captions, we provide some prompt examples so that you can conduct more interesting explorations. You can modify these two variables in the `caption` function to implement different prompt inputs for the caption task, as shown below:
-```
-query = "Generate the detailed caption in English: "
-chat_query = "Generate the detailed caption in English: "
-```
-- Generate the detailed caption in English.
-- Explain the visual content of the image in great detail.
-- Analyze the image in a comprehensive and detailed manner.
-- Describe the image in as much detail as possible in English without duplicating it.
-- Describe the image in as much detail as possible in English, including as many elements from the image as possible, but without repetition.
-
-
 ## Dataset
 
 We have open-sourced the data generated by the multi-level description generation method. You can download it at [Detailed Caption](https://huggingface.co/datasets/echo840/Detailed_Caption).
@@ -122,6 +110,42 @@ We also offer Monkey's model definition and training code, which you can explore
 **ATTENTION:** Specify the path to your training data, which should be a json file consisting of a list of conversations.
 
 
+## Inference
+
+```python
+from transformers import AutoModelForCausalLM, AutoTokenizer
+checkpoint = "echo840/Monkey"
+model = AutoModelForCausalLM.from_pretrained(checkpoint, device_map='cuda', trust_remote_code=True).eval()
+tokenizer = AutoTokenizer.from_pretrained(checkpoint, trust_remote_code=True)
+tokenizer.padding_side = 'left'
+tokenizer.pad_token_id = tokenizer.eod_id
+img_path = ""
+question = ""
+query = f'<img>{img_path}</img> {question} Answer: ' #VQA
+# query = f'<img>{img_path}</img> Generate the detailed caption in English: ' #detailed caption
+
+input_ids = tokenizer(query, return_tensors='pt', padding='longest')
+attention_mask = input_ids.attention_mask
+input_ids = input_ids.input_ids
+
+pred = model.generate(
+            input_ids=input_ids.cuda(),
+            attention_mask=attention_mask.cuda(),
+            do_sample=False,
+            num_beams=1,
+            max_new_tokens=512,
+            min_new_tokens=1,
+            length_penalty=1,
+            num_return_sequences=1,
+            output_hidden_states=True,
+            use_cache=True,
+            pad_token_id=tokenizer.eod_id,
+            eos_token_id=tokenizer.eod_id,
+            )
+response = tokenizer.decode(pred[0][input_ids.size(1):].cpu(), skip_special_tokens=True).strip()
+print(response)
+```
+
 
 ## Citing Monkey
 If you wish to refer to the baseline results published here, please use the following BibTeX entries: