LoupGarou commited on
Commit
2feec28
·
verified ·
1 Parent(s): b26f379

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -6
README.md CHANGED
@@ -23,14 +23,11 @@ Please ensure you are using one of the above versions when working with this mod
23
 
24
  Many issues related to empty plans, tasks, circular questions, and poor model performance are related to the following parameters:
25
 
26
- 1. **Prompt eval batch size (n_batch)**: Impacts how the instruction is divided and sent to the LLM. To prevent empty tasks, plans, and circular questions, set this to match your Context Length (n_ctx). For example, if your **n_ctx = 8192** then set your prompt eval bacth size to match **n_batch = 8192**.
27
- Warning: If the n_batch < n_ctx then your model will give bad results.
28
 
29
- 2. **Context Length (n_ctx)**: Sets the maximum length of the instruction and truncates the instruction once the limit is exceeded. Set this value to the maximum your hardware can handle and the maximum for the model. For example, DeepSeek Coder has a maximum token length of 16,384.
30
- Warning: GPT Pilot will often create instruction prompts 10,000 to 20,000 tokens in length which is why Pythagora-LLM-Proxy was created to permit toggling to higher capacity APIs such as OpenAI.
31
 
32
- 3. **MAX_TOKENS (GPT Pilot .env)**: Sets the maximum tokens the OpenAI API generate in the output.
33
- Warning: Setting this value too low will result in truncated messages.
34
 
35
  ## Model Details
36
 
 
23
 
24
  Many issues related to empty plans, tasks, circular questions, and poor model performance are related to the following parameters:
25
 
26
+ 1. **Prompt eval batch size (n_batch)**: Impacts how the instruction is divided and sent to the LLM. To prevent empty tasks, plans, and circular questions, set this to match your Context Length (n_ctx). For example, if your **n_ctx = 8192** then set your prompt eval bacth size to match **n_batch = 8192**. Warning: If the n_batch < n_ctx then your model will give bad results.
 
27
 
28
+ 2. **Context Length (n_ctx)**: Sets the maximum length of the instruction and truncates the instruction once the limit is exceeded. Set this value to the maximum your hardware can handle and the maximum for the model. For example, DeepSeek Coder has a maximum token length of 16,384. Warning: GPT Pilot will often create instruction prompts 10,000 to 20,000 tokens in length which is why Pythagora-LLM-Proxy was created to permit toggling to higher capacity APIs such as OpenAI.
 
29
 
30
+ 3. **MAX_TOKENS (GPT Pilot .env)**: Sets the maximum tokens the OpenAI API generate in the output. Warning: Setting this value too low will result in truncated messages.
 
31
 
32
  ## Model Details
33