rajabmondal commited on
Commit
3b3128b
·
verified ·
1 Parent(s): d669b73

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +10 -0
README.md CHANGED
@@ -256,7 +256,17 @@ output = llm(
256
  stop=["</s>"], # Example stop token - not necessarily correct for this specific model! Please check before using.
257
  echo=True # Whether to echo the prompt
258
  )
 
 
 
 
 
 
 
 
 
259
 
 
260
  ```
261
 
262
  ## How to use with LangChain
 
256
  stop=["</s>"], # Example stop token - not necessarily correct for this specific model! Please check before using.
257
  echo=True # Whether to echo the prompt
258
  )
259
+ ```
260
+
261
+ #### Simple example code to load one of these GGUF models
262
+
263
+ ```python
264
+ from ctransformers import AutoModelForCausalLM
265
+
266
+ # Set gpu_layers to the number of layers to offload to GPU. Set to 0 if no GPU acceleration is available on your system.
267
+ llm = AutoModelForCausalLM.from_pretrained("infosys/NT-Java-1.1B-GGUF", model_file="NT-Java-1.1B_Q4_K_M.gguf", model_type="gpt_bigcode", gpu_layers=50)
268
 
269
+ print(llm("AI is going to"))
270
  ```
271
 
272
  ## How to use with LangChain