Thinking=True on GGUF?

#7
by MrDevolver - opened

How do we set thinking parameter to true on GGUF? 🤔

IBM Granite org

Hi! We're actively working on an Ollama model with the corresponding go template. Ultimately, enabling thinking is a matter of enabling the right section of system prompt, so in the meantime you can use apply_chat_template on the client side, then use the expanded string with raw generate.

IBM Granite org

The draft Ollama model is now public: https://ollama.com/gabegoodhart/granite3.2-preview

Sign up or log in to comment