File size: 3,083 Bytes
3180b4a d73f096 3180b4a |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 |
{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# AnyCoder Demo Notebook\n",
"\n",
"This notebook illustrates how to:\n",
"\n",
"1. **Instantiate** the unified `hf_client` with automatic provider routing.\n",
"2. **Call** a chat completion (Groq → OpenAI → Gemini fall‑back).\n",
"3. **Trigger** the FastAPI `/predict` endpoint served by *app.py*.\n",
"4. **Run** a quick sentiment‑analysis pipeline using your preferred provider."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# 1. Setup inference client\n",
"from hf_client import get_inference_client\n",
"\n",
"# Choose a model (will route to best provider according to prefix rules)\n",
"model_id = 'openai/gpt-4' # try 'gemini/pro' or any HF model path\n",
"client = get_inference_client(model_id, provider='auto')\n",
"\n",
"# Simple chat completion\n",
"resp = client.chat.completions.create(\n",
" model=model_id,\n",
" messages=[{'role': 'user', 'content': 'Write a Python function to reverse a string.'}]\n",
")\n",
"print(resp.choices[0].message.content)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# 2. Sentiment analysis via HF Inference Providers (OpenAI GPT‑4)\n",
"from transformers import pipeline\n",
"\n",
"sentiment = pipeline(\n",
" 'sentiment-analysis', \n",
" model='openai/gpt-4', # could be 'gemini/pro' etc.\n",
" trust_remote_code=True\n",
")\n",
"sentiment('I love building AI‑powered tools!')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# 3. Call the FastAPI /predict endpoint exposed by app.py\n",
"import json, requests\n",
"\n",
"payload = {\n",
" 'prompt': 'Generate a minimal HTML page.',\n",
" 'model_id': 'gemini/pro',\n",
" 'language': 'html',\n",
" 'web_search': False\n",
"}\n",
"\n",
"r = requests.post('http://localhost:7860/predict', json=payload)\n",
"print('Status:', r.status_code)\n",
"print(json.loads(r.text)['code'][:400])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---\n",
"## Next steps\n",
"\n",
"* Switch `model_id` to **`'gemini/pro'`**, **`'fireworks-ai/fireworks-v1'`**, or any HF model (e.g. `Qwen/Qwen3-32B`)—routing will adjust automatically.\n",
"* Explore **`plugins.py`** for Slack / GitHub integrations.\n",
"* Use **`auth.py`** helpers to pull private Google Drive docs into the pipeline.\n",
"* Extend `/predict` with temperature, max‑tokens, or stream support."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"name": "python",
"version": "3.x"
}
},
"nbformat": 4,
"nbformat_minor": 5
}
|