05_deploying_a_chatbot_to_the_web.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8ec2fef2",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Deploying a Chatbot to the Web\n",
    "* **Created by:** Eric Martinez\n",
    "* **For:** Software Engineering 2\n",
    "* **At:** University of Texas Rio-Grande Valley"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ffb051ff",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## HuggingFace\n",
    "\n",
    "HuggingFace is an AI research organization and platform that provides access to a wide range of pre-trained LLMs and tools for training, fine-tuning, and deploying models. It has a user-friendly interface and a large community, making it a popular choice for working with LLMs."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b3aec8b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Deploying to HuggingFace"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7804a8ce",
   "metadata": {},
   "source": [
    "#### Configuring the files required"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "60c8e7f6",
   "metadata": {},
   "source": [
    "Let's face it! Once we start building cool stuff we are going to want to show it off. It can take us < 10 minutes to deploy our chatbots and LLM applications when using Gradio!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "54de0ddc",
   "metadata": {},
   "source": [
    "Add a username and password for your app to your `.env` file. This will ensure that unauthorized users are not able to access LLM features. Use the following format:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "95dec7cf",
   "metadata": {},
   "outputs": [],
   "source": [
    "APP_USERNAME=<whatever username you want>\n",
    "APP_PASSWORD=<whatever password you want>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5072dc21",
   "metadata": {},
   "source": [
    "Let's start by taking all of our necessary chatbot code into one file which we will name `app.py`. Run the following cell to automatically write it!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aacdcaaf",
   "metadata": {},
   "source": [
    "Take note that this code has been altered a little bit from the last chatbot example in order to add authentication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "710b66f7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting app.py\n"
     ]
    }
   ],
   "source": [
    "%%writefile app.py\n",
    "import gradio as gr\n",
    "import openai\n",
    "import examples as chatbot_examples\n",
    "from dotenv import load_dotenv\n",
    "import os\n",
    "\n",
    "load_dotenv()  # take environment variables from .env.\n",
    "\n",
    "# In order to authenticate, secrets must have been set, and the user supplied credentials match\n",
    "def auth(username, password):\n",
    "    app_username = os.getenv(\"APP_USERNAME\")\n",
    "    app_password = os.getenv(\"APP_PASSWORD\")\n",
    "\n",
    "    if app_username and app_password:\n",
    "        if(username == app_username and password == app_password):\n",
    "            print(\"Logged in successfully.\")\n",
    "            return True\n",
    "        else:\n",
    "            print(\"Username or password does not match.\")\n",
    "    else:\n",
    "        print(\"Credential secrets not set.\")\n",
    "    return False\n",
    "    \n",
    "# Define a function to get the AI's reply using the OpenAI API\n",
    "def get_ai_reply(message, model=\"gpt-3.5-turbo\", system_message=None, temperature=0, message_history=[]):\n",
    "    # Initialize the messages list\n",
    "    messages = []\n",
    "    \n",
    "    # Add the system message to the messages list\n",
    "    if system_message is not None:\n",
    "        messages += [{\"role\": \"system\", \"content\": system_message}]\n",
    "\n",
    "    # Add the message history to the messages list\n",
    "    if message_history is not None:\n",
    "        messages += message_history\n",
    "    \n",
    "    # Add the user's message to the messages list\n",
    "    messages += [{\"role\": \"user\", \"content\": message}]\n",
    "    \n",
    "    # Make an API call to the OpenAI ChatCompletion endpoint with the model and messages\n",
    "    completion = openai.ChatCompletion.create(\n",
    "        model=model,\n",
    "        messages=messages,\n",
    "        temperature=temperature\n",
    "    )\n",
    "    \n",
    "    # Extract and return the AI's response from the API response\n",
    "    return completion.choices[0].message.content.strip()\n",
    "\n",
    "# Define a function to handle the chat interaction with the AI model\n",
    "def chat(model, system_message, message, chatbot_messages, history_state):\n",
    "    # Initialize chatbot_messages and history_state if they are not provided\n",
    "    chatbot_messages = chatbot_messages or []\n",
    "    history_state = history_state or []\n",
    "    \n",
    "    # Try to get the AI's reply using the get_ai_reply function\n",
    "    try:\n",
    "        ai_reply = get_ai_reply(message, model=model, system_message=system_message, message_history=history_state)\n",
    "    except Exception as e:\n",
    "        # If an error occurs, raise a Gradio error\n",
    "        raise gr.Error(e)\n",
    "    \n",
    "    # Append the user's message and the AI's reply to the chatbot_messages list\n",
    "    chatbot_messages.append((message, ai_reply))\n",
    "    \n",
    "    # Append the user's message and the AI's reply to the history_state list\n",
    "    history_state.append({\"role\": \"user\", \"content\": message})\n",
    "    history_state.append({\"role\": \"assistant\", \"content\": ai_reply})\n",
    "    \n",
    "    # Return None (empty out the user's message textbox), the updated chatbot_messages, and the updated history_state\n",
    "    return None, chatbot_messages, history_state\n",
    "\n",
    "# Define a function to launch the chatbot interface using Gradio\n",
    "def get_chatbot_app(additional_examples=[]):\n",
    "    # Load chatbot examples and merge with any additional examples provided\n",
    "    examples = chatbot_examples.load_examples(additional=additional_examples)\n",
    "    \n",
    "    # Define a function to get the names of the examples\n",
    "    def get_examples():\n",
    "        return [example[\"name\"] for example in examples]\n",
    "\n",
    "    # Define a function to choose an example based on the index\n",
    "    def choose_example(index):\n",
    "        if(index!=None):\n",
    "            system_message = examples[index][\"system_message\"].strip()\n",
    "            user_message = examples[index][\"message\"].strip()\n",
    "            return system_message, user_message, [], []\n",
    "        else:\n",
    "            return \"\", \"\", [], []\n",
    "\n",
    "    # Create the Gradio interface using the Blocks layout\n",
    "    with gr.Blocks() as app:\n",
    "        with gr.Tab(\"Conversation\"):\n",
    "            with gr.Row():\n",
    "                with gr.Column():\n",
    "                    # Create a dropdown to select examples\n",
    "                    example_dropdown = gr.Dropdown(get_examples(), label=\"Examples\", type=\"index\")\n",
    "                    # Create a button to load the selected example\n",
    "                    example_load_btn = gr.Button(value=\"Load\")\n",
    "                    # Create a textbox for the system message (prompt)\n",
    "                    system_message = gr.Textbox(label=\"System Message (Prompt)\", value=\"You are a helpful assistant.\")\n",
    "                with gr.Column():\n",
    "                    # Create a dropdown to select the AI model\n",
    "                    model_selector = gr.Dropdown(\n",
    "                        [\"gpt-3.5-turbo\"],\n",
    "                        label=\"Model\",\n",
    "                        value=\"gpt-3.5-turbo\"\n",
    "                    )\n",
    "                    # Create a chatbot interface for the conversation\n",
    "                    chatbot = gr.Chatbot(label=\"Conversation\")\n",
    "                    # Create a textbox for the user's message\n",
    "                    message = gr.Textbox(label=\"Message\")\n",
    "                    # Create a state object to store the conversation history\n",
    "                    history_state = gr.State()\n",
    "                    # Create a button to send the user's message\n",
    "                    btn = gr.Button(value=\"Send\")\n",
    "\n",
    "                # Connect the example load button to the choose_example function\n",
    "                example_load_btn.click(choose_example, inputs=[example_dropdown], outputs=[system_message, message, chatbot, history_state])\n",
    "                # Connect the send button to the chat function\n",
    "                btn.click(chat, inputs=[model_selector, system_message, message, chatbot, history_state], outputs=[message, chatbot, history_state])\n",
    "        # Return the app\n",
    "        return app\n",
    "        \n",
    "# Call the launch_chatbot function to start the chatbot interface using Gradio\n",
    "# Set the share parameter to False, meaning the interface will not be publicly accessible\n",
    "app = get_chatbot_app((\n",
    "app.queue()  # <-- Sets up a queue with default parameters\n",
    "    \n",
    "launch(auth=auth)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6d75af66",
   "metadata": {},
   "source": [
    "We will also need a `requirements.txt` file to store the list of the packages that HuggingFace needs to install to run our chatbot."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "14d0e434",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Overwriting requirements.txt\n"
     ]
    }
   ],
   "source": [
    "%%writefile requirements.txt\n",
    "gradio == 3.27.0\n",
    "openai == 0.27.4\n",
    "python-dotenv == 1.0.0"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4debec45",
   "metadata": {},
   "source": [
    "Now let's go ahead and commit our changes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "14d42a96",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git add app.py"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "d7c5b127",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git add requirements.txt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "18960d9f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[master (root-commit) 39899ef] adding chatbot\r\n",
      " 2 files changed, 101 insertions(+)\r\n",
      " create mode 100644 app.py\r\n",
      " create mode 100644 requirements.txt\r\n"
     ]
    }
   ],
   "source": [
    "!git commit -m \"adding chatbot\""
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09221ee0",
   "metadata": {},
   "source": [
    "#### Using HuggingFace Spaces"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "db789c94",
   "metadata": {},
   "source": [
    "As mentioned before, HuggingFace is a free-to-use platform for hosting AI demos and apps. We will need to make a HuggingFace _Space_ for our chatbot."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d9eedd10",
   "metadata": {},
   "source": [
    "First sign up for a free HuggingFace account [here](https://huggingface.co/join). After you sign up, create a new Space by clicking \"New Space\" on the navigation menu (press on your profile image)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3e042d24",
   "metadata": {},
   "source": [
    "#### Generate a HuggingFace Access Token"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7d0781d",
   "metadata": {},
   "source": [
    "#### Login to HuggingFace Hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "eba83252",
   "metadata": {},
   "source": [
    "Install `huggingface_hub`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "266bf481",
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip -q install --upgrade huggingface_hub"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d50cd84b",
   "metadata": {},
   "source": [
    "Login to HuggingFace"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "53fd5037",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Token is valid.\n",
      "Your token has been saved in your configured git credential helpers (osxkeychain).\n",
      "Your token has been saved to /Users/ericmartinez/.cache/huggingface/token\n",
      "Login successful\n"
     ]
    }
   ],
   "source": [
    "from huggingface_hub import notebook_login\n",
    "notebook_login()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90f9bd4d",
   "metadata": {},
   "source": [
    "#### Now lets setup git and HuggingFace Spaces to work together and deploy"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "66468481",
   "metadata": {},
   "source": [
    "<span style=\"color:red\">REPLACE MY URL WITH YOURS</span>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "827a201d",
   "metadata": {},
   "outputs": [],
   "source": [
    "!git remote add huggingface https://huggingface.co/spaces/ericmichael/gradio-chatbot-demo"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8b3bb3d",
   "metadata": {},
   "source": [
    "Then force push to sync everything for the first time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "86c9ee4e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Total 0 (delta 0), reused 0 (delta 0), pack-reused 0\n",
      "To https://huggingface.co/spaces/ericmichael/gradio-chatbot-demo\n",
      " + 8911ec0...3693bcc main -> main (forced update)\n"
     ]
    }
   ],
   "source": [
    "!git push --force huggingface main"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "3a353ebf",
   "metadata": {},
   "source": [
    "That's it! 🎉 Check your HuggingFace Space URL to access your chatbot!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5a54132b",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "## Deploying using FastAPI"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "181dd4ad",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "celltoolbar": "Raw Cell Format",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}