Delete InteractiveManual.ipynb

InteractiveManual.ipynb

@@ -1,356 +0,0 @@
-{
-  "cells": [
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "B3S1AZp6Kc0R"
-      },
-      "source": [
-        "# Installing\n",
-        "\n",
-        "If running on Google Colab you will need a Colab Pro+ subscription.\n",
-        "\n",
-        "Change the runtime type to a high memory or connect your local machine to colab, launch a vm perhaps.\n",
-        "\n",
-        "First, you will need to obtain the LLaMA weights. \n",
-        "\n",
-        "You can sign up for the official weights here: https://huggingface.co/docs/transformers/main/model_doc/llama\n",
-        "\n",
-        "There are alternative models available on huggingface however. This guide will assume you do not have access to the official weights.\n",
-        "\n",
-        "If you do have access to the official weights skip to: Clone the delta weights"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "8_MM5GNkKc0T"
-      },
-      "source": [
-        "## Clone the LLaMA weights"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "5TioBc7WKc0U"
-      },
-      "outputs": [],
-      "source": [
-        "# Setup git lfs\n",
-        "!git lfs install --skip-smudge --force\n",
-        "!git lfs env\n",
-        "!git config filter.lfs.process \"git-lfs filter-process --skip\"\n",
-        "!git config filter.lfs.smudge \"git-lfs smudge --skip -- %f\""
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "B5Q-x5yBKc0V"
-      },
-      "outputs": [],
-      "source": [
-        "# Cloning the 7b parameter model repo\n",
-        "!git lfs clone https://huggingface.co/decapoda-research/llama-7b-hf"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "d0r8HBUyKc0V"
-      },
-      "outputs": [],
-      "source": [
-        "# Cloning the 13b parameter model repo\n",
-        "!git lfs clone https://huggingface.co/decapoda-research/llama-13b-hf"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "QeDIT-a4Kc0V"
-      },
-      "source": [
-        "## Applying the Vicuna delta weights"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "p7BGGUICKc0W"
-      },
-      "source": [
-        "### Install PyTorch with CUDA support\n",
-        "\n",
-        "If you already have this installed in your environment, you can skip this step."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "IfGE0ALrKc0W"
-      },
-      "outputs": [],
-      "source": [
-        "# First we need to upgrade setuptools, pip and wheel\n",
-        "!pip install --upgrade setuptools pip wheel"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "RpvAYAD8Kc0W"
-      },
-      "outputs": [],
-      "source": [
-        "# For CUDA 11.X:\n",
-        "!pip install nvidia-cuda-runtime-cu11 --index-url https://pypi.ngc.nvidia.com"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "1oEi1ggaKc0X"
-      },
-      "outputs": [],
-      "source": [
-        "# For CUDA 12.x\n",
-        "!pip install nvidia-cuda-runtime-cu12 --index-url https://pypi.ngc.nvidia.com"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "fQYYrn4tKc0X"
-      },
-      "outputs": [],
-      "source": [
-        "# For PyTorch cu117\n",
-        "!pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu117"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "G7lnb0sYKc0X"
-      },
-      "outputs": [],
-      "source": [
-        "# For PyTorch cu118\n",
-        "!pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "oBVyjHuWKc0Y"
-      },
-      "source": [
-        "### Running the Fast-Chat apply delta script"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "a4b11Z8lKc0Y"
-      },
-      "outputs": [],
-      "source": [
-        "# Install FastChat\n",
-        "!pip install fschat\n",
-        "\n",
-        "# Install the latest main branch of huggingface/transformers\n",
-        "!pip install git+https://github.com/huggingface/transformers"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "3vJTP1m0Kc0Y"
-      },
-      "outputs": [],
-      "source": [
-        "import json\n",
-        "\n",
-        "def convert_llama_names(model_name: str) -> None:\n",
-        "    \"\"\"Convert LlamaForCausalLM to LlamaForCausalLM and LLaMATokenizer to LlamaTokenizer\"\"\"\n",
-        "    with open(f\"{model_name}/config.json\", \"r\", encoding='utf-8') as f:\n",
-        "        data = f.read()\n",
-        "\n",
-        "    config = json.loads(data)\n",
-        "    config[\"architectures\"] = [\"LlamaForCausalLM\"]\n",
-        "    with open(f\"{model_name}/config.json\", \"w\", encoding='utf-8') as f:\n",
-        "        json.dump(config, f)\n",
-        "\n",
-        "\n",
-        "    with open(f\"{model_name}/tokenizer_config.json\", \"r\", encoding='utf-8') as f:\n",
-        "        data = f.read()\n",
-        "\n",
-        "    config = json.loads(data)\n",
-        "    config[\"tokenizer_class\"] = \"LlamaTokenizer\"\n",
-        "\n",
-        "    with open(f\"{model_name}/tokenizer_config.json\", \"w\", encoding='utf-8') as f:\n",
-        "        json.dump(config, f)"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "qyboxxkXKc0Y"
-      },
-      "outputs": [],
-      "source": [
-        "!git lfs clone https://huggingface.co/lmsys/vicuna-7b-delta-v1.1"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "9tlvfvapKc0Z"
-      },
-      "outputs": [],
-      "source": [
-        "# 7b Model\n",
-        "convert_llama_names(\"llama-7b-hf\")\n",
-        "!python -m fastchat.model.apply_delta --base llama-7b-hf --target vicuna-7b --delta ./vicuna-7b-delta-v1.1"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "5VzAdoySKc0Z"
-      },
-      "outputs": [],
-      "source": [
-        "!git lfs clone https://huggingface.co/lmsys/vicuna-13b-delta-v1.1"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "jRBc7q5RKc0Z"
-      },
-      "outputs": [],
-      "source": [
-        "# 13b\n",
-        "convert_llama_names(\"llama-13b-hf\")\n",
-        "!python -m fastchat.model.apply_delta --base llama-13b-hf --target vicuna-13b --delta ./vicuna-13b-delta-v1.1"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "7n8Ug87RKc0Z"
-      },
-      "source": [
-        "# Installing Auto-Vicuna\n",
-        "\n",
-        "Note that running this does not work in colab or the notebook, it is for demonstration purposes only."
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "gIlfwNlEKc0Z"
-      },
-      "outputs": [],
-      "source": [
-        "!pip install auto-vicuna"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "0Z5NlQOXKc0Z"
-      },
-      "source": [
-        "# Running Auto-Vicuna"
-      ]
-    },
-    {
-      "cell_type": "code",
-      "execution_count": null,
-      "metadata": {
-        "id": "sBJiQP7MKc0a"
-      },
-      "outputs": [],
-      "source": [
-        "!auto_vicuna --vicuna_weights vicuna-7b"
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "oqqFeF4zKc0a"
-      },
-      "source": [
-        "You can also create a .env file with \n",
-        "\n",
-        "```\n",
-        "VICUNA_WEIGHTS=vicuna-7b\n",
-        "```\n",
-        "\n",
-        "To avoid passing the weights as an arugment."
-      ]
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "jiW98C1KKc0a"
-      },
-      "source": []
-    },
-    {
-      "cell_type": "markdown",
-      "metadata": {
-        "id": "VE8DhmUrKc0a"
-      },
-      "source": [
-        "## Known Issues\n",
-        "\n",
-        "If your model keeps talking about random news articles and suchs the `special_tokens_map.json` and `tokenizer_config.json` need to have to stop tokens populated most likely, you can find them in the repo's root dir."
-      ]
-    }
-  ],
-  "metadata": {
-    "kernelspec": {
-      "display_name": ".venv",
-      "language": "python",
-      "name": "python3"
-    },
-    "language_info": {
-      "codemirror_mode": {
-        "name": "ipython",
-        "version": 3
-      },
-      "file_extension": ".py",
-      "mimetype": "text/x-python",
-      "name": "python",
-      "nbconvert_exporter": "python",
-      "pygments_lexer": "ipython3",
-      "version": "3.9.13"
-    },
-    "orig_nbformat": 4,
-    "colab": {
-      "provenance": []
-    }
-  },
-  "nbformat": 4,
-  "nbformat_minor": 0
-}