Chapter 2. Red Hat Enterprise Linux AI product architecture
Chapter 2. Red Hat Enterprise Linux AI product architecture Red Hat Enterprise Linux AI contains various distinct features and consists of the following components. 2.1. Bootable Red Hat Enterprise Linux with InstructLab You can install RHEL AI and deploy the InstructLab tooling using a bootable RHEL container image provided by Red Hat. This RHEL AI image includes InstructLab, RHEL 9.4, and various inference and training software, including vLLM and DeepSpeed. After you boot this image, you can download various Red Hat and IBM developed Granite models to serve or train. The image and all the tools are compiled to specific Independent Software Vendors (ISV) hardware. For more information about the architecture of the image, see Installation overview . 2.1.1. InstructLab model alignment The Red Hat Enterprise Linux AI bootable image contains InstructLab and its tooling. InstructLab uses a novel approach to LLM fine-tuning called LAB (Large-Scale Alignment for ChatBots). The LAB method uses a taxonomy-based system that implements high-quality synthetic data generation (SDG) and multi-phase training. Using the RHEL AI command line interface (CLI), which is built from the InstructLab CLI, you can create your own custom LLM by tuning a Granite base model on synthetic data generated from your own domain-specific knowledge. For general availability, the RHEL AI LLMs customization workflow consists of the following steps: Installing and initializing RHEL AI on your preferred platform. Using a CLI and Git workflow for adding skills and knowledge to your taxonomy tree. Running synthetic data generation (SDG) using the mixtral-8x7B-Instruct teacher model. SDG can generate hundreds or thousands of synthetic question-and-answer pairs for model tuning based on user-provided specific samples. Using the InstructLab to train the base model with the new synthetically generated data. The prometheus-8x7B-V2.0 judge model evaluates the performance of the newly trained model. Using InstructLab with vLLM to serve the new custom model for inferencing. 2.1.2. Open source licensed Granite models With RHEL AI, you can download the open source licensed IBM Granite family of LLMs. Using the starter Granite model as a base, you can create your model using knowledge or skills data. You can keep these custom LLMs private or you can share them with the AI community. Red Hat Enterprise Linux AI also allows you to serve and chat with Granite models created and fine-tuned by Red Hat and IBM.