Chapter 2. Installing and deploying OpenShift AI
Chapter 2. Installing and deploying OpenShift AI Red Hat OpenShift AI is a platform for data scientists and developers of artificial intelligence (AI) applications. It provides a fully supported environment that lets you rapidly develop, train, test, and deploy machine learning models on-premises and/or in the public cloud. OpenShift AI is provided as a managed cloud service add-on for Red Hat OpenShift or as self-managed software that you can install on-premise or in the public cloud on OpenShift. For information about installing OpenShift AI as self-managed software on your OpenShift cluster in a connected or a disconnected environment, see Product Documentation for Red Hat OpenShift AI Self-Managed . Important Data science pipelines 2.0 contains an installation of Argo Workflows. OpenShift AI does not support direct customer usage of this installation of Argo Workflows. To install OpenShift AI with data science pipelines 2.0, ensure that no separate installation of Argo Workflows exists on your cluster. There are two deployment options for Red Hat OpenShift AI as a managed cloud service add-on: OpenShift Dedicated with a Customer Cloud Subscription on Amazon Web Services or Google Cloud Platform OpenShift Dedicated is a complete OpenShift Container Platform cluster provided as a cloud service, configured for high availability, and dedicated to a single customer. OpenShift Dedicated is professionally managed by Red Hat and hosted on Amazon Web Services (AWS) or Google Cloud Platform (GCP). The Customer Cloud Subscription (CCS) model allows Red Hat to deploy and manage clusters into a customer's AWS or GCP account. Contact your Red Hat account manager to get OpenShift Dedicated through a CCS. Red Hat OpenShift Service on AWS (ROSA Classic) ROSA is a fully-managed, turnkey application platform that allows you to focus on delivering value to your customers by building and deploying applications. You subscribe to the service directly from your AWS account. Installing OpenShift AI as a managed cloud service involves the following high-level tasks: Confirm that your OpenShift cluster meets all requirements. Configure an identity provider for your OpenShift cluster. Add administrative users for your OpenShift cluster. Subscribe to the Red Hat OpenShift AI Add-on. For OpenShift Dedicated with a CCS for AWS or GCP, get a subscription through Red Hat. For ROSA Classic, get a subscription through the AWS Marketplace. Install the Red Hat OpenShift AI Add-on. Access the OpenShift AI dashboard. Optionally, configure and enable your accelerators in OpenShift AI to ensure that your data scientists can use compute-heavy workloads in their models. See Enabling accelerators . 2.1. Requirements for OpenShift AI You must meet the following requirements before you can install OpenShift AI on your Red Hat OpenShift Dedicated or Red Hat OpenShift Service on Amazon Web Services (ROSA Classic) cluster: A subscription for Red Hat OpenShift Dedicated or a subscription for ROSA You can deploy Red Hat OpenShift Dedicated on your Amazon Web Services (AWS) or Google Cloud Platform (GCP) account by using the Customer Cloud Subscription on AWS or Customer Cloud Subscription on GCP model. Note that while Red Hat provides an option to install OpenShift Dedicated on a Red Hat cloud account, if you want to install OpenShift AI then you must install OpenShift Dedicated on your own cloud account. Contact your Red Hat account manager to purchase a new Red Hat OpenShift Dedicated subscription. If you do not yet have an account manager, complete the form at https://cloud.redhat.com/products/dedicated/contact/ to request one. You can subscribe to Red Hat OpenShift Service on AWS (ROSA Classic) directly from your AWS account or by contacting your Red Hat account manager. A Red Hat customer account Go to OpenShift Cluster Manager ( http://console.redhat.com/openshift ) and log in or register for a new account. Cluster administrator access to your OpenShift cluster You must have an OpenShift cluster with cluster administrator access. Use an existing cluster, or create a cluster by following the steps in the relevant documentation: Creating an OpenShift Dedicated cluster Install ROSA Classic clusters An OpenShift Dedicated or ROSA cluster configuration that meets the following requirements: At least 2 worker nodes with at least 8 CPUs and 32 GiB RAM available for OpenShift AI to use when you install the Add-on. If this requirement is not met, the installation process fails to start and an error is displayed. When you create a new cluster, select m6a.2xlarge for the computer node instance type to satisfy the requirements. For an existing ROSA Classic cluster, you can get the compute node instance type by using this command: rosa list machinepools --cluster=cluster-name You cannot alter a cluster's compute node instance type, but you can add an additional machine pool or modify the default pool to meet the minimum requirements. However, the minimum resource requirements must be met by a single machine pool in the cluster. For more information, see the relevant documentation: Creating a machine pool in OpenShift Dedicated OpenShift AI Service Definition Creating a machine pool in ROSA Classic Prepare your environment (ROSA Classic) For a ROSA cluster, select an access management strategy For installing OpenShift AI on a ROSA Classic cluster, decide whether you want to install on a ROSA cluster that uses AWS Security Token Service (STS) or one that uses AWS Identity and Access Management (IAM) credentials. See Install ROSA Classic clusters for advice on deploying a ROSA cluster with or without AWS STS. Install KServe dependencies To support the KServe component, which is used by the single-model serving platform to serve large models, you must also install Operators for Red Hat OpenShift Serverless and Red Hat OpenShift Service Mesh and perform additional configuration. For more information, see About the single-model serving platform . If you want to add an authorization provider for the single-model serving platform, you must install the Red Hat - Authorino Operator. For information, see Adding an authorization provider for the single-model serving platform . Install model registry dependencies (Technology Preview feature) To use the model registry component, you must also install Operators for Red Hat Authorino, Red Hat OpenShift Serverless, and Red Hat OpenShift Service Mesh. For more information about configuring the model registry component, see Configuring the model registry component . Access to object storage Components of OpenShift AI require or can use S3-compatible object storage such as AWS S3, MinIO, Ceph, or IBM Cloud Storage. An object store is a data storage mechanism that enables users to access their data either as an object or as a file. The S3 API is the recognized standard for HTTP-based access to object storage services. Object storage is required for the following components: Single- or multi-model serving platforms, to deploy stored models. See Deploying models on the single-model serving platform or Deploying a model by using the multi-model serving platform . Data science pipelines, to store artifacts, logs, and intermediary results. See Configuring a pipeline server and About pipeline logs . Object storage can be used by the following components: Workbenches, to access large datasets. See Adding a connection to your data science project . Distributed workloads, to pull input data from and push results to. See Running distributed data science workloads from data science pipelines . Code executed inside a pipeline. For example, to store the resulting model in object storage. See Overview of pipelines in Jupyterlab . 2.2. Configuring an identity provider for your OpenShift cluster Configure an identity provider for your OpenShift Dedicated or Red Hat OpenShift Service on Amazon Web Services (ROSA) cluster to manage users and groups. Red Hat OpenShift AI supports the same authentication systems as Red Hat OpenShift Dedicated and ROSA. Check the appropriate documentation for your cluster for more information. Supported identity providers on OpenShift Dedicated Supported identity providers on ROSA Important Adding more than one OpenShift Identity Provider can create problems when the same user name exists in multiple providers. When mappingMethod is set to claim (the default mapping method for identity providers) and multiple providers have credentials associated with the same user name, the first provider used to log in to OpenShift is the one that works for that user, regardless of the order in which identity providers are configured. For more information about mapping methods, see Identity provider parameters in OpenShift Dedicated or Identity provider parameters in ROSA . Prerequisites Credentials for OpenShift Cluster Manager ( https://console.redhat.com/openshift/ ). An existing OpenShift Dedicated or ROSA cluster. Procedure Log in to OpenShift Cluster Manager ( https://console.redhat.com/openshift/ ). Click Clusters . The Clusters page opens. Click the name of the cluster to configure. Click the Access control tab. Click Identity providers . Click Add identity provider . Select your provider from the Identity Provider list. Complete the remaining fields relevant to the identity provider that you selected. For more information, see Configuring identity providers in OpenShift Dedicated or Configuring identity providers in ROSA . Click Confirm . Verification The configured identity providers are visible on the Access control tab of the Cluster details page. Additional resources Configuring identity providers in OpenShift Dedicated Configuring identity providers in ROSA 2.3. Adding administrative users in OpenShift Before you can install and configure OpenShift AI for your data scientist users, you must obtain OpenShift cluster administrator ( cluster-admin ) privileges. Prerequisites Credentials for Red Hat OpenShift Cluster Manager ( https://console.redhat.com/openshift/ ). An existing OpenShift Dedicated or Red Hat OpenShift Service on AWS (ROSA Classic) cluster with an identity provider configured. Procedure Log in to OpenShift Cluster Manager ( https://console.redhat.com/openshift/ ). Click Clusters . The Clusters page opens. Click the name of the cluster to configure. Click the Access control tab. Click Cluster Roles and Access . Under Cluster administrative users click the Add user button. The Add cluster user popover appears. Enter the user name in the User ID field. Select an appropriate Group for the user. Important If this user needs to use existing groups in an identity provider to control OpenShift AI access, select cluster-admins . For more information about these user types, see Managing administration roles and users in the OpenShift Dedicated documentation or Default cluster roles in the ROSA documentation. Click Add user . Verification The user name and selected group are visible in the list of Cluster administrative users . Additional resources OpenShift Dedicated Cluster administration ROSA Cluster administration 2.4. Subscribing to the Red Hat OpenShift AI Cloud Service You can subscribe to the Red Hat OpenShift AI managed cloud service in the following ways: Subscribe through Red Hat if you have a Red Hat OpenShift Dedicated cluster deployed with a Customer Cloud Subscription (CCS) on Amazon Web Services (AWS) or Google Cloud Platform (GCP). Subscribe through the AWS Marketplace if you have a Red Hat OpenShift Service on AWS (ROSA Classic) cluster. Note You can also purchase Red Hat OpenShift AI as self-managed software. To purchase a new subscription, contact your Red Hat account manager. If you do not yet have an account manager, complete the form at https://www.redhat.com/en/contact to request one. 2.4.1. Subscribing to the OpenShift AI managed cloud service on AWS or GCP For a Red Hat OpenShift Dedicated cluster that is deployed on AWS or GCP, contact your Red Hat account manager to purchase a new subscription. If you do not yet have an account manager, complete the form at https://cloud.redhat.com/products/dedicated/contact/ to request one. Prerequisite You have worked with Red Hat Sales to enable a private offer of OpenShift AI, follow these steps to accept your offer and deploy the solution. Procedure Visit your Private Offer with the URL link provided by your Red Hat Sales representative. Click Accept Terms to subscribe to the AMI Private Offer named OpenShift AI from AWS Marketplace . After accepting the offer terms, click Continue to Configuration . 2.4.2. Subscribing to the OpenShift AI managed cloud service on Red Hat OpenShift Service on AWS (ROSA) For a ROSA Classic cluster, you can subscribe to the OpenShift AI managed cloud service through the Amazon Web Services (AWS) Marketplace. Prerequisites Access to a ROSA Classic cluster, including permissions to view and install add-ons. An AWS account with permission to view and subscribe to offerings in the AWS marketplace. Procedure In the AWS Console, navigate to the AWS Marketplace. For example: Click the help icon and then select Getting Started Resource Center. Select AWS Marketplace > Browse AWS Marketplace. In the top Search field, type: Red Hat OpenShift AI . Select one of the two options depending on the geographical location of the billing address for your AWS account (note that this location might differ from the geographical location of the cluster): Europe, the Middle East, and Africa (EMEA region) North America and regions outside EMEA Click Continue to Subscribe . Click Continue to Configuration and then select the appropriate fulfillment options. Note that some selectors might have only one option. Click Continue to Launch . Link your AWS account with your Red Hat account to complete your registration: In the AWS Marketplace console, navigate to the Manage Subscriptions page. On the Red Hat OpenShift AI tile, click Set up product . On the top banner, click Set up account . This link takes you to the Red Hat Hybrid console. If you are not already logged in, log in. Review and then accept the terms and agreements. Click Connect accounts . Verification The Data Science product page opens. 2.5. Installing OpenShift AI on your OpenShift cluster You can use Red Hat OpenShift Cluster Manager to install Red Hat OpenShift AI as an Add-on to your Red Hat OpenShift cluster. Prerequisites A subscription to the Red Hat OpenShift AI Add-on, as described in Subscribing to the OpenShift AI managed cloud service on AWS or GCP . If you purchased the Red Hat OpenShift AI Add-on for ROSA Classic by using the AWS Marketplace, you have associated your AWS account with your Red Hat account as described in Subscribing to the OpenShift AI managed cloud service on Red Hat OpenShift Service on AWS (ROSA) . Credentials for Red Hat OpenShift Cluster Manager ( https://console.redhat.com/openshift/ ). Administrator access to the OpenShift cluster. To support KServe components, you installed the dependent Operators, including the Red Hat OpenShift Serverless and Red Hat OpenShift Service Mesh Operators. For more information, see Serving large models . Note For information about the lifecycle associated with Red Hat OpenShift AI, see the Red Hat OpenShift AI Life Cycle Knowledgebase article. Important Data science pipelines 2.0 contains an installation of Argo Workflows. OpenShift AI does not support direct customer usage of this installation of Argo Workflows. To install OpenShift AI with data science pipelines 2.0, ensure that no separate installation of Argo Workflows exists on your cluster. Procedure Log in to OpenShift Cluster Manager ( https://console.redhat.com/openshift/ ). Click Clusters . The Clusters page opens. Click the name of the cluster you want to install OpenShift AI on. The Details page for the cluster opens. Click the Add-ons tab and locate the Red Hat OpenShift AI tile. Note If there is a Prerequisites not met warning message, click the Prerequisites tab. Note down the error message. If the error message states that you require a new machine pool, or that more resources are required, take the appropriate action to resolve the problem. You might need to add more resources to your cluster, or increase the size of your default machine pool. To increase your cluster's resources, contact your infrastructure administrator. For more information about increasing the size of your machine pool, see Allocating additional resources to OpenShift AI users . Select a Subscription type : If you obtained your Red Hat OpenShift AI subscription through your Red Hat account manager, select Standard and then skip to Step 7. If you obtained your Red Hat OpenShift AI subscription directly from the AWS Marketplace, select Marketplace and then continue to Step 6. For a Marketplace subscription, select your AWS account number from the list. Note If your AWS account number is not in the list, you might need to link your Red Hat and AWS accounts, as described in Subscribing to the OpenShift AI managed cloud service on Red Hat OpenShift Service on AWS (ROSA) . Click Install . The Configure Red Hat OpenShift AI pane appears. In the Notification email field, enter any email addresses that you want to receive important alerts about the state of Red Hat OpenShift AI, such as outage alerts. Click Install . Verification In OpenShift Cluster Manager, on the Add-ons tab for the cluster, confirm that the OpenShift AI tile shows one of the following states: Installing - installation is in progress; wait for this to change to Installed . This takes around 30 minutes. Installed - installation is complete; verify that the View in console button is visible. In OpenShift AI, click Home Projects and confirm that the following project namespaces are visible and listed as Active : redhat-ods-applications redhat-ods-monitoring redhat-ods-operator rhods-notebooks 2.6. Installing and managing Red Hat OpenShift AI components You can use the OpenShift web console to install and manage components of Red Hat OpenShift AI on your OpenShift cluster. 2.6.1. Installing Red Hat OpenShift AI components by using the web console When you install Red Hat OpenShift AI as an add-on to your OpenShift cluster, the install process automatically creates a default DataScienceCluster object. To install Red Hat OpenShift AI components by using the OpenShift web console, you must configure the DataScienceCluster object. Important The following procedure describes how to configure the DataScienceCluster object to install Red Hat OpenShift AI components as part of a new installation. For information about changing the installation status of OpenShift AI components after installation, see Updating the installation status of Red Hat OpenShift AI components by using the web console . For information about upgrading OpenShift AI, see Upgrading OpenShift AI Cloud Service . Prerequisites Red Hat OpenShift AI is installed as an add-on to your Red Hat OpenShift cluster. You have cluster administrator privileges for your OpenShift cluster. Procedure Log in to the OpenShift web console as a cluster administrator. In the web console, click Operators Installed Operators and then click the Red Hat OpenShift AI Operator. Click the Data Science Cluster tab. Click the default-dsc object. Select the YAML tab. An embedded YAML editor opens showing a default custom resource (CR) for the DataScienceCluster object, similar to the following example: apiVersion: datasciencecluster.opendatahub.io/v1 kind: DataScienceCluster metadata: name: default-dsc spec: components: codeflare: managementState: Removed dashboard: managementState: Removed datasciencepipelines: managementState: Removed kserve: managementState: Removed 1 2 kueue: managementState: Removed modelmeshserving: managementState: Removed ray: managementState: Removed trainingoperator: managementState: Removed trustyai: managementState: Removed workbenches: managementState: Removed 1 To fully install the KServe component, which is used by the single-model serving platform to serve large models, you must install Operators for Red Hat OpenShift Service Mesh and Red Hat OpenShift Serverless and perform additional configuration. See Installing the single-model serving platform . 2 If you have not enabled the KServe component (that is, you set the value of the managementState field to Removed ), you must also disable the dependent Service Mesh component to avoid errors. See Disabling KServe dependencies . In the spec.components section of the CR, for each OpenShift AI component shown, set the value of the managementState field to either Managed or Removed . These values are defined as follows: Managed The Operator actively manages the component, installs it, and tries to keep it active. The Operator will upgrade the component only if it is safe to do so. Removed The Operator actively manages the component but does not install it. If the component is already installed, the Operator will try to remove it. Important To learn how to fully install the KServe component, which is used by the single-model serving platform to serve large models, see Installing the single-model serving platform . If you have not enabled the KServe component (that is, you set the value of the managementState field to Removed ), you must also disable the dependent Service Mesh component to avoid errors. See Disabling KServe dependencies . To learn how to install the distributed workloads components, see Installing the distributed workloads components . Click Save . Verification Confirm that there is a running pod for each component: In the OpenShift web console, click Workloads Pods . In the Project list at the top of the page, select redhat-ods-applications . In the applications namespace, confirm that there are running pods for each of the OpenShift AI components that you installed. Confirm the status of all installed components: In the OpenShift web console, click Operators Installed Operators . Click the Red Hat OpenShift AI Operator. Click the Data Science Cluster tab and select the DataScienceCluster object called default-dsc . Select the YAML tab. In the installedComponents section, confirm that the components you installed have a status value of true . Note If a component shows with the component-name: {} format in the spec.components section of the CR, the component is not installed. 2.6.2. Updating the installation status of Red Hat OpenShift AI components by using the web console You can use the OpenShift web console to update the installation status of components of Red Hat OpenShift AI on your OpenShift cluster. Important If you upgraded OpenShift AI, the upgrade process automatically used the values of the version's DataScienceCluster object. New components are not automatically added to the DataScienceCluster object. After upgrading OpenShift AI: Inspect the default DataScienceCluster object to check and optionally update the managementState status of the existing components. Add any new components to the DataScienceCluster object. Prerequisites Red Hat OpenShift AI is installed as an Add-on to your Red Hat OpenShift cluster. You have cluster administrator privileges for your OpenShift cluster. Procedure Log in to the OpenShift web console as a cluster administrator. In the web console, click Operators Installed Operators and then click the Red Hat OpenShift AI Operator. Click the Data Science Cluster tab. On the DataScienceClusters page, click the default object. Click the YAML tab. An embedded YAML editor opens showing the default custom resource (CR) for the DataScienceCluster object, similar to the following example: apiVersion: datasciencecluster.opendatahub.io/v1 kind: DataScienceCluster metadata: name: default-dsc spec: components: codeflare: managementState: Removed dashboard: managementState: Removed datasciencepipelines: managementState: Removed kserve: managementState: Removed kueue: managementState: Removed modelmeshserving: managementState: Removed ray: managementState: Removed trainingoperator: managementState: Removed trustyai: managementState: Removed workbenches: managementState: Removed In the spec.components section of the CR, for each OpenShift AI component shown, set the value of the managementState field to either Managed or Removed . These values are defined as follows: Managed The Operator actively manages the component, installs it, and tries to keep it active. The Operator will upgrade the component only if it is safe to do so. Removed The Operator actively manages the component but does not install it. If the component is already installed, the Operator will try to remove it. Important To learn how to install the KServe component, which is used by the single-model serving platform to serve large models, see Installing the single-model serving platform . If you have not enabled the KServe component (that is, you set the value of the managementState field to Removed ), you must also disable the dependent Service Mesh component to avoid errors. See Disabling KServe dependencies . To learn how to install the distributed workloads feature, see Installing the distributed workloads components . Click Save . For any components that you updated, OpenShift AI initiates a rollout that affects all pods to use the updated image. Verification Confirm that there is a running pod for each component: In the OpenShift web console, click Workloads Pods . In the Project list at the top of the page, select redhat-ods-applications . In the applications namespace, confirm that there are running pods for each of the OpenShift AI components that you installed. Confirm the status of all installed components: In the OpenShift web console, click Operators Installed Operators . Click the Red Hat OpenShift AI Operator. Click the Data Science Cluster tab and select the DataScienceCluster object called default-dsc . Select the YAML tab. In the installedComponents section, confirm that the components you installed have a status value of true . Note If a component shows with the component-name: {} format in the spec.components section of the CR, the component is not installed.
apiVersion: datasciencecluster.opendatahub.io/v1 kind: DataScienceCluster metadata: name: default-dsc spec: components: codeflare: managementState: Removed dashboard: managementState: Removed datasciencepipelines: managementState: Removed kserve: managementState: Removed 1 2 kueue: managementState: Removed modelmeshserving: managementState: Removed ray: managementState: Removed trainingoperator: managementState: Removed trustyai: managementState: Removed workbenches: managementState: Removed apiVersion: datasciencecluster.opendatahub.io/v1 kind: DataScienceCluster metadata: name: default-dsc spec: components: codeflare: managementState: Removed dashboard: managementState: Removed datasciencepipelines: managementState: Removed kserve: managementState: Removed kueue: managementState: Removed modelmeshserving: managementState: Removed ray: managementState: Removed trainingoperator: managementState: Removed trustyai: managementState: Removed workbenches: managementState: Removed