Chapter 1. Multitenancy support in GitOps
Chapter 1. Multitenancy support in GitOps Multitenancy is a software architecture where a single software instance serves multiple distinct user groups, or tenants. Using multitenancy, you can share a single Argo CD instance to deploy resources while maintaining isolation between users. This section helps cluster administrators to understand Argo CD instance scopes and when to choose a particular mode. With Red Hat OpenShift GitOps Operator on OpenShift Container Platform, as a cluster administrator, you can provide multitenant access to the cluster for application delivery teams (tenants). You can allow tenants to create and manage dedicated Argo CD instances in their user-defined namespaces without having administrative permissions. The tenants have full autonomy and can tailor this instance to their needs and requirements without interfering with other tenants. Note For a multitenant cluster, the person managing the Argo CD instance may not be the person who manages the cluster and its users. Therefore, you cannot have the Argo CD Application Controller ( argocd-application-controller component) as a superuser in the cluster. The Argo CD Application Controller reconciles resources in the managed clusters. Therefore, to use multitenancy in the GitOps Operator, depending on your use cases, tenants, and requirements, you must configure, grant, extend, or restrict specific permissions explicitly to Argo CD instances, applications, and on remote clusters to perform certain operations. 1.1. Argo CD instance scopes The Red Hat OpenShift GitOps Operator creates instances that you can classify broadly into the following modes, all supporting multitenancy: Namespace-scoped instance (Application delivery instance) Cluster-scoped instance Default cluster-scoped instance 1.1.1. Namespace-scoped instance (Application delivery instance) After you create an Argo CD custom resource (CR) in one of your namespaces, the GitOps Operator starts an Argo CD in this namespace, which you can use for application delivery. In its initial state, this instance has permission to deploy resources only in the same namespace in which it is installed. However, you may need to configure the instance to meet your specific requirements. With the GitOps Operator, you can extend the permissions of the Argo CD instance to enable the Argo CD Application Controller to deploy resources into other namespaces apart from where it is installed. Note The roles that the GitOps Operator in the namespace creates are namespace-scoped and only have access to namespace resources. The Operator cannot perform any actions outside of the namespaces it manages. How does this method work? The GitOps Operator allows OpenShift users to instantiate Argo CD instances in their namespaces, as long as they have permission to create Argo CD resources in the argoproj.io/v1alpha1 or argoproj.io/v1beta1 API in their namespace. Currently, a namespace-scoped instance has full administrative permissions for one or many namespaces it manages, similar to allowing all verbs for all resources within that namespace. For the Argo CD Application Controller to deploy resources into any other namespace, you need Kubernetes roles and role bindings to label and indicate these namespaces that you want a namespace-scoped instance to manage. The GitOps Operator supports the use of the argocd.argoproj.io/managed-by label to automatically create these roles and role bindings. Use this label and set the value to indicate the managed namespaces. Then, with the argocd.argoproj.io/managed-by label, when you deploy the GitOps Operator in the namespace-scoped instance mode, it creates a set of roles and role bindings in every namespace that the instance manages. Important The argocd.argoproj.io/managed-by label only works for namespaces in the same cluster as the GitOps Operator. For remote clusters, you must define the permissions manually. By default, when labeling the namespace using the managed-by label, the GitOps Operator gives the Argo CD Application Controller permissions equivalent to the Kubernetes default admin cluster role for the labeled namespace. However, you can opt to define an alternate cluster role that is used for the controller and server components by using the CONTROLLER_CLUSTER_ROLE and CONTROLLER_SERVER_ROLE environment variables respectively in the Operator's Subscription resource. When you provide these variables, the Operator will not create a default role in the namespace but rather only create a role binding in the namespace of the corresponding cluster role. It is up to the administrator to create the cluster role, having full control over the permissions. Note When defining your role, Argo CD will attempt to interact with all resources. Therefore, you must either provide permission to view , get , and watch all resources in your custom cluster role, or configure the Argo CD CR to include or exclude specific resources through the resourceInclusions or resourceExclusions fields defined in the role. It is not possible to manage cluster-scoped resources, such as namespaces, custom resource definitions (CRDs), or cluster roles with a namespace-scoped instance. Additional resources Deploying resources to a different namespace 1.1.2. Cluster-scoped instance A cluster-scoped instance is intended to deploy and manage resources across a cluster. Note If you intend to use the Applications in any namespace feature, choose the mode of the Argo CD instance scope as cluster-scoped instance. Cluster-scoped instances have access to cluster-level resources and thus are typically, but not always, used for cluster configuration. You can choose to elevate certain namespace-scoped Argo CD instances to become cluster-scoped. To elevate the instances, you must modify the Subscription resource of the GitOps Operator. Important Ensure you give careful consideration while elevating instances. Do not elevate instances that are self-managed by application delivery teams. Elevating such instances would be a severe security risk to the cluster because this action makes users of self-managed instances cluster administrators, and gives them full control over the permissions. Much care must be taken when setting up multitenancy configuration within Argo CD. For example, in a use case where cluster administrators want to set up an Argo CD instance that is shared among multiple application delivery teams and is managed by the cluster administrators, having a custom cluster-scoped instance might be what you need. How does this method work? To prevent users from deploying Argo CD instances with cluster-admin privileges, you must identify the namespaces with cluster privileges by using the ARGOCD_CLUSTER_CONFIG_NAMESPACES environment variable in the Subscription resource. Because non-cluster administrators do not have access to the Subscription resource, they cannot elevate the privileges of their instance and bypass cluster security. When an instance is designated as cluster-scoped, the Operator automatically creates a set of cluster role and cluster role bindings for the Argo CD Application Controller and server service accounts in that namespace. This default role is not intended to be the equivalent of the standard cluster-admin role. It is given a much smaller set of permissions. These permissions can be extended by creating additional cluster roles or cluster role bindings as needed. Additional resources Managing the application resources in non-control plane namespaces 1.1.3. Default cluster-scoped instance After you install the GitOps Operator, by default, it instantiates a cluster-scoped instance in the openshift-gitops namespace. This instance is set up in a very opinionated way and is intended to allow cluster administrators to manage certain cluster configuration resources. Important Do not use the default cluster-scoped instance for anything else, such as application delivery. Do not grant permission to users who do not have cluster-admin roles to access the default cluster-scoped instance. The default instance does not have full cluster-admin privileges. It has read access to all resources in the cluster, but it can only deploy a limited set of resources. Note Use the default Argo CD instance in the openshift-gitops namespace for cluster configuration and delegate tenant use cases to one or many separate instances in other namespaces. 1.2. Important considerations when adopting the multitenancy model Granting multitenant cluster administration privileges might allow tenants to bypass any multitenancy efforts and permission restrictions because they might elevate their privileges using permissions granted to the application by Argo CD. To prevent such situations, you must understand the permission model for Argo CD installed through the Red Hat OpenShift GitOps Operator and how to leverage it to be successful with OpenShift GitOps for application delivery. 1.2.1. Argo CD role-based access control (RBAC) Access controls in Red Hat OpenShift GitOps are managed at two distinct levels as follows: At the Kubernetes level, the GitOps Argo CD Application Controller interacts with one or more clusters to deploy various resources by using a single Kubernetes service account per cluster. This service account must have sufficient permissions to deploy resources for all tenants and use cases that this instance of Argo CD is managing. At the Argo CD level, the GitOps Argo CD Application Controller includes its own RBAC permissions model independent of Kubernetes. To manage the user-level access, you can use this RBAC model independently of the underlying Kubernetes permissions. You can use this capability to provide access to Argo CD applications that a user would not have access to from a purely Kubernetes perspective. Because these two access controls and the interaction between components are distinct and separate, privilege escalation becomes a concern when designing a multitenant solution with Argo CD. Privilege escalation is when a tenant leverages the increased privileges of the application controller's service account to perform actions they would not typically be permitted to perform. You can mitigate the privilege escalation in an Argo CD instance by using the Argo CD RBAC or separate instances of Argo CD. 1.2.2. Argo CD projects Argo CD projects, not to be confused with OpenShift Container Platform projects, provide a way to group applications together. With Argo CD projects, you can specify restrictions on Applications concerning what resources can be deployed and where they can be deployed. Additionally, you can enable Argo CD role-based access control (RBAC) rules and permissions by defining them more granularly at the project level versus the global level in the Argo CD custom resource (CR). While you can define tenant RBAC globally in the Operator's Argo CD CR, you should define tenant RBAC along with restrictions in the AppProject CR. If you have a large number of tenants, attempting to manage all tenants with global RBAC might lead to a lot of repetition. If you have many instances of tenants, some of the project configurations might be common across tenant projects. To reduce duplication and minimize maintenance, use global projects for common configuration and inherit them in tenant projects. Note Always define your projects and never use the default project created with an Argo CD installation by the Operator. 1.3. Additional resources Glossary of common terms for OpenShift GitOps 5 global environment variables provided by Red Hat OpenShift GitOps Elevating Argo CD instances to cluster-scoped instances Understanding on how to set up the role-based access control (RBAC) policies Configuring global projects Using GitOps and Argo CD with RBAC