Chapter 3. Setting compute resource quota for OpenShift Pipelines
Chapter 3. Setting compute resource quota for OpenShift Pipelines A ResourceQuota object in Red Hat OpenShift Pipelines controls the total resource consumption per namespace. You can use it to limit the quantity of objects created in a namespace, based on the type of the object. In addition, you can specify a compute resource quota to restrict the total amount of compute resources consumed in a namespace. However, you might want to limit the amount of compute resources consumed by pods resulting from a pipeline run, rather than setting quotas for the entire namespace. Currently, Red Hat OpenShift Pipelines does not enable you to directly specify the compute resource quota for a pipeline. 3.1. Alternative approaches for limiting compute resource consumption in OpenShift Pipelines To attain some degree of control over the usage of compute resources by a pipeline, consider the following alternative approaches: Set resource requests and limits for each step in a task. Example: Set resource requests and limits for each step in a task. ... spec: steps: - name: step-with-limts computeResources: requests: memory: 1Gi cpu: 500m limits: memory: 2Gi cpu: 800m ... Set resource limits by specifying values for the LimitRange object. For more information on LimitRange , refer to Restrict resource consumption with limit ranges . Reduce pipeline resource consumption . Set and manage resource quotas per project . Ideally, the compute resource quota for a pipeline should be same as the total amount of compute resources consumed by the concurrently running pods in a pipeline run. However, the pods running the tasks consume compute resources based on the use case. For example, a Maven build task might require different compute resources for different applications that it builds. As a result, you cannot predetermine the compute resource quotas for tasks in a generic pipeline. For greater predictability and control over usage of compute resources, use customized pipelines for different applications. Note When using Red Hat OpenShift Pipelines in a namespace configured with a ResourceQuota object, the pods resulting from task runs and pipeline runs might fail with an error, such as: failed quota: <quota name> must specify cpu, memory . To avoid this error, do any one of the following: (Recommended) Specify a limit range for the namespace. Explicitly define requests and limits for all containers. For more information, refer to the issue and the resolution . If your use case is not addressed by these approaches, you can implement a workaround by using a resource quota for a priority class. 3.2. Specifying pipelines resource quota using priority class A PriorityClass object maps priority class names to the integer values that indicates their relative priorities. Higher values increase the priority of a class. After you create a priority class, you can create pods that specify the priority class name in their specifications. In addition, you can control a pod's consumption of system resources based on the pod's priority. Specifying resource quota for a pipeline is similar to setting a resource quota for the subset of pods created by a pipeline run. The following steps provide an example of the workaround by specifying resource quota based on priority class. Procedure Create a priority class for a pipeline. Example: Priority class for a pipeline apiVersion: scheduling.k8s.io/v1 kind: PriorityClass metadata: name: pipeline1-pc value: 1000000 description: "Priority class for pipeline1" Create a resource quota for a pipeline. Example: Resource quota for a pipeline apiVersion: v1 kind: ResourceQuota metadata: name: pipeline1-rq spec: hard: cpu: "1000" memory: 200Gi pods: "10" scopeSelector: matchExpressions: - operator : In scopeName: PriorityClass values: ["pipeline1-pc"] Verify the resource quota usage for the pipeline. Example: Verify resource quota usage for the pipeline USD oc describe quota Sample output Because pods are not running, the quota is unused. Create the pipelines and tasks. Example: YAML for the pipeline apiVersion: tekton.dev/v1 kind: Pipeline metadata: name: maven-build spec: params: - name: GIT_URL workspaces: - name: local-maven-repo - name: source tasks: - name: git-clone taskRef: resolver: cluster params: - name: kind value: task - name: name value: git-clone - name: namespace value: openshift-pipelines workspaces: - name: output workspace: source params: - name: URL value: USD(params.GIT_URL) - name: build taskRef: name: mvn runAfter: ["git-clone"] params: - name: GOALS value: ["package"] workspaces: - name: maven-repo workspace: local-maven-repo - name: source workspace: source - name: int-test taskRef: name: mvn runAfter: ["build"] params: - name: GOALS value: ["verify"] workspaces: - name: maven-repo workspace: local-maven-repo - name: source workspace: source - name: gen-report taskRef: name: mvn runAfter: ["build"] params: - name: GOALS value: ["site"] workspaces: - name: maven-repo workspace: local-maven-repo - name: source workspace: source Example: YAML for a task in the pipeline apiVersion: tekton.dev/v1 kind: Task metadata: name: mvn spec: workspaces: - name: maven-repo - name: source params: - name: GOALS description: The Maven goals to run type: array default: ["package"] steps: - name: mvn image: gcr.io/cloud-builders/mvn workingDir: USD(workspaces.source.path) command: ["/usr/bin/mvn"] args: - -Dmaven.repo.local=USD(workspaces.maven-repo.path) - "USD(params.GOALS)" Create and start the pipeline run. Example: YAML for a pipeline run apiVersion: tekton.dev/v1 kind: PipelineRun metadata: generateName: petclinic-run- spec: pipelineRef: name: maven-build params: - name: GIT_URL value: https://github.com/spring-projects/spring-petclinic taskRunTemplate: podTemplate: priorityClassName: pipeline1-pc workspaces: - name: local-maven-repo emptyDir: {} - name: source volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 200M Note The pipeline run might fail with an error: failed quota: <quota name> must specify cpu, memory . To avoid this error, set a limit range for the namespace, where the defaults from the LimitRange object apply to pods created during the build process. For more information about setting limit ranges, refer to Restrict resource consumption with limit ranges in the Additional resources section. After the pods are created, verify the resource quota usage for the pipeline run. Example: Verify resource quota usage for the pipeline USD oc describe quota Sample output The output indicates that you can manage the combined resource quota for all concurrent running pods belonging to a priority class, by specifying the resource quota per priority class. 3.3. Additional resources Restrict resource consumption with limit ranges Resource quotas in Kubernetes Limit ranges in Kubernetes Resource requests and limits in Kubernetes
spec: steps: - name: step-with-limts computeResources: requests: memory: 1Gi cpu: 500m limits: memory: 2Gi cpu: 800m apiVersion: scheduling.k8s.io/v1 kind: PriorityClass metadata: name: pipeline1-pc value: 1000000 description: "Priority class for pipeline1" apiVersion: v1 kind: ResourceQuota metadata: name: pipeline1-rq spec: hard: cpu: "1000" memory: 200Gi pods: "10" scopeSelector: matchExpressions: - operator : In scopeName: PriorityClass values: ["pipeline1-pc"] oc describe quota Name: pipeline1-rq Namespace: default Resource Used Hard -------- ---- ---- cpu 0 1k memory 0 200Gi pods 0 10 apiVersion: tekton.dev/v1 kind: Pipeline metadata: name: maven-build spec: params: - name: GIT_URL workspaces: - name: local-maven-repo - name: source tasks: - name: git-clone taskRef: resolver: cluster params: - name: kind value: task - name: name value: git-clone - name: namespace value: openshift-pipelines workspaces: - name: output workspace: source params: - name: URL value: USD(params.GIT_URL) - name: build taskRef: name: mvn runAfter: ["git-clone"] params: - name: GOALS value: ["package"] workspaces: - name: maven-repo workspace: local-maven-repo - name: source workspace: source - name: int-test taskRef: name: mvn runAfter: ["build"] params: - name: GOALS value: ["verify"] workspaces: - name: maven-repo workspace: local-maven-repo - name: source workspace: source - name: gen-report taskRef: name: mvn runAfter: ["build"] params: - name: GOALS value: ["site"] workspaces: - name: maven-repo workspace: local-maven-repo - name: source workspace: source apiVersion: tekton.dev/v1 kind: Task metadata: name: mvn spec: workspaces: - name: maven-repo - name: source params: - name: GOALS description: The Maven goals to run type: array default: ["package"] steps: - name: mvn image: gcr.io/cloud-builders/mvn workingDir: USD(workspaces.source.path) command: ["/usr/bin/mvn"] args: - -Dmaven.repo.local=USD(workspaces.maven-repo.path) - "USD(params.GOALS)" apiVersion: tekton.dev/v1 kind: PipelineRun metadata: generateName: petclinic-run- spec: pipelineRef: name: maven-build params: - name: GIT_URL value: https://github.com/spring-projects/spring-petclinic taskRunTemplate: podTemplate: priorityClassName: pipeline1-pc workspaces: - name: local-maven-repo emptyDir: {} - name: source volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 200M oc describe quota Name: pipeline1-rq Namespace: default Resource Used Hard -------- ---- ---- cpu 500m 1k memory 10Gi 200Gi pods 1 10