Chapter 3. Customizing configurations in the TektonConfig custom resource
Chapter 3. Customizing configurations in the TektonConfig custom resource In Red Hat OpenShift Pipelines, you can customize the following configurations by using the TektonConfig custom resource (CR): Optimizing OpenShift Pipelines performance, including high-availability mode for the OpenShift Pipelines controller Configuring the Red Hat OpenShift Pipelines control plane Changing the default service account Disabling the service monitor Configuring pipeline resolvers Disabling cluster tasks and pipeline templates Disabling the integration of Tekton Hub Disabling the automatic creation of RBAC resources Pruning of task runs and pipeline runs 3.1. Prerequisites You have installed the Red Hat OpenShift Pipelines Operator. 3.2. Performance tuning using TektonConfig CR You can modify the fields under the .spec.pipeline.performance parameter in the TektonConfig custom resource (CR) to change high availability (HA) support and performance configuration for the OpenShift Pipelines controller. Example TektonConfig performance fields apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: performance: disable-ha: false buckets: 7 replicas: 5 threads-per-controller: 2 kube-api-qps: 5.0 kube-api-burst: 10 All fields are optional. If you set them, the Red Hat OpenShift Pipelines Operator includes most of the fields as arguments in the openshift-pipelines-controller deployment under the openshift-pipelines-controller container. The OpenShift Pipelines Operator also updates the buckets field in the config-leader-election configuration map under the openshift-pipelines namespace. If you do not specify the values, the OpenShift Pipelines Operator does not update those fields and applies the default values for the OpenShift Pipelines controller. Note If you modify or remove any of the performance fields, the OpenShift Pipelines Operator updates the openshift-pipelines-controller deployment and the config-leader-election configuration map (if the buckets field changed) and re-creates openshift-pipelines-controller pods. High-availability (HA) mode applies to the OpenShift Pipelines controller, which creates and starts pods based on pipeline run and task run definitions. Without HA mode, a single pod executes these operations, potentially creating significant delays under a high load. In HA mode, OpenShift Pipelines uses several pods (replicas) to execute these operations. Initially, OpenShift Pipelines assigns every controller operation into a bucket. Each replica picks operations from one or more buckets. If two replicas could pick the same operation at the same time, the controller internally determines a leader that executes this operation. HA mode does not affect execution of task runs after the pods are created. Table 3.1. Modifiable fields for tuning OpenShift Pipelines performance Name Description Default value for the OpenShift Pipelines controller disable-ha Enable or disable the high availability (HA) mode. By default, the HA mode is enabled. false buckets In HA mode, the number of buckets used to process controller operations. The maximum value is 10 1 replicas In HA mode, the number of pods created to process controller operations. Set this value to the same or lower number than the buckets value. 1 threads-per-controller The number of threads (workers) to use when the work queue of the OpenShift Pipelines controller is processed. 2 kube-api-qps The maximum queries per second (QPS) to the cluster master from the REST client. 5.0 kube-api-burst The maximum burst for a throttle. 10 Note The OpenShift Pipelines Operator does not control the number of replicas of the OpenShift Pipelines controller. The replicas setting of the deployment determines the number of replicas. For example, to change the number of replicas to 3, enter the following command: USD oc --namespace openshift-pipelines scale deployment openshift-pipelines-controller --replicas=3 Important The kube-api-qps and kube-api-burst fields are multiplied by 2 in the OpenShift Pipelines controller. For example, if the kube-api-qps and kube-api-burst values are 10 , the actual QPS and burst values become 20 . 3.3. Configuring the Red Hat OpenShift Pipelines control plane You can customize the OpenShift Pipelines control plane by editing the configuration fields in the TektonConfig custom resource (CR). The Red Hat OpenShift Pipelines Operator automatically adds the configuration fields with their default values so that you can use the OpenShift Pipelines control plane. Procedure In the Administrator perspective of the web console, navigate to Administration CustomResourceDefinitions . Use the Search by name box to search for the tektonconfigs.operator.tekton.dev custom resource definition (CRD). Click TektonConfig to see the CRD details page. Click the Instances tab. Click the config instance to see the TektonConfig CR details. Click the YAML tab. Edit the TektonConfig YAML file based on your requirements. Example of TektonConfig CR with default values apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: running-in-environment-with-injected-sidecars: true metrics.taskrun.duration-type: histogram metrics.pipelinerun.duration-type: histogram await-sidecar-readiness: true params: - name: enableMetrics value: 'true' default-service-account: pipeline require-git-ssh-secret-known-hosts: false enable-tekton-oci-bundles: false metrics.taskrun.level: task metrics.pipelinerun.level: pipeline enable-api-fields: stable enable-provenance-in-status: false enable-custom-tasks: true disable-creds-init: false disable-affinity-assistant: true 3.3.1. Modifiable fields with default values The following list includes all modifiable fields with their default values in the TektonConfig CR: running-in-environment-with-injected-sidecars (default: true ): Set this field to false if pipelines run in a cluster that does not use injected sidecars, such as Istio. Setting it to false decreases the time a pipeline takes for a task run to start. Note For clusters that use injected sidecars, setting this field to false can lead to an unexpected behavior. await-sidecar-readiness (default: true ): Set this field to false to stop OpenShift Pipelines from waiting for TaskRun sidecar containers to run before it begins to operate. This allows tasks to be run in environments that do not support the downwardAPI volume type. default-service-account (default: pipeline ): This field contains the default service account name to use for the TaskRun and PipelineRun resources, if none is specified. require-git-ssh-secret-known-hosts (default: false ): Setting this field to true requires that any Git SSH secret must include the known_hosts field. For more information about configuring Git SSH secrets, see Configuring SSH authentication for Git in the Additional resources section. enable-tekton-oci-bundles (default: false ): Set this field to true to enable the use of an experimental alpha feature named Tekton OCI bundle. enable-api-fields (default: stable ): Setting this field determines which features are enabled. Acceptable value is stable , beta , or alpha . Note Red Hat OpenShift Pipelines does not support the alpha value. enable-provenance-in-status (default: false ): Set this field to true to enable populating the provenance field in TaskRun and PipelineRun statuses. The provenance field contains metadata about resources used in the task run and pipeline run, such as the source from where a remote task or pipeline definition was fetched. enable-custom-tasks (default: true ): Set this field to false to disable the use of custom tasks in pipelines. disable-creds-init (default: false ): Set this field to true to prevent OpenShift Pipelines from scanning attached service accounts and injecting any credentials into your steps. disable-affinity-assistant (default: true ): Set this field to false to enable affinity assistant for each TaskRun resource sharing a persistent volume claim workspace. Metrics options You can modify the default values of the following metrics fields in the TektonConfig CR: metrics.taskrun.duration-type and metrics.pipelinerun.duration-type (default: histogram ): Setting these fields determines the duration type for a task or pipeline run. Acceptable value is gauge or histogram . metrics.taskrun.level (default: task ): This field determines the level of the task run metrics. Acceptable value is taskrun , task , or namespace . metrics.pipelinerun.level (default: pipeline ): This field determines the level of the pipeline run metrics. Acceptable value is pipelinerun , pipeline , or namespace . 3.3.2. Optional configuration fields The following fields do not have a default value, and are considered only if you configure them. By default, the Operator does not add and configure these fields in the TektonConfig custom resource (CR). default-timeout-minutes : This field sets the default timeout for the TaskRun and PipelineRun resources, if none is specified when creating them. If a task run or pipeline run takes more time than the set number of minutes for its execution, then the task run or pipeline run is timed out and cancelled. For example, default-timeout-minutes: 60 sets 60 minutes as default. default-managed-by-label-value : This field contains the default value given to the app.kubernetes.io/managed-by label that is applied to all TaskRun pods, if none is specified. For example, default-managed-by-label-value: tekton-pipelines . default-pod-template : This field sets the default TaskRun and PipelineRun pod templates, if none is specified. default-cloud-events-sink : This field sets the default CloudEvents sink that is used for the TaskRun and PipelineRun resources, if none is specified. default-task-run-workspace-binding : This field contains the default workspace configuration for the workspaces that a Task resource declares, but a TaskRun resource does not explicitly declare. default-affinity-assistant-pod-template : This field sets the default PipelineRun pod template that is used for affinity assistant pods, if none is specified. default-max-matrix-combinations-count : This field contains the default maximum number of combinations generated from a matrix, if none is specified. 3.4. Changing the default service account for OpenShift Pipelines You can change the default service account for OpenShift Pipelines by editing the default-service-account field in the .spec.pipeline and .spec.trigger specifications. The default service account name is pipeline . Example apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: default-service-account: pipeline trigger: default-service-account: pipeline enable-api-fields: stable 3.5. Setting labels and annotations for the OpenShift Pipelines installation namespace You can set labels and annotations for the openshift-pipelines namespace in which the operator installs OpenShift Pipelines. Note Changing the name of the openshift-pipelines namespace is not supported. Specify the labels and annotations by adding them to the spec.targetNamespaceMetadata specification in the TektonConfig custom resource (CR). Example of setting labels and annotations for the openshift-pipelines namespace apiVersion: operator.tekton.dev/v1 kind: TektonConfig metadata: name: config spec: targetNamespaceMetadata: labels: {"example-label":"example-value"} annotations: {"example-annotation":"example-value"} 3.6. Disabling the service monitor You can disable the service monitor, which is part of OpenShift Pipelines, to expose the telemetry data. To disable the service monitor, set the enableMetrics parameter to false in the .spec.pipeline specification of the TektonConfig custom resource (CR): Example apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: params: - name: enableMetrics value: 'false' 3.7. Configuring pipeline resolvers You can configure pipeline resolvers in the TektonConfig custom resource (CR). You can enable or disable these pipeline resolvers: enable-bundles-resolver enable-cluster-resolver enable-git-resolver enable-hub-resolver Example apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: enable-bundles-resolver: true enable-cluster-resolver: true enable-git-resolver: true enable-hub-resolver: true You can also provide resolver specific configurations in the TektonConfig CR. For example, define the following fields in the map[string]string format to set configurations for each pipeline resolver: Example apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: bundles-resolver-config: default-service-account: pipelines cluster-resolver-config: default-namespace: test git-resolver-config: server-url: localhost.com hub-resolver-config: default-tekton-hub-catalog: tekton 3.8. Disabling cluster tasks and pipeline templates By default, the TektonAddon custom resource (CR) installs clusterTasks and pipelineTemplates resources along with OpenShift Pipelines on the cluster. You can disable installation of the clusterTasks and pipelineTemplates resources by setting the parameter value to false in the .spec.addon specification. In addition, you can disable the communityClusterTasks parameter. Example apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: addon: params: - name: clusterTasks value: 'false' - name: pipelineTemplates value: 'false' - name: communityClusterTasks value: 'true' Important In Red Hat OpenShift Pipelines 1.10, ClusterTask functionality is deprecated and is planned to be removed in a future release. 3.9. Disabling the integration of Tekton Hub You can disable the integration of Tekton Hub in the web console Developer perspective by setting the enable-devconsole-integration parameter to false in the TektonConfig custom resource (CR). Example of disabling Tekton Hub apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: hub: params: - name: enable-devconsole-integration value: false 3.10. Disabling the automatic creation of RBAC resources The default installation of the Red Hat OpenShift Pipelines Operator creates multiple role-based access control (RBAC) resources for all namespaces in the cluster, except the namespaces matching the ^(openshift|kube)-* regular expression pattern. Among these RBAC resources, the pipelines-scc-rolebinding security context constraint (SCC) role binding resource is a potential security issue, because the associated pipelines-scc SCC has the RunAsAny privilege. To disable the automatic creation of cluster-wide RBAC resources after the Red Hat OpenShift Pipelines Operator is installed, cluster administrators can set the createRbacResource parameter to false in the cluster-level TektonConfig custom resource (CR). Example TektonConfig CR apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: params: - name: createRbacResource value: "false" ... Warning As a cluster administrator or an user with appropriate privileges, when you disable the automatic creation of RBAC resources for all namespaces, the default ClusterTask resource does not work. For the ClusterTask resource to function, you must create the RBAC resources manually for each intended namespace. 3.11. Disabling inline specification of pipelines and tasks By default, OpenShift Pipelines supports inline specification of pipelines and tasks in the following cases: You can create a Pipeline CR that includes one or more task specifications, as in the following example: Example of an inline specification in a Pipeline CR apiVersion: operator.tekton.dev/v1 kind: Pipeline metadata: name: pipelineInline spec: tasks: taskSpec: # ... You can create a PipelineRun custom resource (CR) that includes a pipeline specification, as in the following example: Example of an inline specification in a PipelineRun CR apiVersion: operator.tekton.dev/v1 kind: PipelineRun metadata: name: pipelineRunInline spec: pipelineSpec: tasks: # ... You can create a TaskRun custom resource (CR) that includes a task specification, as in the following example: Example of an inline specification in a TaskRun CR apiVersion: operator.tekton.dev/v1 kind: TaskRun metadata: name: taskRunInline spec: taskSpec: steps: # ... You can disable inline specification in some or all of these cases. To disable the inline specification, set the disable-inline-spec field of the .spec.pipeline specification of the TektonConfig CR, as in the following example: Example configuration that disables inline specification apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: disable-inline-spec: "pipeline,pipelinerun,taskrun" # ... You can set the disable-inline-spec parameter to any single value or to a comma-separated list of multiple values. The following values for the parameter are valid: Table 3.2. Supported values for the disable-inline-spec parameter Value Description pipeline You cannot use a taskSpec: spec to define a task inside a Pipeline CR. Instead, you must use a taskRef: spec to incorporate a task from a Task CR or to specify a task using a resolver. pipelinerun You cannot use a pipelineSpec: spec to define a pipeline inside a PipelineRun CR. Instead, you must use a pipelineRef: spec to incorporate a pipeline from a Pipeline CR or to specify a pipeline using a resolver. taskrun You cannot use a taskSpec: spec to define a task inside a TaskRun CR. Instead, you must use a taskRef: spec to incorporate a task from a Task CR or to specify a task using a resolver. 3.12. Automatic pruning of task runs and pipeline runs Stale TaskRun and PipelineRun objects and their executed instances occupy physical resources that can be used for active runs. For optimal utilization of these resources, Red Hat OpenShift Pipelines provides a pruner component that automatically removes unused objects and their instances in various namespaces. Note You can configure the pruner for your entire installation by using the TektonConfig custom resource and modify configuration for a namespace by using namespace annotations. However, you cannot selectively auto-prune an individual task run or pipeline run in a namespace. 3.12.1. Configuring the pruner You can use the TektonConfig custom resource to configure periodic pruning of resources associated with pipeline runs and task runs. The following example corresponds to the default configuration: Example of the pruner configuration apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config # ... spec: pruner: resources: - taskrun - pipelinerun keep: 100 prune-per-resource: false schedule: "* 8 * * *" startingDeadlineSeconds: 60 # ... Table 3.3. Supported parameters for pruner configuration Parameter Description schedule The Cron schedule for running the pruner process. The default schedule runs the process at 08:00 every day. For more information about the Cron schedule syntax, see Cron schedule syntax in the Kubernetes documentation. resources The resource types to which the pruner applies. The available resource types are taskrun and pipelinerun keep The number of most recent resources of every type to keep. prune-per-resource If set to false , the value for the keep parameter denotes the total number of task runs or pipeline runs. For example, if keep is set to 100 , then the pruner keeps 100 most recent task runs and 100 most recent pipeline runs and removes all other resources. If set to true , the value for the keep parameter is calculated separately for pipeline runs referencing each pipeline and for task runs referencing each task. For example, if keep is set to 100 , then the pruner keeps 100 most recent pipeline runs for Pipeline1 , 100 most recent pipeline runs for Pipeline2 , 100 most recent task runs for Task1 , and so on, and removes all other resources. keep-since The maximum time for which to keep resources, in minutes. For example, to retain resources which were created not more than five days ago, set keep-since to 7200 . startingDeadlineSeconds This parameter is optional. If the pruner job is not started at the scheduled time for any reason, this setting configures the maximum time, in seconds, in which the job can still be started. If the job is not started within the specified time, OpenShift Pipelines considers this job failed and starts the pruner at the scheduled time. If you do not specify this parameter and the pruner job does not start at the scheduled time, OpenShift Pipelines attempts to start the job at any later time possible. Note The keep and keep-since parameters are mutually exclusive. Use only one of them in your configuration. 3.12.2. Annotations for automatically pruning task runs and pipeline runs To modify the configuration for automatic pruning of task runs and pipeline runs in a namespace, you can set annotations in the namespace. The following namespace annotations have the same meanings as the corresponding keys in the TektonConfig custom resource: operator.tekton.dev/prune.schedule operator.tekton.dev/prune.resources operator.tekton.dev/prune.keep operator.tekton.dev/prune.prune-per-resource operator.tekton.dev/prune.keep-since Note The operator.tekton.dev/prune.resources annotation accepts a comma-separated list. To prune both task runs and pipeline runs, set this annotation to "taskrun, pipelinerun" . The following additional namespace annotations are available: operator.tekton.dev/prune.skip : When set to true , the namespace for which the annotation is configured is not pruned. operator.tekton.dev/prune.strategy : Set the value of this annotation to either keep or keep-since . For example, the following annotations retain all task runs and pipeline runs created in the last five days and delete the older resources: Example of auto-pruning annotations kind: Namespace apiVersion: v1 # ... spec: annotations: operator.tekton.dev/prune.resources: "taskrun, pipelinerun" operator.tekton.dev/prune.keep-since: 7200 # ... 3.13. Additional resources Authenticating pipelines with repositories using secrets Managing non-versioned and versioned cluster tasks Creating pipeline templates in the Administrator perspective Pruning objects to reclaim resources
apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: performance: disable-ha: false buckets: 7 replicas: 5 threads-per-controller: 2 kube-api-qps: 5.0 kube-api-burst: 10 oc --namespace openshift-pipelines scale deployment openshift-pipelines-controller --replicas=3 apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: running-in-environment-with-injected-sidecars: true metrics.taskrun.duration-type: histogram metrics.pipelinerun.duration-type: histogram await-sidecar-readiness: true params: - name: enableMetrics value: 'true' default-service-account: pipeline require-git-ssh-secret-known-hosts: false enable-tekton-oci-bundles: false metrics.taskrun.level: task metrics.pipelinerun.level: pipeline enable-api-fields: stable enable-provenance-in-status: false enable-custom-tasks: true disable-creds-init: false disable-affinity-assistant: true apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: default-service-account: pipeline trigger: default-service-account: pipeline enable-api-fields: stable apiVersion: operator.tekton.dev/v1 kind: TektonConfig metadata: name: config spec: targetNamespaceMetadata: labels: {"example-label":"example-value"} annotations: {"example-annotation":"example-value"} apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: params: - name: enableMetrics value: 'false' apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: enable-bundles-resolver: true enable-cluster-resolver: true enable-git-resolver: true enable-hub-resolver: true apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: bundles-resolver-config: default-service-account: pipelines cluster-resolver-config: default-namespace: test git-resolver-config: server-url: localhost.com hub-resolver-config: default-tekton-hub-catalog: tekton apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: addon: params: - name: clusterTasks value: 'false' - name: pipelineTemplates value: 'false' - name: communityClusterTasks value: 'true' apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: hub: params: - name: enable-devconsole-integration value: false apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: params: - name: createRbacResource value: "false" apiVersion: operator.tekton.dev/v1 kind: Pipeline metadata: name: pipelineInline spec: tasks: taskSpec: apiVersion: operator.tekton.dev/v1 kind: PipelineRun metadata: name: pipelineRunInline spec: pipelineSpec: tasks: apiVersion: operator.tekton.dev/v1 kind: TaskRun metadata: name: taskRunInline spec: taskSpec: steps: apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pipeline: disable-inline-spec: "pipeline,pipelinerun,taskrun" apiVersion: operator.tekton.dev/v1alpha1 kind: TektonConfig metadata: name: config spec: pruner: resources: - taskrun - pipelinerun keep: 100 prune-per-resource: false schedule: "* 8 * * *" startingDeadlineSeconds: 60 kind: Namespace apiVersion: v1 spec: annotations: operator.tekton.dev/prune.resources: "taskrun, pipelinerun" operator.tekton.dev/prune.keep-since: 7200