Chapter 6. Building of container images using Buildah as a non-root user
Chapter 6. Building of container images using Buildah as a non-root user Running OpenShift Pipelines as the root user on a container can expose the container processes and the host to other potentially malicious resources. You can reduce this type of exposure by running the workload as a specific non-root user in the container. In most cases, you can run Buildah without root privileges by creating a custom task for building the image and configuring user namespaces in this task. If your image does not build successfully using this configuration, you can use custom service account (SA) and security context constraint (SCC) definitions; however, if you use this option, you must enable the Buildah step to raise its privileges ( allowPrivilegeEscalation: true ). 6.1. Running Buildah as a non-root user by configuring user namespaces Configuring user namespaces is the simplest way to run Buildah in a task as a non-root user. However, some images might not build using this option. Prerequisites You have installed the oc command-line utility. Procedure To create a copy of the buildah task, which is provided in the openshift-pipelines namespace, and to change the name of the copy to buildah-as-user , enter the following command: USD oc get task buildah -n openshift-pipelines -o yaml | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f - Edit the copied buildah task by entering the following command: USD oc edit task buildah-as-user In the new task, create annotations and stepTemplate sections, as shown in the following example: Example additions to the buildah-as-user task apiVersion: tekton.dev/v1 kind: Task metadata: annotations: io.kubernetes.cri-o.userns-mode: 'auto:size=65536;map-to-root=true' io.openshift.builder: 'true' name: assemble-containerimage namespace: pipeline-namespace spec: description: This task builds an image. # ... stepTemplate: env: - name: HOME value: /tekton/home image: USD(params.builder-image) imagePullPolicy: IfNotPresent name: '' resources: limits: cpu: '1' memory: 4Gi requests: cpu: 100m memory: 2Gi securityContext: capabilities: add: - SETFCAP runAsNonRoot: true runAsUser: 1000 1 workingDir: USD(workspaces.working-directory.path) # ... 1 The runAsUser: setting is not strictly necessary, because podTemplate is used. Use the new buildah-as-user task to build the image in your pipeline. 6.2. Running Buildah as a non-root user by defining a custom SA and SCC To run builds of container images using Buildah as a non-root user, you can perform the following steps: Define custom service account (SA) and security context constraint (SCC). Configure Buildah to use the build user with id 1000 . Start a task run with a custom config map, or integrate it with a pipeline run. 6.2.1. Configuring custom service account and security context constraint The default pipeline SA allows using a user id outside of the namespace range. To reduce dependency on the default SA, you can define a custom SA and SCC with necessary cluster role and role bindings for the build user with user id 1000 . Important At this time, enabling the allowPrivilegeEscalation setting is required for Buildah to run successfully in the container. With this setting, Buildah can leverage SETUID and SETGID capabilities when running as a non-root user. Procedure Create a custom SA and SCC with necessary cluster role and role bindings. Example: Custom SA and SCC for used id 1000 apiVersion: v1 kind: ServiceAccount metadata: name: pipelines-sa-userid-1000 1 --- kind: SecurityContextConstraints metadata: annotations: name: pipelines-scc-userid-1000 2 allowHostDirVolumePlugin: false allowHostIPC: false allowHostNetwork: false allowHostPID: false allowHostPorts: false allowPrivilegeEscalation: true 3 allowPrivilegedContainer: false allowedCapabilities: null apiVersion: security.openshift.io/v1 defaultAddCapabilities: null fsGroup: type: MustRunAs groups: - system:cluster-admins priority: 10 readOnlyRootFilesystem: false requiredDropCapabilities: - MKNOD - KILL runAsUser: 4 type: MustRunAs uid: 1000 seLinuxContext: type: MustRunAs supplementalGroups: type: RunAsAny users: [] volumes: - configMap - downwardAPI - emptyDir - persistentVolumeClaim - projected - secret --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: pipelines-scc-userid-1000-clusterrole 5 rules: - apiGroups: - security.openshift.io resourceNames: - pipelines-scc-userid-1000 resources: - securitycontextconstraints verbs: - use --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: pipelines-scc-userid-1000-rolebinding 6 roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: pipelines-scc-userid-1000-clusterrole subjects: - kind: ServiceAccount name: pipelines-sa-userid-1000 1 Define a custom SA. 2 Define a custom SCC created based on restricted privileges, with modified runAsUser field. 3 At this time, enabling the allowPrivilegeEscalation setting is required for Buildah to run successfully in the container. With this setting, Buildah can leverage SETUID and SETGID capabilities when running as a non-root user. 4 Restrict any pod that gets attached with the custom SCC through the custom SA to run as user id 1000 . 5 Define a cluster role that uses the custom SCC. 6 Bind the cluster role that uses the custom SCC to the custom SA. 6.2.2. Configuring Buildah to use build user You can define a Buildah task to use the build user with user id 1000 . Procedure Create a copy of the buildah task, which is provided in the openshift-pipelines namespace; change the name of the copy to buildah-as-user . USD oc get task buildah -n openshift-pipelines -o yaml | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f - Edit the copied buildah task. USD oc edit task buildah-as-user Example: Modified Buildah task with build user apiVersion: tekton.dev/v1 kind: Task metadata: name: buildah-as-user spec: description: >- Buildah task builds source into a container image and then pushes it to a container registry. Buildah Task builds source into a container image using Project Atomic's Buildah build tool.It uses Buildah's support for building from Dockerfiles, using its buildah bud command.This command executes the directives in the Dockerfile to assemble a container image, then pushes that image to a container registry. params: - name: IMAGE description: Reference of the image buildah will produce. - name: BUILDER_IMAGE description: The location of the buildah builder image. default: registry.redhat.io/rhel8/buildah@sha256:99cae35f40c7ec050fed3765b2b27e0b8bbea2aa2da7c16408e2ca13c60ff8ee - name: STORAGE_DRIVER description: Set buildah storage driver default: vfs - name: DOCKERFILE description: Path to the Dockerfile to build. default: ./Dockerfile - name: CONTEXT description: Path to the directory to use as context. default: . - name: TLSVERIFY description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry) default: "true" - name: FORMAT description: The format of the built container, oci or docker default: "oci" - name: BUILD_EXTRA_ARGS description: Extra parameters passed for the build command when building images. default: "" - description: Extra parameters passed for the push command when pushing images. name: PUSH_EXTRA_ARGS type: string default: "" - description: Skip pushing the built image name: SKIP_PUSH type: string default: "false" results: - description: Digest of the image just built. name: IMAGE_DIGEST type: string workspaces: - name: source steps: - name: build securityContext: runAsUser: 1000 1 image: USD(params.BUILDER_IMAGE) workingDir: USD(workspaces.source.path) script: | echo "Running as USER ID `id`" 2 buildah --storage-driver=USD(params.STORAGE_DRIVER) bud \ USD(params.BUILD_EXTRA_ARGS) --format=USD(params.FORMAT) \ --tls-verify=USD(params.TLSVERIFY) --no-cache \ -f USD(params.DOCKERFILE) -t USD(params.IMAGE) USD(params.CONTEXT) [[ "USD(params.SKIP_PUSH)" == "true" ]] && echo "Push skipped" && exit 0 buildah --storage-driver=USD(params.STORAGE_DRIVER) push \ USD(params.PUSH_EXTRA_ARGS) --tls-verify=USD(params.TLSVERIFY) \ --digestfile USD(workspaces.source.path)/image-digest USD(params.IMAGE) \ docker://USD(params.IMAGE) cat USD(workspaces.source.path)/image-digest | tee /tekton/results/IMAGE_DIGEST volumeMounts: - name: varlibcontainers mountPath: /home/build/.local/share/containers 3 volumes: - name: varlibcontainers emptyDir: {} 1 Run the container explicitly as the user id 1000 , which corresponds to the build user in the Buildah image. 2 Display the user id to confirm that the process is running as user id 1000 . 3 You can change the path for the volume mount as necessary. 6.2.3. Starting a task run with custom config map, or a pipeline run After defining the custom Buildah task, you can create a TaskRun object that builds an image as a build user with user id 1000 . In addition, you can integrate the TaskRun object as part of a PipelineRun object. Procedure Create a TaskRun object with a custom ConfigMap and Dockerfile objects. Example: A task run that runs Buildah as user id 1000 apiVersion: v1 data: Dockerfile: | ARG BASE_IMG=registry.access.redhat.com/ubi9/ubi FROM USDBASE_IMG AS buildah-runner RUN dnf -y update && \ dnf -y install git && \ dnf clean all CMD git kind: ConfigMap metadata: name: dockerfile 1 --- apiVersion: tekton.dev/v1 kind: TaskRun metadata: name: buildah-as-user-1000 spec: taskRunTemplate: serviceAccountName: pipelines-sa-userid-1000 2 params: - name: IMAGE value: image-registry.openshift-image-registry.svc:5000/test/buildahuser taskRef: kind: Task name: buildah-as-user workspaces: - configMap: name: dockerfile 3 name: source 1 Use a config map because the focus is on the task run, without any prior task that fetches some sources with a Dockerfile. 2 The name of the service account that you created. 3 Mount a config map as the source workspace for the buildah-as-user task. (Optional) Create a pipeline and a corresponding pipeline run. Example: A pipeline and corresponding pipeline run apiVersion: tekton.dev/v1 kind: Pipeline metadata: name: pipeline-buildah-as-user-1000 spec: params: - name: IMAGE - name: URL workspaces: - name: shared-workspace - name: sslcertdir optional: true tasks: - name: fetch-repository 1 taskRef: resolver: cluster params: - name: kind value: task - name: name value: git-clone - name: namespace value: openshift-pipelines workspaces: - name: output workspace: shared-workspace params: - name: URL value: USD(params.URL) - name: SUBDIRECTORY value: "" - name: DELETE_EXISTING value: "true" - name: buildah taskRef: name: buildah-as-user 2 runAfter: - fetch-repository workspaces: - name: source workspace: shared-workspace - name: sslcertdir workspace: sslcertdir params: - name: IMAGE value: USD(params.IMAGE) --- apiVersion: tekton.dev/v1 kind: PipelineRun metadata: name: pipelinerun-buildah-as-user-1000 spec: taskRunSpecs: - pipelineTaskName: buildah taskServiceAccountName: pipelines-sa-userid-1000 3 params: - name: URL value: https://github.com/openshift/pipelines-vote-api - name: IMAGE value: image-registry.openshift-image-registry.svc:5000/test/buildahuser pipelineRef: name: pipeline-buildah-as-user-1000 workspaces: - name: shared-workspace 4 volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Mi 1 Use the git-clone task to fetch the source containing a Dockerfile and build it using the modified Buildah task. 2 Refer to the modified Buildah task. 3 Use the service account that you created for the Buildah task. 4 Share data between the git-clone task and the modified Buildah task using a persistent volume claim (PVC) created automatically by the controller. Start the task run or the pipeline run. 6.3. Limitations of unprivileged builds The process for unprivileged builds works with most Dockerfile objects. However, there are some known limitations might cause a build to fail: Using the --mount=type=cache option might fail due to lack of necessay permissions issues. For more information, see this article . Using the --mount=type=secret option fails because mounting resources requires additionnal capabilities that are not provided by the custom SCC. Additional resources Managing security context constraints (SCCs)
oc get task buildah -n openshift-pipelines -o yaml | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f - oc edit task buildah-as-user apiVersion: tekton.dev/v1 kind: Task metadata: annotations: io.kubernetes.cri-o.userns-mode: 'auto:size=65536;map-to-root=true' io.openshift.builder: 'true' name: assemble-containerimage namespace: pipeline-namespace spec: description: This task builds an image. stepTemplate: env: - name: HOME value: /tekton/home image: USD(params.builder-image) imagePullPolicy: IfNotPresent name: '' resources: limits: cpu: '1' memory: 4Gi requests: cpu: 100m memory: 2Gi securityContext: capabilities: add: - SETFCAP runAsNonRoot: true runAsUser: 1000 1 workingDir: USD(workspaces.working-directory.path) apiVersion: v1 kind: ServiceAccount metadata: name: pipelines-sa-userid-1000 1 --- kind: SecurityContextConstraints metadata: annotations: name: pipelines-scc-userid-1000 2 allowHostDirVolumePlugin: false allowHostIPC: false allowHostNetwork: false allowHostPID: false allowHostPorts: false allowPrivilegeEscalation: true 3 allowPrivilegedContainer: false allowedCapabilities: null apiVersion: security.openshift.io/v1 defaultAddCapabilities: null fsGroup: type: MustRunAs groups: - system:cluster-admins priority: 10 readOnlyRootFilesystem: false requiredDropCapabilities: - MKNOD - KILL runAsUser: 4 type: MustRunAs uid: 1000 seLinuxContext: type: MustRunAs supplementalGroups: type: RunAsAny users: [] volumes: - configMap - downwardAPI - emptyDir - persistentVolumeClaim - projected - secret --- apiVersion: rbac.authorization.k8s.io/v1 kind: ClusterRole metadata: name: pipelines-scc-userid-1000-clusterrole 5 rules: - apiGroups: - security.openshift.io resourceNames: - pipelines-scc-userid-1000 resources: - securitycontextconstraints verbs: - use --- apiVersion: rbac.authorization.k8s.io/v1 kind: RoleBinding metadata: name: pipelines-scc-userid-1000-rolebinding 6 roleRef: apiGroup: rbac.authorization.k8s.io kind: ClusterRole name: pipelines-scc-userid-1000-clusterrole subjects: - kind: ServiceAccount name: pipelines-sa-userid-1000 oc get task buildah -n openshift-pipelines -o yaml | yq '. |= (del .metadata |= with_entries(select(.key == "name" )))' | yq '.kind="Task"' | yq '.metadata.name="buildah-as-user"' | oc create -f - oc edit task buildah-as-user apiVersion: tekton.dev/v1 kind: Task metadata: name: buildah-as-user spec: description: >- Buildah task builds source into a container image and then pushes it to a container registry. Buildah Task builds source into a container image using Project Atomic's Buildah build tool.It uses Buildah's support for building from Dockerfiles, using its buildah bud command.This command executes the directives in the Dockerfile to assemble a container image, then pushes that image to a container registry. params: - name: IMAGE description: Reference of the image buildah will produce. - name: BUILDER_IMAGE description: The location of the buildah builder image. default: registry.redhat.io/rhel8/buildah@sha256:99cae35f40c7ec050fed3765b2b27e0b8bbea2aa2da7c16408e2ca13c60ff8ee - name: STORAGE_DRIVER description: Set buildah storage driver default: vfs - name: DOCKERFILE description: Path to the Dockerfile to build. default: ./Dockerfile - name: CONTEXT description: Path to the directory to use as context. default: . - name: TLSVERIFY description: Verify the TLS on the registry endpoint (for push/pull to a non-TLS registry) default: "true" - name: FORMAT description: The format of the built container, oci or docker default: "oci" - name: BUILD_EXTRA_ARGS description: Extra parameters passed for the build command when building images. default: "" - description: Extra parameters passed for the push command when pushing images. name: PUSH_EXTRA_ARGS type: string default: "" - description: Skip pushing the built image name: SKIP_PUSH type: string default: "false" results: - description: Digest of the image just built. name: IMAGE_DIGEST type: string workspaces: - name: source steps: - name: build securityContext: runAsUser: 1000 1 image: USD(params.BUILDER_IMAGE) workingDir: USD(workspaces.source.path) script: | echo "Running as USER ID `id`" 2 buildah --storage-driver=USD(params.STORAGE_DRIVER) bud USD(params.BUILD_EXTRA_ARGS) --format=USD(params.FORMAT) --tls-verify=USD(params.TLSVERIFY) --no-cache -f USD(params.DOCKERFILE) -t USD(params.IMAGE) USD(params.CONTEXT) [[ "USD(params.SKIP_PUSH)" == "true" ]] && echo "Push skipped" && exit 0 buildah --storage-driver=USD(params.STORAGE_DRIVER) push USD(params.PUSH_EXTRA_ARGS) --tls-verify=USD(params.TLSVERIFY) --digestfile USD(workspaces.source.path)/image-digest USD(params.IMAGE) docker://USD(params.IMAGE) cat USD(workspaces.source.path)/image-digest | tee /tekton/results/IMAGE_DIGEST volumeMounts: - name: varlibcontainers mountPath: /home/build/.local/share/containers 3 volumes: - name: varlibcontainers emptyDir: {} apiVersion: v1 data: Dockerfile: | ARG BASE_IMG=registry.access.redhat.com/ubi9/ubi FROM USDBASE_IMG AS buildah-runner RUN dnf -y update && dnf -y install git && dnf clean all CMD git kind: ConfigMap metadata: name: dockerfile 1 --- apiVersion: tekton.dev/v1 kind: TaskRun metadata: name: buildah-as-user-1000 spec: taskRunTemplate: serviceAccountName: pipelines-sa-userid-1000 2 params: - name: IMAGE value: image-registry.openshift-image-registry.svc:5000/test/buildahuser taskRef: kind: Task name: buildah-as-user workspaces: - configMap: name: dockerfile 3 name: source apiVersion: tekton.dev/v1 kind: Pipeline metadata: name: pipeline-buildah-as-user-1000 spec: params: - name: IMAGE - name: URL workspaces: - name: shared-workspace - name: sslcertdir optional: true tasks: - name: fetch-repository 1 taskRef: resolver: cluster params: - name: kind value: task - name: name value: git-clone - name: namespace value: openshift-pipelines workspaces: - name: output workspace: shared-workspace params: - name: URL value: USD(params.URL) - name: SUBDIRECTORY value: "" - name: DELETE_EXISTING value: "true" - name: buildah taskRef: name: buildah-as-user 2 runAfter: - fetch-repository workspaces: - name: source workspace: shared-workspace - name: sslcertdir workspace: sslcertdir params: - name: IMAGE value: USD(params.IMAGE) --- apiVersion: tekton.dev/v1 kind: PipelineRun metadata: name: pipelinerun-buildah-as-user-1000 spec: taskRunSpecs: - pipelineTaskName: buildah taskServiceAccountName: pipelines-sa-userid-1000 3 params: - name: URL value: https://github.com/openshift/pipelines-vote-api - name: IMAGE value: image-registry.openshift-image-registry.svc:5000/test/buildahuser pipelineRef: name: pipeline-buildah-as-user-1000 workspaces: - name: shared-workspace 4 volumeClaimTemplate: spec: accessModes: - ReadWriteOnce resources: requests: storage: 100Mi