How to allow a non-root user to write to a mounted EFS in EKS I am having trouble configuring a statically provisioned EFS such that multiple pods, which run as a non-root user, can read and write the file system.
I am using the AWS EFS CSI Driver. My version info is as follows:
Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;18&quot;, GitVersion:&quot;v1.18.18&quot;, GitCommit:&quot;6f6ce59dc8fefde25a3ba0ef0047f4ec6662ef24&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2021-04-15T03:31:30Z&quot;, GoVersion:&quot;go1.13.15&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;18+&quot;, GitVersion:&quot;v1.18.9-eks-d1db3c&quot;, GitCommit:&quot;d1db3c46e55f95d6a7d3e5578689371318f95ff9&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2020-10-20T22:53:22Z&quot;, GoVersion:&quot;go1.13.15&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}

I followed the example from the github repo (https://github.com/kubernetes-sigs/aws-efs-csi-driver/tree/master/examples/kubernetes/multiple_pods) updating the volumeHandle appropriately. The busybox containers defined in the specs for the example are able to read and write the file system, but when I add the same PVC to a pod which does not run as the root user the pod is unable to write to the mounted EFS.
I have tried a couple other things to get this working as I expected it to:

just applying the annotation described here: https://kubernetes.io/docs/tasks/configure-pod-container/configure-persistent-volume-storage/#access-control to the persistent volume definition.
applying the aforementioned annotation and including securityContext.runAsGroup to the pod definition (with the appropriate value)
applying the annotation, the runAsGroup, as well as fsGroup for the Pod

None of these configurations allowed a non-root user to write to the mounted EFS.
What am I missing in terms of configuring a statically provisioned EFS so that multiple pods, all of which run as a non-root user, can read and write in the mounted EFS?
For reference here are the pod definitions:
apiVersion: v1
kind: Pod
metadata:
  name: app1
spec:
  containers:
  - name: app1
    image: busybox
    command: [&quot;/bin/sh&quot;]
    args: [&quot;-c&quot;, &quot;while true; do echo $(date -u) &gt;&gt; /data/out1.txt; sleep 5; done&quot;]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim
---
apiVersion: v1
kind: Pod
metadata:
  name: app2
spec:
  containers:
  - name: app2
    image: busybox
    command: [&quot;/bin/sh&quot;]
    args: [&quot;-c&quot;, &quot;while true; do echo $(date -u) &gt;&gt; /data/out2.txt; sleep 5; done&quot;]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim
---
apiVersion: v1
kind: Pod
metadata:
  name: app3
spec:
  containers:
  - name: app3
    image: busybox
    command: [&quot;/bin/sh&quot;]
    args: [&quot;-c&quot;, &quot;while true; do echo $(date -u) &gt;&gt; /data/out3.txt; sleep 5; done&quot;]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
  securityContext:
    runAsUser: 1000
    runAsGroup: 1337
    fsGroup: 1337
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim

And the SC/PVC/PV:
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: efs-sc
provisioner: efs.csi.aws.com
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: efs-claim
spec:
  accessModes:
    - ReadWriteMany
  storageClassName: efs-sc
  resources:
    requests:
      storage: 5Gi  
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: efs-pv
  annotations:
    pv.beta.kubernetes.io/gid: {{ .Values.groupId | quote }}
spec:
  capacity:
    storage: 5Gi
  volumeMode: Filesystem
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  storageClassName: efs-sc
  csi:
    driver: efs.csi.aws.com
    volumeHandle: fs-asdf123
If anyone comes across this later I resolved my issue by using an initContainer for any Pod that needed to write to the file system.
For example:
apiVersion: v1
kind: Pod
metadata:
  name: app3
spec:
  containers:
  - name: app3
    image: busybox
    command: [&quot;/bin/sh&quot;]
    args: [&quot;-c&quot;, &quot;while true; do echo $(date -u) &gt;&gt; /data/out3.txt; sleep 5; done&quot;]
    volumeMounts:
    - name: persistent-storage
      mountPath: /data
    securityContext:
      runAsGroup: 1337
  initContainers:
  - name: fs-owner-change
    image: busybox
    command:
    - chown
    - &quot;root:1337&quot;
    - &quot;/efs-fs&quot;
    volumeMounts
    - mountPath: /efs-fs
      name: cog-data
  securityContext:
    fsGroup: 1337
  volumes:
  - name: persistent-storage
    persistentVolumeClaim:
      claimName: efs-claim

The rest of the definitions match what was in my question.