ENVIRONMENT:

Kubernetes version: v1.16.3 OS: CentOS 7 Kernel: Linux k8s02-master01 3.10.0-1062.4.3.el7.x86_64 #1 SMP Wed Nov 13 23:58:53 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux

WHAT HAPPENED:

I have a Wordpress Deployment running a container built from a custom Apache/Wordpress image. The image exposes port 8080 instead of 80 (Dockerfile below). The Pod is exposed to the world through Traefik reverse proxy. Everything works fine without any liveness or readiness checks. Pod gets ready and Wordpress is accessible from https://www.example.com/.

I tried adding liveness and readiness probes and they both repeatedly fail with "connection refused". When I remove both probes and reapply the Deployment, it works again. It works until the probe hits the failure threshhold, at which point the container goes into an endless restart loop and becomes unaccessible.

POD EVENTS:

Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal Scheduled <unknown> default-scheduler Successfully assigned development/blog-wordpress-5dbcd9c7c7-kdgpc to gg-k8s02-worker02 Normal Killing 16m (x2 over 17m) kubelet, gg-k8s02-worker02 Container blog-wordpress failed liveness probe, will be restarted Normal Created 16m (x3 over 18m) kubelet, gg-k8s02-worker02 Created container blog-wordpress Normal Started 16m (x3 over 18m) kubelet, gg-k8s02-worker02 Started container blog-wordpress Normal Pulled 13m (x5 over 18m) kubelet, gg-k8s02-worker02 Container image "wordpress-test:test12" already present on machine Warning Unhealthy 8m17s (x35 over 18m) kubelet, gg-k8s02-worker02 Liveness probe failed: Get http://10.244.3.83/: dial tcp 10.244.3.83:80: connect: connection refused Warning BackOff 3m27s (x27 over 11m) kubelet, gg-k8s02-worker02 Back-off restarting failed container

POD LOGS:

WordPress not found in /var/www/html - copying now... WARNING: /var/www/html is not empty! (copying anyhow) Complete! WordPress has been successfully copied to /var/www/html AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 10.244.3.83. Set the 'ServerName' directive globally to suppress this message AH00558: apache2: Could not reliably determine the server's fully qualified domain name, using 10.244.3.83. Set the 'ServerName' directive globally to suppress this message [Wed Dec 11 06:39:07.502247 2019] [mpm_prefork:notice] [pid 1] AH00163: Apache/2.4.38 (Debian) PHP/7.3.11 configured -- resuming normal operations [Wed Dec 11 06:39:07.502323 2019] [core:notice] [pid 1] AH00094: Command line: 'apache2 -D FOREGROUND' 10.244.3.1 - - [11/Dec/2019:06:39:18 +0000] "GET /index.php HTTP/1.1" 301 264 "-" "kube-probe/1.16" 10.244.3.1 - - [11/Dec/2019:06:39:33 +0000] "GET /index.php HTTP/1.1" 301 264 "-" "kube-probe/1.16" 10.244.3.1 - - [11/Dec/2019:06:39:48 +0000] "GET /index.php HTTP/1.1" 301 264 "-" "kube-probe/1.16" 10.244.3.1 - - [11/Dec/2019:06:40:03 +0000] "GET /index.php HTTP/1.1" 301 264 "-" "kube-probe/1.16" 10.244.3.1 - - [11/Dec/2019:06:40:18 +0000] "GET /index.php HTTP/1.1" 301 264 "-" "kube-probe/1.16"

DOCKERFILE ("wordpress-test:test12"):

FROM wordpress:5.2.4-apache RUN sed -i 's/Listen 80/Listen 8080/g' /etc/apache2/ports.conf; RUN sed -i 's/:80/:8080/g' /etc/apache2/sites-enabled/000-default.conf; # RUN sed -i 's/#ServerName www.example.com/ServerName localhost/g' /etc/apache2/sites-enabled/000-default.conf; EXPOSE 8080 CMD ["apache2-foreground"]

DEPLOYMENT:

apiVersion: apps/v1 kind: Deployment metadata: name: blog-wordpress namespace: development labels: app: blog spec: selector: matchLabels: app: blog tier: wordpress replicas: 4 strategy: type: RollingUpdate rollingUpdate: maxSurge: 2 maxUnavailable: 2 template: metadata: labels: app: blog tier: wordpress spec: volumes: - name: blog-wordpress persistentVolumeClaim: claimName: blog-wordpress containers: - name: blog-wordpress # image: wordpress:5.2.4-apache image: wordpress-test:test12 securityContext: runAsUser: 65534 allowPrivilegeEscalation: false capabilities: add: - "NET_ADMIN" - "NET_BIND_SERVICE" - "SYS_TIME" resources: requests: cpu: "250m" memory: "64Mi" limits: cpu: "500m" memory: "128Mi" ports: - name: liveness-port containerPort: 8080 readinessProbe: initialDelaySeconds: 15 httpGet: path: /index.php port: 8080 timeoutSeconds: 15 periodSeconds: 15 failureThreshold: 5 livenessProbe: initialDelaySeconds: 10 httpGet: path: /index.php port: 8080 timeoutSeconds: 10 periodSeconds: 15 failureThreshold: 5 env: # Database - name: WORDPRESS_DB_HOST value: blog-mysql - name: WORDPRESS_DB_NAME value: wordpress - name: WORDPRESS_DB_USER valueFrom: secretKeyRef: name: blog-mysql key: username - name: WORDPRESS_DB_PASSWORD valueFrom: secretKeyRef: name: blog-mysql key: password - name: WORDPRESS_TABLE_PREFIX value: wp_ - name: WORDPRESS_AUTH_KEY valueFrom: secretKeyRef: name: blog-wordpress key: auth-key - name: WORDPRESS_SECURE_AUTH_KEY valueFrom: secretKeyRef: name: blog-wordpress key: secure-auth-key - name: WORDPRESS_LOGGED_IN_KEY valueFrom: secretKeyRef: name: blog-wordpress key: logged-in-key - name: WORDPRESS_NONCE_KEY valueFrom: secretKeyRef: name: blog-wordpress key: nonce-key - name: WORDPRESS_AUTH_SALT valueFrom: secretKeyRef: name: blog-wordpress key: auth-salt - name: WORDPRESS_SECURE_AUTH_SALT valueFrom: secretKeyRef: name: blog-wordpress key: secure-auth-salt - name: WORDPRESS_LOGGED_IN_SALT valueFrom: secretKeyRef: name: blog-wordpress key: logged-in-salt - name: WORDPRESS_NONCE_SALT valueFrom: secretKeyRef: name: blog-wordpress key: nonce-salt - name: WORDPRESS_CONFIG_EXTRA value: | define('WPLANG', 'fr_FR'); define('WP_CACHE', false); define('WP_MEMORY_LIMIT', '64M'); volumeMounts: - name: blog-wordpress mountPath: "/var/www/html/wp-content"

DEPLOYMENT SERVICE:

apiVersion: v1 kind: Service metadata: name: blog-wordpress namespace: development labels: app: blog spec: ports: - protocol: TCP port: 80 targetPort: 8080 selector: app: blog tier: wordpress type: ClusterIP

TRAEFIK INGRESSROUTE:

## # HTTP ## apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: blog namespace: development spec: entryPoints: - http routes: - match: Host(`example.com`) kind: Rule services: - name: blog-wordpress port: 80 middlewares: - name: redirect-to-https namespace: kube-system --- ## # HTTPS ## apiVersion: traefik.containo.us/v1alpha1 kind: IngressRoute metadata: name: blog-https namespace: development spec: entryPoints: - https routes: - match: Host(`example.com`) && PathPrefix(`/`) kind: Rule services: - name: blog-wordpress port: 80 tls: certResolver: letsencrypt

Thank you!