title: Connecting to redis (from helm chart) inside Kubernetes cluster
body: <p>I've deployed a Kubernetes cluster via kubeadm on an Openstack cluster with 1 master and 1 node and allowed scheduling pods to both. The internal network is provided by Calico with default parameters. I install helm. I then used the redis helm chart to deploy redis, with the following config:</p>
  
  <pre><code>## Global Docker image registry
  ## Please, note that this will override the image registry for all the images, including dependencies, configured to use the global value
  ##
  # global:
  #   imageRegistry:
  
  ## Bitnami Redis image version
  ## ref: https://hub.docker.com/r/bitnami/redis/tags/
  ##
  image:
    registry: docker.io
    repository: bitnami/redis
    ## Bitnami Redis image tag
    ## ref: https://github.com/bitnami/bitnami-docker-redis#supported-tags-and-respective-dockerfile-links
    ##
    tag: 5.0.3
    ## Specify a imagePullPolicy
    ## Defaults to 'Always' if image tag is 'latest', else set to 'IfNotPresent'
    ## ref: http://kubernetes.io/docs/user-guide/images/#pre-pulling-images
    ##
    pullPolicy: IfNotPresent
    ## Optionally specify an array of imagePullSecrets.
    ## Secrets must be manually created in the namespace.
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
    ##
    # pullSecrets:
    #   - myRegistrKeySecretName
  
  ## Cluster settings
  cluster:
    enabled: true
    slaveCount: 1
  
  networkPolicy:
    ## Specifies whether a NetworkPolicy should be created
    ##
    enabled: true
  
    ## The Policy model to apply. When set to false, only pods with the correct
    ## client label will have network access to the port Redis is listening
    ## on. When true, Redis will accept connections from any source
    ## (with the correct destination port).
    ##
    allowExternal: true
  
  serviceAccount:
    ## Specifies whether a ServiceAccount should be created
    ##
    create: false
    ## The name of the ServiceAccount to use.
    ## If not set and create is true, a name is generated using the fullname template
    name:
  
  rbac:
    ## Specifies whether RBAC resources should be created
    ##
    create: false
  
    role:
      ## Rules to create. It follows the role specification
      # rules:
      #  - apiGroups:
      #    - extensions
      #    resources:
      #      - podsecuritypolicies
      #    verbs:
      #      - use
      #    resourceNames:
      #      - gce.unprivileged
      rules: []
  
  ## Use password authentication
  usePassword: false
  ## Redis password (both master and slave)
  ## Defaults to a random 10-character alphanumeric string if not set and usePassword is true
  ## ref: https://github.com/bitnami/bitnami-docker-redis#setting-the-server-password-on-first-run
  ##
  password:
  ## Use existing secret (ignores previous password)
  # existingSecret:
  
  ## Mount secrets as files instead of environment variables
  usePasswordFile: false
  
  ## Persist data to a persistent volume
  persistence:
    ## A manually managed Persistent Volume and Claim
    ## Requires persistence.enabled: true
    ## If defined, PVC must be created manually before volume will be bound
    existingClaim: redis-store-pvc
  
  ##
  ## Redis Master parameters
  ##
  master:
    ## Redis port
    port: 6379
    ## Redis command arguments
    ##
    ## Can be used to specify command line arguments, for example:
    ##
    command:
    - "/run.sh"
    ## Redis additional command line flags
    ##
    ## Can be used to specify command line flags, for example:
    ##
    ## extraFlags:
    ##  - "--maxmemory-policy volatile-ttl"
    ##  - "--repl-backlog-size 1024mb"
    extraFlags: []
    ## Comma-separated list of Redis commands to disable
    ##
    ## Can be used to disable Redis commands for security reasons.
    ## Commands will be completely disabled by renaming each to an empty string.
    ## ref: https://redis.io/topics/security#disabling-of-specific-commands
    ##
    disableCommands:
  #  - FLUSHDB
  #  - FLUSHALL
  
    ## Redis Master additional pod labels and annotations
    ## ref: https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/
    podLabels: {}
    podAnnotations: {}
  
    ## Redis Master resource requests and limits
    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
    # resources:
    #   requests:
    #     memory: 256Mi
    #     cpu: 100m
    ## Use an alternate scheduler, e.g. "stork".
    ## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
    ##
    # schedulerName:
  
    ## Configure extra options for Redis Master liveness and readiness probes
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
    ##
    livenessProbe:
      enabled: true
      initialDelaySeconds: 5
      periodSeconds: 5
      timeoutSeconds: 5
      successThreshold: 1
      failureThreshold: 5
    readinessProbe:
      enabled: true
      initialDelaySeconds: 5
      periodSeconds: 5
      timeoutSeconds: 1
      successThreshold: 1
      failureThreshold: 5
  
    ## Redis Master Node selectors and tolerations for pod assignment
    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#nodeselector
    ## ref: https://kubernetes.io/docs/concepts/configuration/assign-pod-node/#taints-and-tolerations-beta-feature
    ##
    # nodeSelector: {"beta.kubernetes.io/arch": "amd64"}
    # tolerations: []
    ## Redis Master pod/node affinity/anti-affinity
    ##
    affinity: {}
  
    ## Redis Master Service properties
    service:
      ##  Redis Master Service type
      type: ClusterIP
      port: 6379
  
      ## Specify the nodePort value for the LoadBalancer and NodePort service types.
      ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
      ##
      # nodePort:
  
      ## Provide any additional annotations which may be required. This can be used to
      ## set the LoadBalancer service type to internal only.
      ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
      ##
      annotations: {}
      loadBalancerIP:
  
    ## Redis Master Pod Security Context
    securityContext:
      enabled: true
      fsGroup: 1001
      runAsUser: 1001
  
    ## Enable persistence using Persistent Volume Claims
    ## ref: http://kubernetes.io/docs/user-guide/persistent-volumes/
    ##
    persistence:
      enabled: true
      ## The path the volume will be mounted at, useful when using different
      ## Redis images.
      path: /redis-store
      ## The subdirectory of the volume to mount to, useful in dev environments
      ## and one PV for multiple services.
      subPath: /data
      ## redis data Persistent Volume Storage Class
      ## If defined, storageClassName: &lt;storageClass&gt;
      ## If set to "-", storageClassName: "", which disables dynamic provisioning
      ## If undefined (the default) or set to null, no storageClassName spec is
      ##   set, choosing the default provisioner.  (gp2 on AWS, standard on
      ##   GKE, AWS &amp; OpenStack)
      ##
      # storageClass: "-"
      storageClass: local-storage
      accessModes:
      - ReadWriteOnce
      size: 90Gi
  
    ## Update strategy, can be set to RollingUpdate or onDelete by default.
    ## https://kubernetes.io/docs/tutorials/stateful-application/basic-stateful-set/#updating-statefulsets
    statefulset:
      updateStrategy: RollingUpdate
      ## Partition update strategy
      ## https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/#partitions
      # rollingUpdatePartition:
  
    ## Redis Master pod priorityClassName
    # priorityClassName: {}
  
  
  ##
  ## Redis Slave properties
  ## Note: service.type is a mandatory parameter
  ## The rest of the parameters are either optional or, if undefined, will inherit those declared in Redis Master
  ##
  slave:
    ## Slave Service properties
    service:
      ## Redis Slave Service type
      type: ClusterIP
      ## Specify the nodePort value for the LoadBalancer and NodePort service types.
      ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#type-nodeport
      ##
      # nodePort:
  
      ## Provide any additional annotations which may be required. This can be used to
      ## set the LoadBalancer service type to internal only.
      ## ref: https://kubernetes.io/docs/concepts/services-networking/service/#internal-load-balancer
      ##
      annotations: {}
      loadBalancerIP:
  
    ## Redis port
    # port: 6379
    ## Redis extra flags
    # extraFlags: []
    ## List of Redis commands to disable
    # disableCommands: []
  
    ## Redis Slave pod/node affinity/anti-affinity
    ##
    affinity: {}
  
    ## Configure extra options for Redis Slave liveness and readiness probes
    ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/#configure-probes)
    ##
    # livenessProbe:
    #   enabled: true
    #   initialDelaySeconds: 30
    #   periodSeconds: 10
    #   timeoutSeconds: 5
    #   successThreshold: 1
    #   failureThreshold: 5
    # readinessProbe:
    #   enabled: true
    #   initialDelaySeconds: 5
    #   periodSeconds: 10
    #   timeoutSeconds: 10
    #   successThreshold: 1
    #   failureThreshold: 5
  
    ## Redis slave Resource
    # resources:
    #   requests:
    #     memory: 256Mi
    #     cpu: 100m
  
    ## Redis slave selectors and tolerations for pod assignment
    # nodeSelector: {"beta.kubernetes.io/arch": "amd64"}
    # tolerations: []
  
    ## Use an alternate scheduler, e.g. "stork".
    ## ref: https://kubernetes.io/docs/tasks/administer-cluster/configure-multiple-schedulers/
    ##
    # schedulerName:
  
    ## Redis slave pod Annotation and Labels
    # podLabels: {}
    # podAnnotations: {}
  
    ## Redis slave pod Security Context
    # securityContext:
    #   enabled: true
    #   fsGroup: 1001
    #   runAsUser: 1001
  
    ## Redis slave pod priorityClassName
    # priorityClassName: {}
  
  ## Prometheus Exporter / Metrics
  ##
  metrics:
    enabled: false
  
    image:
      registry: docker.io
      repository: oliver006/redis_exporter
      tag: v0.25.0
      pullPolicy: IfNotPresent
      ## Optionally specify an array of imagePullSecrets.
      ## Secrets must be manually created in the namespace.
      ## ref: https://kubernetes.io/docs/tasks/configure-pod-container/pull-image-private-registry/
      ##
      # pullSecrets:
      #   - myRegistrKeySecretName
  
    service:
      type: ClusterIP
      ## Use serviceLoadBalancerIP to request a specific static IP,
      ## otherwise leave blank
      # loadBalancerIP:
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "9121"
  
    ## Metrics exporter resource requests and limits
    ## ref: http://kubernetes.io/docs/user-guide/compute-resources/
    ##
    # resources: {}
  
    ## Extra arguments for Metrics exporter, for example:
    ## extraArgs:
    ##   check-keys: myKey,myOtherKey
    # extraArgs: {}
  
    ## Metrics exporter labels and tolerations for pod assignment
    # nodeSelector: {"beta.kubernetes.io/arch": "amd64"}
    # tolerations: []
  
    ## Metrics exporter pod Annotation and Labels
    # podAnnotations: {}
    # podLabels: {}
  
    # Enable this if you're using https://github.com/coreos/prometheus-operator
    serviceMonitor:
      enabled: false
      ## Specify a namespace if needed
      # namespace: monitoring
      # fallback to the prometheus default unless specified
      # interval: 10s
      ## Defaults to what's used if you follow CoreOS [Prometheus Install Instructions](https://github.com/helm/charts/tree/master/stable/prometheus-operator#tldr)
      ## [Prometheus Selector Label](https://github.com/helm/charts/tree/master/stable/prometheus-operator#prometheus-operator-1)
      ## [Kube Prometheus Selector Label](https://github.com/helm/charts/tree/master/stable/prometheus-operator#exporters)
      selector:
        prometheus: kube-prometheus
  
    ## Metrics exporter pod priorityClassName
    # priorityClassName: {}
  
  ##
  ## Init containers parameters:
  ## volumePermissions: Change the owner of the persist volume mountpoint to RunAsUser:fsGroup
  ##
  volumePermissions:
    image:
      registry: docker.io
      repository: bitnami/minideb
      tag: latest
      pullPolicy: IfNotPresent
  
  ## Redis config file
  ## ref: https://redis.io/topics/config
  ##
  configmap: |-
    # maxmemory-policy volatile-lru
  </code></pre>
  
  <p>So far so good, the pods deploy successfully. I can see the network policy being created:</p>
  
  <pre><code>[root@tracker redis]# kubectl describe  networkpolicy redis 
  Name:         redis
  Namespace:    default
  Created on:   2019-01-26 10:09:23 +0000 UTC
  Labels:       app=redis
                chart=redis-5.3.0
                heritage=Tiller
                release=redis
  Annotations:  &lt;none&gt;
  Spec:
    PodSelector:     app=redis,release=redis
    Allowing ingress traffic:
      To Port: 6379/TCP
      From: &lt;any&gt; (traffic not restricted by source)
    Allowing egress traffic:
      &lt;none&gt; (Selected pods are isolated for egress connectivity)
    Policy Types: Ingress
  </code></pre>
  
  <p>I now launched a pod with the redis client to test connecting to the redis server and the client is not able to establish a connection with redis-master. I checked that the name properly resolves to the service IP. I tried accessing the service by IP, and the pod that runs the redis-master directly by ip, and none of these work. </p>
  
  <p>I can log directly onto the pod that runs redis-master and create a client session there. I can actually port forward from outside the kubernetes cluster using</p>
  
  <pre><code>kubectl port-forward --namespace default svc/redis-master 6379:6379 &amp;
  redis-cli -h 127.0.0.1 -p 6379
  </code></pre>
  
  <p>and this does work. I installed netstat on the redis-master pod and it looks like the service is bound on 0.0.0.0 so should be accepting outside connections:</p>
  
  <pre><code>Every 2.0s: netstat -at                                                                                      
  
      redis-master-0: Sat Jan 26 11:24:13 2019
  
  Active Internet connections (servers and established)
  Proto Recv-Q Send-Q Local Address           Foreign Address         State
  tcp        0      0 0.0.0.0:6379            0.0.0.0:*               LISTEN
  </code></pre>
  
  <p>I guess it's something about the internal cluster networking that's tripping this up but I don't know how to debug it further. Any help is appreciated.</p>