title: Kubernetes networking and firewall issue
body: <p>I try to expose a service. The goal is to access it from a cli (who know nothing about the cluster) with just his ip.</p>
  
  <p>I  have create a deployment of the image and then create the service by exposing it with nodport type.</p>
  
  <p>When I expose it on port 80, I can access to the svc but no other port working.
  I have try with adding iptables rules but not working.
  k8s doesn't do it automaticaly ?</p>
  
  <p>I use kubeadm on centos</p>
  
  <pre><code>swapoff -a
  systemctl disable firewalld
  systemctl stop firewalld
  setenforce 0
  </code></pre>
  
  <p>iptables-save</p>
  
  <pre><code># Generated by iptables-save v1.4.21 on Tue May 29 14:36:29 2018
  *nat
  :PREROUTING ACCEPT [4:812]
  :INPUT ACCEPT [4:812]
  :OUTPUT ACCEPT [2:120]
  :POSTROUTING ACCEPT [2:120]
  :DOCKER - [0:0]
  :KUBE-MARK-DROP - [0:0]
  :KUBE-MARK-MASQ - [0:0]
  :KUBE-NODEPORTS - [0:0]
  :KUBE-POSTROUTING - [0:0]
  :KUBE-SEP-5GE65UWKUZXJBCHC - [0:0]
  :KUBE-SEP-7PPXA5JT5ALVQPIV - [0:0]
  :KUBE-SEP-CTNKE6SP4U52GYW7 - [0:0]
  :KUBE-SEP-FO2LZ42N5CRZ6GVT - [0:0]
  :KUBE-SEP-HWIIVMKETERLJ5EZ - [0:0]
  :KUBE-SEP-IWBXS2W6OTONAINX - [0:0]
  :KUBE-SEP-JMXD3AUAOAUBCCUM - [0:0]
  :KUBE-SEP-PGKOTXVCEGHQUOMC - [0:0]
  :KUBE-SEP-SNPTLXDNVSPZ5ND2 - [0:0]
  :KUBE-SEP-T3255DXCOSMHHF7M - [0:0]
  :KUBE-SEP-ZKRGYSR5PGCBUGKL - [0:0]
  :KUBE-SERVICES - [0:0]
  :KUBE-SVC-BJM46V3U5RZHCFRZ - [0:0]
  :KUBE-SVC-EM2CH54TJVNBSB67 - [0:0]
  :KUBE-SVC-ERIFXISQEP7F7OF4 - [0:0]
  :KUBE-SVC-GRFCLVVBA4S2E2F4 - [0:0]
  :KUBE-SVC-JRXTEHDDTAFMSEAS - [0:0]
  :KUBE-SVC-NPX46M4PTMTKRN6Y - [0:0]
  :KUBE-SVC-Q6XJQ2I55QTBQCWT - [0:0]
  :KUBE-SVC-TCOU7JCQXEZGVUNU - [0:0]
  :KUBE-SVC-XGLOHA7QRQ3V22RZ - [0:0]
  -A PREROUTING -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
  -A PREROUTING -m addrtype --dst-type LOCAL -j DOCKER
  -A OUTPUT -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
  -A OUTPUT ! -d 127.0.0.0/8 -m addrtype --dst-type LOCAL -j DOCKER
  -A POSTROUTING -m comment --comment "kubernetes postrouting rules" -j KUBE-POSTROUTING
  -A POSTROUTING -s 172.17.0.0/16 ! -o docker0 -j MASQUERADE
  -A POSTROUTING -s 10.244.0.0/16 -d 10.244.0.0/16 -j RETURN
  -A POSTROUTING -s 10.244.0.0/16 ! -d 224.0.0.0/4 -j MASQUERADE
  -A POSTROUTING ! -s 10.244.0.0/16 -d 172.25.0.0/24 -j RETURN
  -A POSTROUTING ! -s 10.244.0.0/16 -d 10.244.0.0/16 -j MASQUERADE
  -A DOCKER -i docker0 -j RETURN
  -A KUBE-MARK-DROP -j MARK --set-xmark 0x8000/0x8000
  -A KUBE-MARK-MASQ -j MARK --set-xmark 0x4000/0x4000
  -A KUBE-NODEPORTS -p tcp -m comment --comment "default/dark-svc:" -m tcp --dport 30047 -j KUBE-MARK-MASQ
  -A KUBE-NODEPORTS -p tcp -m comment --comment "default/dark-svc:" -m tcp --dport 30047 -j KUBE-SVC-GRFCLVVBA4S2E2F4
  -A KUBE-NODEPORTS -p tcp -m comment --comment "default/dark-svc2:" -m tcp --dport 32205 -j KUBE-MARK-MASQ
  -A KUBE-NODEPORTS -p tcp -m comment --comment "default/dark-svc2:" -m tcp --dport 32205 -j KUBE-SVC-EM2CH54TJVNBSB67
  -A KUBE-POSTROUTING -m comment --comment "kubernetes service traffic requiring SNAT" -m mark --mark 0x4000/0x4000 -j MASQUERADE
  -A KUBE-SEP-5GE65UWKUZXJBCHC -s 172.17.0.9/32 -m comment --comment "default/dark-svc:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-5GE65UWKUZXJBCHC -p tcp -m comment --comment "default/dark-svc:" -m tcp -j DNAT --to-destination 172.17.0.9:80
  -A KUBE-SEP-7PPXA5JT5ALVQPIV -s 172.17.0.2/32 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-MARK-MASQ
  -A KUBE-SEP-7PPXA5JT5ALVQPIV -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp" -m tcp -j DNAT --to-destination 172.17.0.2:53
  -A KUBE-SEP-CTNKE6SP4U52GYW7 -s 172.17.0.5/32 -m comment --comment "kube-system/monitoring-influxdb:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-CTNKE6SP4U52GYW7 -p tcp -m comment --comment "kube-system/monitoring-influxdb:" -m tcp -j DNAT --to-destination 172.17.0.5:8086
  -A KUBE-SEP-FO2LZ42N5CRZ6GVT -s 172.17.0.10/32 -m comment --comment "default/dark-svc:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-FO2LZ42N5CRZ6GVT -p tcp -m comment --comment "default/dark-svc:" -m tcp -j DNAT --to-destination 172.17.0.10:80
  -A KUBE-SEP-HWIIVMKETERLJ5EZ -s 172.17.0.9/32 -m comment --comment "default/dark-svc2:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-HWIIVMKETERLJ5EZ -p tcp -m comment --comment "default/dark-svc2:" -m tcp -j DNAT --to-destination 172.17.0.9:8085
  -A KUBE-SEP-IWBXS2W6OTONAINX -s 172.17.0.4/32 -m comment --comment "kube-system/monitoring-grafana:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-IWBXS2W6OTONAINX -p tcp -m comment --comment "kube-system/monitoring-grafana:" -m tcp -j DNAT --to-destination 172.17.0.4:3000
  -A KUBE-SEP-JMXD3AUAOAUBCCUM -s 172.17.0.10/32 -m comment --comment "default/dark-svc2:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-JMXD3AUAOAUBCCUM -p tcp -m comment --comment "default/dark-svc2:" -m tcp -j DNAT --to-destination 172.17.0.10:8085
  -A KUBE-SEP-PGKOTXVCEGHQUOMC -s 10.66.222.223/32 -m comment --comment "default/kubernetes:https" -j KUBE-MARK-MASQ
  -A KUBE-SEP-PGKOTXVCEGHQUOMC -p tcp -m comment --comment "default/kubernetes:https" -m recent --set --name KUBE-SEP-PGKOTXVCEGHQUOMC --mask 255.255.255.255 --rsource -m tcp -j DNAT --to-destination 10.66.222.223:6443
  -A KUBE-SEP-SNPTLXDNVSPZ5ND2 -s 172.17.0.2/32 -m comment --comment "kube-system/kube-dns:dns" -j KUBE-MARK-MASQ
  -A KUBE-SEP-SNPTLXDNVSPZ5ND2 -p udp -m comment --comment "kube-system/kube-dns:dns" -m udp -j DNAT --to-destination 172.17.0.2:53
  -A KUBE-SEP-T3255DXCOSMHHF7M -s 172.17.0.6/32 -m comment --comment "kube-system/heapster:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-T3255DXCOSMHHF7M -p tcp -m comment --comment "kube-system/heapster:" -m tcp -j DNAT --to-destination 172.17.0.6:8082
  -A KUBE-SEP-ZKRGYSR5PGCBUGKL -s 172.17.0.8/32 -m comment --comment "kube-system/kubernetes-dashboard:" -j KUBE-MARK-MASQ
  -A KUBE-SEP-ZKRGYSR5PGCBUGKL -p tcp -m comment --comment "kube-system/kubernetes-dashboard:" -m tcp -j DNAT --to-destination 172.17.0.8:8443
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.96.0.10/32 -p tcp -m comment --comment "kube-system/kube-dns:dns-tcp cluster IP" -m tcp --dport 53 -j KUBE-SVC-ERIFXISQEP7F7OF4
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.108.154.85/32 -p tcp -m comment --comment "kube-system/monitoring-influxdb: cluster IP" -m tcp --dport 8086 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.108.154.85/32 -p tcp -m comment --comment "kube-system/monitoring-influxdb: cluster IP" -m tcp --dport 8086 -j KUBE-SVC-Q6XJQ2I55QTBQCWT
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.100.27.82/32 -p tcp -m comment --comment "default/dark-svc: cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.100.27.82/32 -p tcp -m comment --comment "default/dark-svc: cluster IP" -m tcp --dport 80 -j KUBE-SVC-GRFCLVVBA4S2E2F4
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.96.0.10/32 -p udp -m comment --comment "kube-system/kube-dns:dns cluster IP" -m udp --dport 53 -j KUBE-SVC-TCOU7JCQXEZGVUNU
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.108.155.161/32 -p tcp -m comment --comment "kube-system/heapster: cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.108.155.161/32 -p tcp -m comment --comment "kube-system/heapster: cluster IP" -m tcp --dport 80 -j KUBE-SVC-BJM46V3U5RZHCFRZ
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.96.203.18/32 -p tcp -m comment --comment "kube-system/kubernetes-dashboard: cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.96.203.18/32 -p tcp -m comment --comment "kube-system/kubernetes-dashboard: cluster IP" -m tcp --dport 443 -j KUBE-SVC-XGLOHA7QRQ3V22RZ
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.102.95.216/32 -p tcp -m comment --comment "kube-system/monitoring-grafana: cluster IP" -m tcp --dport 80 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.102.95.216/32 -p tcp -m comment --comment "kube-system/monitoring-grafana: cluster IP" -m tcp --dport 80 -j KUBE-SVC-JRXTEHDDTAFMSEAS
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.107.240.220/32 -p tcp -m comment --comment "default/dark-svc2: cluster IP" -m tcp --dport 8085 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.107.240.220/32 -p tcp -m comment --comment "default/dark-svc2: cluster IP" -m tcp --dport 8085 -j KUBE-SVC-EM2CH54TJVNBSB67
  -A KUBE-SERVICES ! -s 172.25.0.0/16 -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-MARK-MASQ
  -A KUBE-SERVICES -d 10.96.0.1/32 -p tcp -m comment --comment "default/kubernetes:https cluster IP" -m tcp --dport 443 -j KUBE-SVC-NPX46M4PTMTKRN6Y
  -A KUBE-SERVICES -m comment --comment "kubernetes service nodeports; NOTE: this must be the last rule in this chain" -m addrtype --dst-type LOCAL -j KUBE-NODEPORTS
  -A KUBE-SVC-BJM46V3U5RZHCFRZ -m comment --comment "kube-system/heapster:" -j KUBE-SEP-T3255DXCOSMHHF7M
  -A KUBE-SVC-EM2CH54TJVNBSB67 -m comment --comment "default/dark-svc2:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-JMXD3AUAOAUBCCUM
  -A KUBE-SVC-EM2CH54TJVNBSB67 -m comment --comment "default/dark-svc2:" -j KUBE-SEP-HWIIVMKETERLJ5EZ
  -A KUBE-SVC-ERIFXISQEP7F7OF4 -m comment --comment "kube-system/kube-dns:dns-tcp" -j KUBE-SEP-7PPXA5JT5ALVQPIV
  -A KUBE-SVC-GRFCLVVBA4S2E2F4 -m comment --comment "default/dark-svc:" -m statistic --mode random --probability 0.50000000000 -j KUBE-SEP-FO2LZ42N5CRZ6GVT
  -A KUBE-SVC-GRFCLVVBA4S2E2F4 -m comment --comment "default/dark-svc:" -j KUBE-SEP-5GE65UWKUZXJBCHC
  -A KUBE-SVC-JRXTEHDDTAFMSEAS -m comment --comment "kube-system/monitoring-grafana:" -j KUBE-SEP-IWBXS2W6OTONAINX
  -A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -m recent --rcheck --seconds 10800 --reap --name KUBE-SEP-PGKOTXVCEGHQUOMC --mask 255.255.255.255 --rsource -j KUBE-SEP-PGKOTXVCEGHQUOMC
  -A KUBE-SVC-NPX46M4PTMTKRN6Y -m comment --comment "default/kubernetes:https" -j KUBE-SEP-PGKOTXVCEGHQUOMC
  -A KUBE-SVC-Q6XJQ2I55QTBQCWT -m comment --comment "kube-system/monitoring-influxdb:" -j KUBE-SEP-CTNKE6SP4U52GYW7
  -A KUBE-SVC-TCOU7JCQXEZGVUNU -m comment --comment "kube-system/kube-dns:dns" -j KUBE-SEP-SNPTLXDNVSPZ5ND2
  -A KUBE-SVC-XGLOHA7QRQ3V22RZ -m comment --comment "kube-system/kubernetes-dashboard:" -j KUBE-SEP-ZKRGYSR5PGCBUGKL
  COMMIT
  # Completed on Tue May 29 14:36:29 2018
  # Generated by iptables-save v1.4.21 on Tue May 29 14:36:29 2018
  *filter
  :INPUT ACCEPT [2819:674508]
  :FORWARD DROP [0:0]
  :OUTPUT ACCEPT [2766:742748]
  :DOCKER - [0:0]
  :DOCKER-ISOLATION - [0:0]
  :KUBE-EXTERNAL-SERVICES - [0:0]
  :KUBE-FIREWALL - [0:0]
  :KUBE-FORWARD - [0:0]
  :KUBE-SERVICES - [0:0]
  :LOGGING - [0:0]
  -A INPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes externally-visible service portals" -j KUBE-EXTERNAL-SERVICES
  -A INPUT -j KUBE-FIREWALL
  -A INPUT -p tcp -m tcp --dport 35055 -j ACCEPT
  -A INPUT -j LOGGING
  -A FORWARD -j DOCKER-ISOLATION
  -A FORWARD -o docker0 -j DOCKER
  -A FORWARD -o docker0 -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
  -A FORWARD -i docker0 ! -o docker0 -j ACCEPT
  -A FORWARD -i docker0 -o docker0 -j ACCEPT
  -A FORWARD -m comment --comment "kubernetes forwarding rules" -j KUBE-FORWARD
  -A FORWARD -s 10.244.0.0/16 -j ACCEPT
  -A FORWARD -d 10.244.0.0/16 -j ACCEPT
  -A FORWARD -j ACCEPT
  -A OUTPUT -m conntrack --ctstate NEW -m comment --comment "kubernetes service portals" -j KUBE-SERVICES
  -A OUTPUT -j KUBE-FIREWALL
  -A DOCKER-ISOLATION -j RETURN
  -A KUBE-FIREWALL -m comment --comment "kubernetes firewall for dropping marked packets" -m mark --mark 0x8000/0x8000 -j DROP
  -A KUBE-FORWARD -m comment --comment "kubernetes forwarding rules" -m mark --mark 0x4000/0x4000 -j ACCEPT
  -A KUBE-FORWARD -s 172.25.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod source rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
  -A KUBE-FORWARD -d 172.25.0.0/16 -m comment --comment "kubernetes forwarding conntrack pod destination rule" -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
  COMMIT
  # Completed on Tue May 29 14:36:29 2018
  </code></pre>
  
  <p>kubectl cluster-info</p>
  
  <pre><code>Kubernetes master is running at https://10.66.222.223:6443
  Heapster is running at https://10.66.222.223:6443/api/v1/namespaces/kube-system/services/heapster/proxy
  KubeDNS is running at https://10.66.222.223:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
  monitoring-grafana is running at https://10.66.222.223:6443/api/v1/namespaces/kube-system/services/monitoring-grafana/proxy
  monitoring-influxdb is running at https://10.66.222.223:6443/api/v1/namespaces/kube-system/services/monitoring-influxdb/proxy
  </code></pre>
  
  <p>kubectl get pods</p>
  
  <pre><code>NAME                             READY     STATUS    RESTARTS   AGE
  dark-room-dep-577bf64bb8-9n5p7   1/1       Running   0          4d
  dark-room-dep-577bf64bb8-jmppg   1/1       Running   0          4d
  </code></pre>
  
  <p>kubectl get svc</p>
  
  <pre><code>NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)          AGE
  dark-svc     NodePort    10.100.27.82     &lt;none&gt;        80:30047/TCP     1d
  dark-svc2    NodePort    10.107.240.220   &lt;none&gt;        8085:32205/TCP   4h
  kubernetes   ClusterIP   10.96.0.1        &lt;none&gt;        443/TCP          12d
  </code></pre>
  
  <p>from master node:
  curl 10.66.222.223</p>
  
  <pre><code>curl: (7) Failed connect to 10.66.222.223:80; Connexion refusée
  </code></pre>
  
  <p>curl 127.0.0.1</p>
  
  <pre><code>curl: (7) Failed connect to 127.0.0.1:80; Connexion refusée
  </code></pre>
  
  <p>from a firefox client it working fine.</p>
  
  <p>If I try an other port:
  curl 10.66.222.223:8085</p>
  
  <pre><code>curl: (7) Failed connect to 10.66.222.223:8085; Connexion refusée
  </code></pre>
  
  <p>curl 127.0.0.1:8085</p>
  
  <pre><code>curl: (7) Failed connect to 127.0.0.1:8085; Connexion refusée
  </code></pre>
  
  <p>and when i try on firefox client it give me a connection refused.</p>