title: network: failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address
body: <p><strong>What I try to do:</strong>
  EKS with both Linux and Windows (2019) nodes, nginx pod on Linux should access IIS pod on Windows.</p>
  
  <p><strong>The issue:</strong>
  The Windows pods don't start.</p>
  
  <p><strong>Log:</strong></p>
  
  <pre><code>E0526 10:59:31.963644    4392 pod_workers.go:186] Error syncing pod b35e92cc-7fa2-11e9-b07b-0ac0c740dc70 ("phoenix-57b76c578c-cczs2_kaltura(b35e92cc-7fa2-11e9-b07b-0ac0c740dc70)"), skipping: failed to "KillPodSandbox" for "b35e92cc-7fa2-11e9-b07b-0ac0c740dc70" with KillPodSandboxError: "rpc error: code = Unknown desc = NetworkPlugin cni failed to teardown pod \"phoenix-57b76c578c-cczs2_kaltura\" network: failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address"
  I0526 10:59:37.049583    5020 proxier.go:117] Hns Endpoint resource, {"ID":"9638A3AE-DCB9-4F85-B682-9D2879E09D98","Name":"Ethernet","VirtualNetwork":"82363D68-76A8-4225-8EFC-76F179330CC1","VirtualNetworkName":"vpcbr0a05d9b85b68","Policies":[{"Type":"L2Driver"}],"MacAddress":"00:11:22:33:44:55","IPAddress":"172.31.32.190","PrefixLength":20,"IsRemoteEndpoint":true}
  I0526 10:59:37.051589    5020 proxier.go:117] Hns Endpoint resource, {"ID":"8A4C02B1-537B-4650-ADC5-BA24598E3ABA","Name":"Ethernet","VirtualNetwork":"82363D68-76A8-4225-8EFC-76F179330CC1","VirtualNetworkName":"vpcbr0a05d9b85b68","Policies":[{"Type":"L2Driver"}],"MacAddress":"00:11:22:33:44:55","IPAddress":"172.31.36.90","PrefixLength":20,"IsRemoteEndpoint":true}
  E0526 10:59:37.064582    5020 proxier.go:1034] Policy creation failed: hnsCall failed in Win32: The provided policy configuration is invalid or missing parameters. (0x803b000d)
  E0526 10:59:37.064582    5020 proxier.go:1018] Endpoint information not available for service kaltura/phoenix:https. Not applying any policy
  E0526 10:59:38.433836    4392 kubelet_network.go:102] Failed to ensure that nat chain KUBE-MARK-DROP exists: error creating chain "KUBE-MARK-DROP": executable file not found in %PATH%: 
  E0526 10:59:39.362013    4392 helpers.go:735] eviction manager: failed to construct signal: "allocatableMemory.available" error: system container "pods" not found in metrics
  W0526 10:59:39.362013    4392 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
  E0526 10:59:48.965710    4392 cni.go:280] Error deleting network: failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address
  E0526 10:59:48.965710    4392 remote_runtime.go:115] StopPodSandbox "04961285217a628c589467359f6ff6335355c73fdd61f3c975215105a6c307f6" from runtime service failed: rpc error: code = Unknown desc = NetworkPlugin cni failed to teardown pod "phoenix-57b76c578c-cczs2_kaltura" network: failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address
  E0526 10:59:48.965710    4392 kuberuntime_manager.go:799] Failed to stop sandbox {"docker" "04961285217a628c589467359f6ff6335355c73fdd61f3c975215105a6c307f6"}
  E0526 10:59:48.965710    4392 kuberuntime_manager.go:594] killPodWithSyncResult failed: failed to "KillPodSandbox" for "b35e92cc-7fa2-11e9-b07b-0ac0c740dc70" with KillPodSandboxError: "rpc error: code = Unknown desc = NetworkPlugin cni failed to teardown pod \"phoenix-57b76c578c-cczs2_kaltura\" network: failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address"
  E0526 10:59:48.965710    4392 pod_workers.go:186] Error syncing pod b35e92cc-7fa2-11e9-b07b-0ac0c740dc70 ("phoenix-57b76c578c-cczs2_kaltura(b35e92cc-7fa2-11e9-b07b-0ac0c740dc70)"), skipping: failed to "KillPodSandbox" for "b35e92cc-7fa2-11e9-b07b-0ac0c740dc70" with KillPodSandboxError: "rpc error: code = Unknown desc = NetworkPlugin cni failed to teardown pod \"phoenix-57b76c578c-cczs2_kaltura\" network: failed to parse Kubernetes args: pod does not have label vpc.amazonaws.com/PrivateIPv4Address"
  E0526 10:59:49.368785    4392 helpers.go:735] eviction manager: failed to construct signal: "allocatableMemory.available" error: system container "pods" not found in metrics
  W0526 10:59:49.368785    4392 helpers.go:808] eviction manager: no observation found for eviction signal nodefs.inodesFree
  </code></pre>
  
  <p>kubectl -n kaltura describe pods phoenix-695b5bdff8-zzbq6</p>
  
  <pre><code>Name:               phoenix-695b5bdff8-zzbq6
  Namespace:          kaltura
  Priority:           0
  PriorityClassName:  &lt;none&gt;
  Node:               ip-10-10-12-97.us-east-2.compute.internal/10.10.12.97
  Start Time:         Tue, 28 May 2019 12:30:48 +0300
  Labels:             app.kubernetes.io/instance=kaltura-core
                      app.kubernetes.io/name=phoenix
                      pod-template-hash=2516168994
  Annotations:        &lt;none&gt;
  Status:             Pending
  IP:
  Controlled By:      ReplicaSet/phoenix-695b5bdff8
  Containers:
    kaltura:
      Container ID:
      Image:          &lt;my-account-id&gt;.dkr.ecr.us-east-2.amazonaws.com/vfd1-phoenix:latest
      Image ID:
      Port:           8040/TCP
      Host Port:      0/TCP
      State:          Waiting
        Reason:       ContainerCreating
      Ready:          False
      Restart Count:  0
      Liveness:       http-get http://:80/tvp_api delay=0s timeout=1s period=10s #success=1 #failure=3
      Readiness:      http-get http://:80/tvp_api delay=0s timeout=1s period=10s #success=1 #failure=3
      Environment:
        TCM_SECTION:  kaltura-core
        TCM_URL:      https://10.10.12.99
      Mounts:
        /var/run/secrets/kubernetes.io/serviceaccount from default-token-jdd98 (ro)
  Conditions:
    Type              Status
    Initialized       True
    Ready             False
    ContainersReady   False
    PodScheduled      True
  Volumes:
    default-token-jdd98:
      Type:        Secret (a volume populated by a Secret)
      SecretName:  default-token-jdd98
      Optional:    false
  QoS Class:       BestEffort
  Node-Selectors:  beta.kubernetes.io/arch=amd64
                   beta.kubernetes.io/os=windows
                   kaltura.role=api
  Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                   node.kubernetes.io/unreachable:NoExecute for 300s
  Events:
    Type    Reason          Age                      From                                                Message
    ----    ------          ----                     ----                                                -------
    Normal  SandboxChanged  113s (x1707 over 7h27m)  kubelet, ip-10-10-12-97.us-east-2.compute.internal  Pod sandbox changed, it will be killed and re-created.
  </code></pre>
  
  <p>Deployment yaml (from helm):</p>
  
  <pre><code>apiVersion: v1
  kind: Service
  metadata:
    name: phoenix
    labels:
      app.kubernetes.io/name: phoenix
      app.kubernetes.io/instance: {{ .Release.Name }}
      app.kubernetes.io/managed-by: {{ .Release.Service }}
  spec:
    type: ClusterIP
    ports:
      - port: 8080
        targetPort: 443
        protocol: TCP
        name: https
    selector:
      app.kubernetes.io/name: phoenix
      app.kubernetes.io/instance: {{ .Release.Name }}
  
  
  ---
  apiVersion: apps/v1
  kind: Deployment
  metadata:
    name: phoenix
    labels:
      app.kubernetes.io/name: phoenix
      app.kubernetes.io/instance: {{ .Release.Name }}
      app.kubernetes.io/managed-by: {{ .Release.Service }}
  spec:
    replicas: 2
    selector:
      matchLabels:
        app.kubernetes.io/name: phoenix
        app.kubernetes.io/instance: {{ .Release.Name }}
    template:
      metadata:
        labels:
          app.kubernetes.io/name: phoenix
          app.kubernetes.io/instance: {{ .Release.Name }}
      spec:
        containers:
          - name: {{ .Chart.Name }}
            image: "{{ .Values.env.repository }}/{{ .Values.env.tag }}-phoenix:latest"
            imagePullPolicy: Always
            env:
              - name: TCM_SECTION
                value: {{ .Values.env.tag }}
            ports:
              - name: http
                containerPort: 8040
                protocol: TCP
            livenessProbe:
              httpGet:
                path: /tvp_api
                port: 80
            readinessProbe:
              httpGet:
                path: /tvp_api
                port: 80
        strategy:
          type: RollingUpdate
          maxUnavailable: 1
        nodeSelector:
          kaltura.role: api
          beta.kubernetes.io/os: windows
          beta.kubernetes.io/arch: amd64
  
  </code></pre>
  
  <p>Additionally to this pod I have an nginx pod running on linux nodes, that pod is load-balanced using aws-alb-ingress-controller.</p>