title: Add new control plane node got failed k8s 1.21.0 [solved]
body: <p>I want to add a new control plane node into the cluster.</p>
  <p>So, I run in an existing control plane server:
  <code>kubeadm token create --print-join-command</code></p>
  <p>I run this command in new control plane node:</p>
  <pre><code>kubeadm join 10.0.0.151:8443 --token m3g8pf.gdop9wz08yhd7a8a --discovery-token-ca-cert-hash sha256:634db22bc69b47b8f2b9f733d2f5e95cf8e56b349e68ac611a56d9da0cf481b8 --control-plane --apiserver-advertise-address 10.0.0.10 --apiserver-bind-port 6443 --certificate-key 33cf0a1d30da4c714755b4de4f659d6d5a02e7a0bd522af2ebc2741487e53166
  </code></pre>
  <ol start="3">
  <li>I got this message:</li>
  </ol>
  <pre><code>[download-certs] Downloading the certificates in Secret &quot;kubeadm-certs&quot; in the &quot;kube-system&quot; Namespace 
  error execution phase control-plane-prepare/download-certs: error downloading certs: the Secret does not include the required certificate or key - name: external-e
  tcd.crt, path: /etc/kubernetes/pki/apiserver-etcd-client.crt
  </code></pre>
  <ol start="4">
  <li>I run in an existing production control plane node:</li>
  </ol>
  <pre><code>kubeadm init phase upload-certs --upload-certs
  [upload-certs] Storing the certificates in Secret &quot;kubeadm-certs&quot; in the &quot;kube-system&quot; Namespace
  [upload-certs] Using certificate key:
  0a3f5486c3b9303a4ace70ad0a9870c2605d67eebcd500d68a5e776bbd628a3b
  </code></pre>
  <ol start="5">
  <li>Re-run this command in the new control plane node:</li>
  </ol>
  <pre><code>kubeadm join 10.0.0.151:8443 --token m3g8pf.gdop9wz08yhd7a8a --discovery-token-ca-cert-hash sha256:634db22bc69b47b8f2b9f733d2f5e95cf8e56b349e68ac611a56d9da0cf481b8 --control-plane --apiserver-advertise-address 10.0.0.10 --apiserver-bind-port 6443 --certificate-key 0a3f5486c3b9303a4ace70ad0a9870c2605d67eebcd500d68a5e776bbd628a3b
  </code></pre>
  <p>I got the same message:</p>
  <pre><code>[download-certs] Downloading the certificates in Secret &quot;kubeadm-certs&quot; in the &quot;kube-system&quot; Namespace
  error execution phase control-plane-prepare/download-certs: error downloading certs: the Secret does not include the required certificate or key - name: external-etcd.crt, path: /etc/kubernetes/pki/apiserver-etcd-client.crt
  To see the stack trace of this error execute with --v=5 or higher
  </code></pre>
  <p>What's I am wrong?</p>
  <p>I have all certs in the new node installed before doing this op:</p>
  <pre><code># ls /etc/kubernetes/pki/
  apiserver.crt              apiserver.key                 ca.crt  front-proxy-ca.crt      front-proxy-client.key
  apiserver-etcd-client.crt  apiserver-kubelet-client.crt  ca.key  front-proxy-ca.key      sa.key
  apiserver-etcd-client.key  apiserver-kubelet-client.key  etcd    front-proxy-client.crt  sa.pub
  </code></pre>
  <p>I didn't see how to specify etcd certs files:</p>
  <pre><code>Usage:
    kubeadm init phase upload-certs [flags]
  
  Flags:
        --certificate-key string       Key used to encrypt the control-plane certificates in the kubeadm-certs Secret.
        --config string                Path to a kubeadm configuration file.
    -h, --help                         help for upload-certs
        --kubeconfig string            The kubeconfig file to use when talking to the cluster. If the flag is not set, a set of standard locations can be searched for an existing kubeconfig file. (default &quot;/etc/kubernetes/admin.conf&quot;)
        --skip-certificate-key-print   Don't print the key used to encrypt the control-plane certificates.
        --upload-certs                 Upload control-plane certificates to the kubeadm-certs Secret.
  
  Global Flags:
        --add-dir-header           If true, adds the file directory to the header of the log messages
        --log-file string          If non-empty, use this log file
        --log-file-max-size uint   Defines the maximum size a log file can grow to. Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)
        --one-output               If true, only write logs to their native severity level (vs also writing to each lower severity level)
        --rootfs string            [EXPERIMENTAL] The path to the 'real' host root filesystem.
        --skip-headers             If true, avoid header prefixes in the log messages
        --skip-log-headers         If true, avoid headers when opening log files
    -v, --v Level                  number for the log level verbosity
  
  </code></pre>