<p dir="auto"><strong>Kubernetes version</strong> :</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Client Version: version.Info{Major:&quot;1&quot;, Minor:&quot;3&quot;, GitVersion:&quot;v1.3.4&quot;, GitCommit:&quot;dd6b458ef8dbf24aff55795baa68f83383c9b3a9&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-08-01T16:45:16Z&quot;, GoVersion:&quot;go1.6.2&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}
Server Version: version.Info{Major:&quot;1&quot;, Minor:&quot;4+&quot;, GitVersion:&quot;v1.4.0-beta.8+coreos.0&quot;, GitCommit:&quot;9c19ded313d3b3b86eadf179aed553854138abd7&quot;, GitTreeState:&quot;clean&quot;, BuildDate:&quot;2016-09-19T18:58:14Z&quot;, GoVersion:&quot;go1.6.3&quot;, Compiler:&quot;gc&quot;, Platform:&quot;linux/amd64&quot;}"><pre class="notranslate"><code class="notranslate">Client Version: version.Info{Major:"1", Minor:"3", GitVersion:"v1.3.4", GitCommit:"dd6b458ef8dbf24aff55795baa68f83383c9b3a9", GitTreeState:"clean", BuildDate:"2016-08-01T16:45:16Z", GoVersion:"go1.6.2", Compiler:"gc", Platform:"linux/amd64"}
Server Version: version.Info{Major:"1", Minor:"4+", GitVersion:"v1.4.0-beta.8+coreos.0", GitCommit:"9c19ded313d3b3b86eadf179aed553854138abd7", GitTreeState:"clean", BuildDate:"2016-09-19T18:58:14Z", GoVersion:"go1.6.3", Compiler:"gc", Platform:"linux/amd64"}
</code></pre></div>
<p dir="auto">(I'm using v1.4.0-beta.8+coreos.0 version since this fixes "dmidecode" binary dependency, which is not available neither in hyperkube, nor in Core OS <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="160079002" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/27331" data-hovercard-type="pull_request" data-hovercard-url="/kubernetes/kubernetes/pull/27331/hovercard" href="https://github.com/kubernetes/kubernetes/pull/27331">#27331</a>)</p>
<p dir="auto"><strong>Environment</strong>:</p>
<ul dir="auto">
<li><strong>Cloud provider or hardware configuration</strong>:</li>
</ul>
<p dir="auto">VMWare virtual machine, deployed on as ESXi 6.0.0 Update 2 (Build 4192238) host.</p>
<p dir="auto">Output from govc datacenter.info</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Name:                ha-datacenter
  Hosts:             1
  Clusters:          0
  Virtual Machines:  1
  Networks:          1
  Datastores:        4"><pre class="notranslate"><code class="notranslate">Name:                ha-datacenter
  Hosts:             1
  Clusters:          0
  Virtual Machines:  1
  Networks:          1
  Datastores:        4
</code></pre></div>
<p dir="auto">Output from govc vm.info -vm.ip=192.168.0.144</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Name:           ci01
  Path:         /ha-datacenter/vm/ci01
  UUID:         564d8885-7d5b-3c5c-3525-d382ca47ab1c
  Guest name:   Other 2.6.x Linux (64-bit)
  Memory:       1024MB
  CPU:          2 vCPU(s)
  Power state:  poweredOn
  Boot time:    2016-09-22 05:59:31.517386 +0000 UTC
  IP address:   10.2.24.1
  Host:         hostname.mydomain.com"><pre class="notranslate"><code class="notranslate">Name:           ci01
  Path:         /ha-datacenter/vm/ci01
  UUID:         564d8885-7d5b-3c5c-3525-d382ca47ab1c
  Guest name:   Other 2.6.x Linux (64-bit)
  Memory:       1024MB
  CPU:          2 vCPU(s)
  Power state:  poweredOn
  Boot time:    2016-09-22 05:59:31.517386 +0000 UTC
  IP address:   10.2.24.1
  Host:         hostname.mydomain.com
</code></pre></div>
<p dir="auto">where 192.168.0.144 is the public IP of the virtual machine</p>
<ul dir="auto">
<li><strong>OS</strong> :</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="NAME=CoreOS
ID=coreos
VERSION=1122.2.0
VERSION_ID=1122.2.0
BUILD_ID=2016-09-06-1449
PRETTY_NAME=&quot;CoreOS 1122.2.0 (MoreOS)&quot;
ANSI_COLOR=&quot;1;32&quot;
HOME_URL=&quot;https://coreos.com/&quot;
BUG_REPORT_URL=&quot;https://github.com/coreos/bugs/issues&quot;"><pre class="notranslate"><code class="notranslate">NAME=CoreOS
ID=coreos
VERSION=1122.2.0
VERSION_ID=1122.2.0
BUILD_ID=2016-09-06-1449
PRETTY_NAME="CoreOS 1122.2.0 (MoreOS)"
ANSI_COLOR="1;32"
HOME_URL="https://coreos.com/"
BUG_REPORT_URL="https://github.com/coreos/bugs/issues"
</code></pre></div>
<p dir="auto">I'm using <a href="https://stable.release.core-os.net/amd64-usr/current/coreos_production_vmware_ova.ova" rel="nofollow">coreos_production_vmware_ova.ova</a>  available in CoreOs stable channel.</p>
<ul dir="auto">
<li><strong>Kernel</strong> :</li>
</ul>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Linux ci01 4.7.0-coreos #1 SMP Tue Sep 6 14:39:20 UTC 2016 x86_64 Intel(R) Xeon(R) CPU L5640 @ 2.27GHz GenuineIntel GNU/Linux"><pre class="notranslate"><code class="notranslate">Linux ci01 4.7.0-coreos #1 SMP Tue Sep 6 14:39:20 UTC 2016 x86_64 Intel(R) Xeon(R) CPU L5640 @ 2.27GHz GenuineIntel GNU/Linux
</code></pre></div>
<ul dir="auto">
<li><strong>Install tools</strong>:</li>
</ul>
<p dir="auto">VMWare tools version 2147483647</p>
<ul dir="auto">
<li><strong>Others</strong>:</li>
</ul>
<p dir="auto">kubelet service:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[Service]
EnvironmentFile=/etc/environment
Environment=KUBELET_VERSION=v1.4.0-beta.8_coreos.0
TimeoutStartSec=infinity
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
ExecStartPre=/bin/mkdir -p /srv/kubernetes/manifests
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
ExecStartPre=/bin/rkt fetch quay.io/coreos/hyperkube:v1.4.0-beta.8_coreos.0 --trust-keys-from-https
ExecStart=/usr/lib/coreos/kubelet-wrapper \
--api-servers=https://192.168.0.144:443 \
--kubeconfig=/etc/kubernetes/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \
--exit-on-lock-contention=true \
--allow-privileged \
--config=/etc/kubernetes/manifests \
--hostname-override=192.168.0.144 \
--node-labels=master=true \
--minimum-container-ttl-duration=3m0s \
--cluster_dns=10.3.0.10 \
--cluster_domain=cluster.local \
--cloud-config=/etc/kubernetes/vsphere.conf \
--cloud-provider=vsphere

Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target"><pre class="notranslate"><code class="notranslate">[Service]
EnvironmentFile=/etc/environment
Environment=KUBELET_VERSION=v1.4.0-beta.8_coreos.0
TimeoutStartSec=infinity
ExecStartPre=/bin/mkdir -p /etc/kubernetes/manifests
ExecStartPre=/bin/mkdir -p /srv/kubernetes/manifests
ExecStartPre=/bin/mkdir -p /etc/kubernetes/checkpoint-secrets
ExecStartPre=/bin/rkt fetch quay.io/coreos/hyperkube:v1.4.0-beta.8_coreos.0 --trust-keys-from-https
ExecStart=/usr/lib/coreos/kubelet-wrapper \
--api-servers=https://192.168.0.144:443 \
--kubeconfig=/etc/kubernetes/kubeconfig \
--lock-file=/var/run/lock/kubelet.lock \
--exit-on-lock-contention=true \
--allow-privileged \
--config=/etc/kubernetes/manifests \
--hostname-override=192.168.0.144 \
--node-labels=master=true \
--minimum-container-ttl-duration=3m0s \
--cluster_dns=10.3.0.10 \
--cluster_domain=cluster.local \
--cloud-config=/etc/kubernetes/vsphere.conf \
--cloud-provider=vsphere

Restart=always
RestartSec=5

[Install]
WantedBy=multi-user.target
</code></pre></div>
<p dir="auto">vsphere.conf (cloud-config):</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="[Global]
server=ESXI_host_IP
port=443
user=myuser
password=mypassword
insecure-flag=true
datacenter=ha-datacenter"><pre class="notranslate"><code class="notranslate">[Global]
server=ESXI_host_IP
port=443
user=myuser
password=mypassword
insecure-flag=true
datacenter=ha-datacenter
</code></pre></div>
<p dir="auto"><strong>What happened</strong>:<br>
Kubelet fails to start with the following errors:</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content=" systemd[1]: Starting kubelet.service...
 rkt[1866]: image: using image from local store for image name quay.io/coreos/hyperkube:v1.4.0-beta.8_coreos.0
 rkt[1866]: sha512-406472bdf336d7a2a353354ae0597540
 systemd[1]: Started kubelet.service.
 kubelet-wrapper[1883]: image: using image from file /usr/lib64/rkt/stage1-images/stage1-fly.aci
 kubelet-wrapper[1883]: image: using image from local store for image name quay.io/coreos/hyperkube:v1.4.0-beta.8_coreos.0
 kubelet-wrapper[1883]: Flag --api-servers has been deprecated, Use --kubeconfig instead. Will be removed in a future version.
 kubelet-wrapper[1883]: Flag --config has been deprecated, Use --pod-manifest-path instead. Will be removed in a future version.
 kubelet-wrapper[1883]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
 kubelet-wrapper[1883]: I0921 13:41:24.809785    1883 server.go:309] acquiring lock on &quot;/var/run/lock/kubelet.lock&quot;
 kubelet-wrapper[1883]: I0921 13:41:24.810116    1883 server.go:314] watching for inotify events for: /var/run/lock/kubelet.lock
 kubelet-wrapper[1883]: panic: reflect.Set: value of type mo.ComputeResource is not assignable to type mo.ClusterComputeResource
 kubelet-wrapper[1883]: goroutine 1 [running]:
 kubelet-wrapper[1883]: panic(0x388bb80, 0xc82034aec0)
 kubelet-wrapper[1883]:         /usr/local/go/src/runtime/panic.go:481 +0x3e6
 kubelet-wrapper[1883]: reflect.Value.assignTo(0x4a042a0, 0xc82050e1e0, 0x99, 0x4d90950, 0xb, 0x49ce7e0, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /usr/local/go/src/reflect/value.go:2164 +0x3be
 kubelet-wrapper[1883]: reflect.Value.Set(0x49ce7e0, 0xc8202a4700, 0x199, 0x4a042a0, 0xc82050e1e0, 0x99)
 kubelet-wrapper[1883]:         /usr/local/go/src/reflect/value.go:1334 +0x95
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo.LoadRetrievePropertiesResponse(0xc820aba760, 0x4690fe0, 0xc8202a4700, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo/retrieve.go:128 +0xe21
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).Retrieve(0xc820962b60, 0x7f3ecae79290, 0xc820256e80, 0xc820581260, 0x1, x1, 0x1, 0x4690fe0, ...)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:167 +0x52f
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).RetrieveOne(0xc820962b60, 0x7f3ecae79290, 0xc820256e80, 0xc82045d180, xe, 0xc82045dd90, 0x1, 0x1, ...)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:173 +0x10e
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object.Common.Properties(0x0, 0x0, 0xc82022a500, 0xc8202cd890, 0xb, 0xc8202cd8f0, 0xe, 20256e80, 0xc82045d180, ...)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object/common.go:97 +0x19f
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.readInstance(0xc8201c2820, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:219 +0xe8f
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.newVSphere(0xc820211250, 0x5, 0xc820211490, 0x9, 0xc820210ce0, 0xd, 0xc820210f18, 0x3, ..)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:233 +0x7e
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.init.1.func1(0x7f3ecae69488, 0xc82001c210, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:149 +0xdf
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider.GetCloudProvider(0x7ffc4d24cf63, 0x7, 0x7f3ecae69488, 0xc82001c210, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:62 +0x112
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider.InitCloudProvider(0x7ffc4d24cf63, 0x7, 0x7ffc4d24cf35, 0x1c, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:84 +0x3e2
 kubelet-wrapper[1883]: k8s.io/kubernetes/cmd/kubelet/app.run(0xc8204fc000, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:348 +0x17ca
 kubelet-wrapper[1883]: k8s.io/kubernetes/cmd/kubelet/app.Run(0xc8204fc000, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:267 +0x4a
 kubelet-wrapper[1883]: main.NewKubelet.func1(0xc8201f7400, 0xc82033b6c0, 0x0, 0xd, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/kubelet.go:37 +0x35
 kubelet-wrapper[1883]: main.(*HyperKube).Run(0xc82042d560, 0xc82000a0f0, 0xd, 0xd, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/hyperkube.go:181 +0x670
 kubelet-wrapper[1883]: main.(*HyperKube).RunToExit(0xc82042d560, 0xc82000a0e0, 0xe, 0xe)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/hyperkube.go:191 +0x5f
 kubelet-wrapper[1883]: main.main()
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/main.go:48 +0x79e
 systemd[1]: kubelet.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
 systemd[1]: kubelet.service: Unit entered failed state.
 systemd[1]: kubelet.service: Failed with result 'exit-code'."><pre class="notranslate"><code class="notranslate"> systemd[1]: Starting kubelet.service...
 rkt[1866]: image: using image from local store for image name quay.io/coreos/hyperkube:v1.4.0-beta.8_coreos.0
 rkt[1866]: sha512-406472bdf336d7a2a353354ae0597540
 systemd[1]: Started kubelet.service.
 kubelet-wrapper[1883]: image: using image from file /usr/lib64/rkt/stage1-images/stage1-fly.aci
 kubelet-wrapper[1883]: image: using image from local store for image name quay.io/coreos/hyperkube:v1.4.0-beta.8_coreos.0
 kubelet-wrapper[1883]: Flag --api-servers has been deprecated, Use --kubeconfig instead. Will be removed in a future version.
 kubelet-wrapper[1883]: Flag --config has been deprecated, Use --pod-manifest-path instead. Will be removed in a future version.
 kubelet-wrapper[1883]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
 kubelet-wrapper[1883]: I0921 13:41:24.809785    1883 server.go:309] acquiring lock on "/var/run/lock/kubelet.lock"
 kubelet-wrapper[1883]: I0921 13:41:24.810116    1883 server.go:314] watching for inotify events for: /var/run/lock/kubelet.lock
 kubelet-wrapper[1883]: panic: reflect.Set: value of type mo.ComputeResource is not assignable to type mo.ClusterComputeResource
 kubelet-wrapper[1883]: goroutine 1 [running]:
 kubelet-wrapper[1883]: panic(0x388bb80, 0xc82034aec0)
 kubelet-wrapper[1883]:         /usr/local/go/src/runtime/panic.go:481 +0x3e6
 kubelet-wrapper[1883]: reflect.Value.assignTo(0x4a042a0, 0xc82050e1e0, 0x99, 0x4d90950, 0xb, 0x49ce7e0, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /usr/local/go/src/reflect/value.go:2164 +0x3be
 kubelet-wrapper[1883]: reflect.Value.Set(0x49ce7e0, 0xc8202a4700, 0x199, 0x4a042a0, 0xc82050e1e0, 0x99)
 kubelet-wrapper[1883]:         /usr/local/go/src/reflect/value.go:1334 +0x95
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo.LoadRetrievePropertiesResponse(0xc820aba760, 0x4690fe0, 0xc8202a4700, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo/retrieve.go:128 +0xe21
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).Retrieve(0xc820962b60, 0x7f3ecae79290, 0xc820256e80, 0xc820581260, 0x1, x1, 0x1, 0x4690fe0, ...)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:167 +0x52f
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).RetrieveOne(0xc820962b60, 0x7f3ecae79290, 0xc820256e80, 0xc82045d180, xe, 0xc82045dd90, 0x1, 0x1, ...)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:173 +0x10e
 kubelet-wrapper[1883]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object.Common.Properties(0x0, 0x0, 0xc82022a500, 0xc8202cd890, 0xb, 0xc8202cd8f0, 0xe, 20256e80, 0xc82045d180, ...)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object/common.go:97 +0x19f
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.readInstance(0xc8201c2820, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:219 +0xe8f
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.newVSphere(0xc820211250, 0x5, 0xc820211490, 0x9, 0xc820210ce0, 0xd, 0xc820210f18, 0x3, ..)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:233 +0x7e
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.init.1.func1(0x7f3ecae69488, 0xc82001c210, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:149 +0xdf
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider.GetCloudProvider(0x7ffc4d24cf63, 0x7, 0x7f3ecae69488, 0xc82001c210, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:62 +0x112
 kubelet-wrapper[1883]: k8s.io/kubernetes/pkg/cloudprovider.InitCloudProvider(0x7ffc4d24cf63, 0x7, 0x7ffc4d24cf35, 0x1c, 0x0, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:84 +0x3e2
 kubelet-wrapper[1883]: k8s.io/kubernetes/cmd/kubelet/app.run(0xc8204fc000, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:348 +0x17ca
 kubelet-wrapper[1883]: k8s.io/kubernetes/cmd/kubelet/app.Run(0xc8204fc000, 0x0, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:267 +0x4a
 kubelet-wrapper[1883]: main.NewKubelet.func1(0xc8201f7400, 0xc82033b6c0, 0x0, 0xd, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/kubelet.go:37 +0x35
 kubelet-wrapper[1883]: main.(*HyperKube).Run(0xc82042d560, 0xc82000a0f0, 0xd, 0xd, 0x0, 0x0)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/hyperkube.go:181 +0x670
 kubelet-wrapper[1883]: main.(*HyperKube).RunToExit(0xc82042d560, 0xc82000a0e0, 0xe, 0xe)
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/hyperkube.go:191 +0x5f
 kubelet-wrapper[1883]: main.main()
 kubelet-wrapper[1883]:         /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/hyperkube/main.go:48 +0x79e
 systemd[1]: kubelet.service: Main process exited, code=exited, status=2/INVALIDARGUMENT
 systemd[1]: kubelet.service: Unit entered failed state.
 systemd[1]: kubelet.service: Failed with result 'exit-code'.
</code></pre></div>
<p dir="auto"><strong>What you expected to happen</strong>:<br>
It works as expected if I comment out lines related to cloud-config and cloud-provider from kubelet service.</p>
<p dir="auto">I wolud like to use those flags in orther to be able to use vsphere volumes in kubernetes.</p>
<p dir="auto"><strong>How to reproduce it</strong>:<br>
Start kubelet service, it always fails trying to read vm instances on vsphere.go provider plugin</p>
<p dir="auto"><strong>Anything else do we need to know</strong>:<br>
It is a sinlge node, self hosted, kubernetes cluster. I generate kubernetes manifests using <a href="https://github.com/kubernetes-incubator/bootkube">bootkube</a> binary with its render method, then modify manifest to use v1.4.0-beta.8_coreos.0 version, and finally launch cluster using bootkube start.</p>
<p dir="auto">As I said before, it works when you don't put cloud-provider and cloud-config in the kubelet servide.</p>
<p dir="auto">Recent changes in master <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="172010677" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/30935" data-hovercard-type="pull_request" data-hovercard-url="/kubernetes/kubernetes/pull/30935/hovercard" href="https://github.com/kubernetes/kubernetes/pull/30935">#30935</a> that were pulled into the 1.4 release branch lead to vSphere cloud provider to panic when only a single ESX node is present under a vCenter.</p>
<p dir="auto">I am in the process of testing and creating a PR for this.</p>
<div class="snippet-clipboard-content notranslate position-relative overflow-auto" data-snippet-clipboard-copy-content="Oct 14 19:11:46 kube kubelet[22061]: panic: reflect.Set: value of type mo.ComputeResource is not assignable to type mo.ClusterComputeResource
Oct 14 19:11:46 kube kubelet[22061]: goroutine 1 [running]:
Oct 14 19:11:46 kube kubelet[22061]: panic(0x287dbe0, 0xc82043b790)
Oct 14 19:11:46 kube kubelet[22061]: /usr/local/go/src/runtime/panic.go:481 +0x3e6
Oct 14 19:11:46 kube kubelet[22061]: reflect.Value.assignTo(0x359f380, 0xc8200f8780, 0x99, 0x382f5a0, 0xb, 0x357c7e0, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /usr/local/go/src/reflect/value.go:2164 +0x3be
Oct 14 19:11:46 kube kubelet[22061]: reflect.Value.Set(0x357c7e0, 0xc82058c000, 0x199, 0x359f380, 0xc8200f8780, 0x99)
Oct 14 19:11:46 kube kubelet[22061]: /usr/local/go/src/reflect/value.go:1334 +0x95
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo.LoadRetrievePropertiesResponse(0xc820241c20, 0x32db640, 0xc82058c000, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo/retrieve.go:128 +0xe21
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).Retrieve(0xc8205a3060, 0x7f1cb57faf18, 0xc82038dc00, 0xc8202418a0, 0x1, 0x1, 0xc82043a170, 0x1, 0x1, 0x32db640, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:167 +0x52f
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).RetrieveOne(0xc8205a3060, 0x7f1cb57faf18, 0xc82038dc00, 0xc820451750, 0xf, 0xc820451770, 0xb, 0xc82043a170, 0x1, 0x1, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:173 +0x10e
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object.Common.Properties(0x0, 0x0, 0xc820359900, 0xc82032bdd0, 0xb, 0xc82032be00, 0xb, 0x7f1cb57faf18, 0xc82038dc00, 0xc820451750, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object/common.go:97 +0x19f
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.readInstance(0xc8203cc5a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:256 +0xe8f
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.newVSphere(0xc82042dbe0, 0x1b, 0xc82048a8b0, 0x8, 0xc82048ac70, 0xd, 0xc82048af90, 0x3, 0x1, 0xc82048b550, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:270 +0x7e
Oct 14 19:11:46 kube systemd[1]: kubelet.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 14 19:11:46 kube systemd[1]: Unit kubelet.service entered failed state.
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.init.1.func1(0x7f1cb57d62c8, 0xc82001e6a8, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:186 +0xdf
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider.GetCloudProvider(0x7ffcb7cebd22, 0x7, 0x7f1cb57d62c8, 0xc82001e6a8, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:85 +0x112
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider.InitCloudProvider(0x7ffcb7cebd22, 0x7, 0x7ffcb7cebd39, 0x19, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:107 +0x3e2
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/cmd/kubelet/app.run(0xc820072b00, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:349 +0x18c2
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/cmd/kubelet/app.Run(0xc820072b00, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:268 +0x4a
Oct 14 19:11:46 kube kubelet[22061]: main.main()
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/kubelet.go:48 +0x94"><pre class="notranslate"><code class="notranslate">Oct 14 19:11:46 kube kubelet[22061]: panic: reflect.Set: value of type mo.ComputeResource is not assignable to type mo.ClusterComputeResource
Oct 14 19:11:46 kube kubelet[22061]: goroutine 1 [running]:
Oct 14 19:11:46 kube kubelet[22061]: panic(0x287dbe0, 0xc82043b790)
Oct 14 19:11:46 kube kubelet[22061]: /usr/local/go/src/runtime/panic.go:481 +0x3e6
Oct 14 19:11:46 kube kubelet[22061]: reflect.Value.assignTo(0x359f380, 0xc8200f8780, 0x99, 0x382f5a0, 0xb, 0x357c7e0, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /usr/local/go/src/reflect/value.go:2164 +0x3be
Oct 14 19:11:46 kube kubelet[22061]: reflect.Value.Set(0x357c7e0, 0xc82058c000, 0x199, 0x359f380, 0xc8200f8780, 0x99)
Oct 14 19:11:46 kube kubelet[22061]: /usr/local/go/src/reflect/value.go:1334 +0x95
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo.LoadRetrievePropertiesResponse(0xc820241c20, 0x32db640, 0xc82058c000, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/vim25/mo/retrieve.go:128 +0xe21
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).Retrieve(0xc8205a3060, 0x7f1cb57faf18, 0xc82038dc00, 0xc8202418a0, 0x1, 0x1, 0xc82043a170, 0x1, 0x1, 0x32db640, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:167 +0x52f
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property.(*Collector).RetrieveOne(0xc8205a3060, 0x7f1cb57faf18, 0xc82038dc00, 0xc820451750, 0xf, 0xc820451770, 0xb, 0xc82043a170, 0x1, 0x1, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/property/collector.go:173 +0x10e
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object.Common.Properties(0x0, 0x0, 0xc820359900, 0xc82032bdd0, 0xb, 0xc82032be00, 0xb, 0x7f1cb57faf18, 0xc82038dc00, 0xc820451750, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/vendor/github.com/vmware/govmomi/object/common.go:97 +0x19f
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.readInstance(0xc8203cc5a0, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:256 +0xe8f
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.newVSphere(0xc82042dbe0, 0x1b, 0xc82048a8b0, 0x8, 0xc82048ac70, 0xd, 0xc82048af90, 0x3, 0x1, 0xc82048b550, ...)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:270 +0x7e
Oct 14 19:11:46 kube systemd[1]: kubelet.service: main process exited, code=exited, status=2/INVALIDARGUMENT
Oct 14 19:11:46 kube systemd[1]: Unit kubelet.service entered failed state.
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere.init.1.func1(0x7f1cb57d62c8, 0xc82001e6a8, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/providers/vsphere/vsphere.go:186 +0xdf
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider.GetCloudProvider(0x7ffcb7cebd22, 0x7, 0x7f1cb57d62c8, 0xc82001e6a8, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:85 +0x112
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/pkg/cloudprovider.InitCloudProvider(0x7ffcb7cebd22, 0x7, 0x7ffcb7cebd39, 0x19, 0x0, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/pkg/cloudprovider/plugins.go:107 +0x3e2
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/cmd/kubelet/app.run(0xc820072b00, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:349 +0x18c2
Oct 14 19:11:46 kube kubelet[22061]: k8s.io/kubernetes/cmd/kubelet/app.Run(0xc820072b00, 0x0, 0x0, 0x0)
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/app/server.go:268 +0x4a
Oct 14 19:11:46 kube kubelet[22061]: main.main()
Oct 14 19:11:46 kube kubelet[22061]: /go/src/k8s.io/kubernetes/_output/dockerized/go/src/k8s.io/kubernetes/cmd/kubelet/kubelet.go:48 +0x94
</code></pre></div>