title: Agent Unreachable after restore VMware Horizon Connection Server
body: <p>After a power outage I got both of my View Connection Servers unbootable with BSOD and I could not recovery it and also I don't have backup of it.</p>
  <p>After all steps below I could not get things fixed, all VMs are &quot;Agent Unreachable&quot;</p>
  <p>Created a new VM for connection server (WITH THE SAME NAME AS THE OLD &quot;VIEWCS01&quot;)</p>
  <blockquote>
  <p>Before doing it correctly I connected the base disk, and not the last
  snapshot, to the new VM and broke up the whole thing with the error
  &quot;file system specific implementation of ioctl [file] failed&quot;, I solved
  this correcting the CID - <a href="https://kb.vmware.com/s/article/1007969" rel="nofollow noreferrer">https://kb.vmware.com/s/article/1007969</a></p>
  </blockquote>
  <ul>
  <li>Installed Windows (Same version)</li>
  <li>As in <a href="https://kb.vmware.com/s/article/76770" rel="nofollow noreferrer">https://kb.vmware.com/s/article/76770</a>:
  <ul>
  <li>Installed Connection Server (Same Version)</li>
  <li>Restored LDF backup</li>
  <li>Removed all View Connection Servers and all Security Servers <code>(vdmadmin -S -s -r viewcs01, viewss01 and viewcs02)</code></li>
  <li>Uninstalled the connection server</li>
  <li>Reinstalled the connection server reusing the AD LDS</li>
  </ul>
  </li>
  </ul>
  <blockquote>
  <p>I did removed the &quot;viewcs01&quot; because with my previous tests I was not removing it, I think because of &gt;this, after the recovery steps done no console was opening, In previous tests I also not using the old &gt;machine name, instead of it I was using &quot;viewcs03&quot;.</p>
  </blockquote>
  <p>Ok, Console opened, I changed the vCenter credentials (Just putting password was not working with error - <a href="https://kb.vmware.com/s/article/60152" rel="nofollow noreferrer">https://kb.vmware.com/s/article/60152</a> - Log below:</p>
  <pre><code>2020-12-31T18:23:48.108-02:00 ERROR (18F8-0D0C) &lt;MessageFrameWorkDispatch&gt; [ws_java_bridgeDLL] BCryptDecrypt FAILED, status={Data Error}
  An error in reading or writing data occurred. (0xC000003E)
  2020-12-31T18:23:48.109-02:00 ERROR (18F8-0C54) &lt;VCHealthUpdate&gt; [SecurityManagerUtil] decryptAsText: com.vmware.vdi.crypto.SecurityManagerException: decrypt: Cannot decrypt: Cipher scheme decryption failed.
  2020-12-31T18:23:48.109-02:00 DEBUG (18F8-0C54) &lt;VCHealthUpdate&gt; [ServiceConnection25] Connecting instance VCHealth Test instance at URL https://vcenterd.DOMAIN.net:443/sdk
  </code></pre>
  <p>Corrected Composer credentials, and added license.</p>
  <p>All machines are &quot;Agent Unreachable&quot; - Connection Server Log below:</p>
  <pre><code>2020-12-31T18:23:49.160-02:00 DEBUG (18F8-1A6C) &lt;DesktopControlJMS&gt; [DesktopTracker] CHANGEKEY message from agent/bda3fbe6-029c-41f8-b9f8-017af574f56b accepted as key and thumbprints match machine record
  2020-12-31T18:23:49.162-02:00 DEBUG (18F8-1A6C) &lt;DesktopControlJMS&gt; [DesktopTracker] found broker thumbprints: 0f:9e:80:5d:f6:33:c7:1b:a2:d5:8c:9a:9f:12:45:16:0f:6f:c0:2b:46:8d:d0:33:62:87:53:a9:48:8d:57:8c#SHA_256;51:c5:d0:44:02:7f:ca:6d:5a:ad:5b:f6:8d:f5:11:23:e8:aa:e1:91:d0:5c:ff:71:3b:fb:e2:4b:f4:12:5e:d5#SHA_256
  2020-12-31T18:23:49.162-02:00 WARN  (18F8-1A6C) &lt;DesktopControlJMS&gt; [JMSMessageSecurity] Failed to sign message: Cannot sign message
  2020-12-31T18:23:49.162-02:00 DEBUG (18F8-1A6C) &lt;DesktopControlJMS&gt; [DesktopTracker] CHANGEKEY message from agent/bda3fbe6-029c-41f8-b9f8-017af574f56b result: true (success)
  </code></pre>
  <p>Excerpt from VM agent log:</p>
  <pre><code>2020-12-31T19:53:44.322-03:00 DEBUG (1EDC-0FA8) &lt;Thread-4&gt; [AgentJmsConfig] Using paired signing key
  2020-12-31T19:53:44.322-03:00 DEBUG (1EDC-0FA8) &lt;Thread-4&gt; [AgentMessageSecurityHandler] Configuring message security (ENHANCED).
  2020-12-31T19:53:44.369-03:00 DEBUG (1EDC-0FA8) &lt;Thread-4&gt; [BrokerUpdateUtility] Published CHANGEKEY request
  2020-12-31T19:53:59.386-03:00 DEBUG (1EDC-0FA8) &lt;Thread-4&gt; [BrokerUpdateUtility] Timeout waiting for success response
  2020-12-31T19:59:33.944-03:00 DEBUG (1430-2558) &lt;Thread-4&gt; [JmsManager] Using connection broker viewcs01.DOMAIN.net
  2020-12-31T19:59:33.944-03:00 DEBUG (1430-2494) &lt;MessageFrameWorkDispatch&gt; [MessageFrameWork] KeyVault service got operation=getEndEntityCertificates, ok=1, msecs=0
  2020-12-31T19:59:33.944-03:00 DEBUG (1430-2494) &lt;MessageFrameWorkDispatch&gt; [MessageFrameWork] KeyVault service got operation=getEndEntityCertificates, ok=1, msecs=0
  2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) &lt;Thread-4&gt; [JmsManager] username for swiftmq connection is: agent/90916ab8-704c-4fe3-a605-c4a7745b246e
  2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) &lt;Thread-4&gt; [AgentJmsConfig] Skipping pair operation: already paired
  2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) &lt;Thread-4&gt; [AgentMessageSecurityHandler] Configuring message security (ENHANCED).
  2020-12-31T19:59:33.975-03:00 DEBUG (1430-2558) &lt;Thread-4&gt; [JmsManager] Re-connecting using secure port 4002
  2020-12-31T19:59:34.381-03:00 DEBUG (1430-2780) &lt;SwiftMQ-ConnectorPool-2&gt; [AgentSSLSocketFactory] Received cert with subject cn=router/viewcs01
  2020-12-31T19:59:34.381-03:00 WARN  (1430-2780) &lt;SwiftMQ-ConnectorPool-2&gt; [AgentSSLSocketFactory] Certificate thumbprint verification failed, no matching thumbprint. Presented identity: router/viewcs01
  2020-12-31T19:59:34.381-03:00 DEBUG (1430-2558) &lt;Thread-4&gt; [JmsManager] Unable to connect to JMS server viewcs01.DOMAIN.net com.vmware.vdi.logger.Logger.debug(Logger.java:44)
  javax.net.ssl.SSLHandshakeException: java.security.cert.CertificateException: Unexpected certificate: router/viewcs01
  2020-12-31T19:59:34.381-03:00 WARN (1430-2558) &lt;Thread-4&gt; [JmsManager] Unable to connect to any listed host. The agent will continue to retry: [viewcs02.DOMAIN.net, viewcs01.DOMAIN.net]
  </code></pre>
  <p>Reinstalled the agent and also tried the command below, as mentioned in <a href="https://kb.vmware.com/s/article/2038679" rel="nofollow noreferrer">https://kb.vmware.com/s/article/2038679</a>, nothing has worked at all.</p>
  <p><code>vdmadmin -A -d desktop-pool-name -m name-of-machine-in-pool -resetkey</code></p>
  <p>Update to question:
  Some piece (That I don't have now) of the log lead me to <a href="https://kb.vmware.com/s/article/76302" rel="nofollow noreferrer">This KB</a>, so I have uninstalled the Connection Server, remove all certificates and reinstalled it again, nothing changed.</p>
  <p>After reading the following links <a href="https://docs.vmware.com/en/VMware-Horizon/2006/horizon-security/GUID-4BBC13C6-900A-44B9-B4A3-C90EE3F3BCA9.html" rel="nofollow noreferrer">[1]</a>, <a href="https://www.reddit.com/r/vmware/comments/4wp9i6/horizon_7_cant_change_message_security_mode/" rel="nofollow noreferrer">[2]</a>, <a href="https://kb.vmware.com/s/article/2110760" rel="nofollow noreferrer">[3]</a>:
  I've changed the security mode to Mixed, nothing changed.</p>
  <p>But after change it to mixed and after reinstalling the Agent (I've reinstalled it before change to mixed and it didn't worked) from the VM have turned it to Available, I still not able to access machine with tunnel errors (Changed tunnel configurations also).</p>
  <p>Updated the Connection Server to 7.13, stopped to open console.</p>
  <p>Started the whole process from zero, now machine name is viewcs04, not worked also.</p>
  <p>For any who encounter the same problem, I decided to create a new Connection Server and I will create manual pools, so people can work and I will migrate everyone to new linked-clone pools.</p>
  <p>Just to mention, I cannot just create new pools, all pools are dedicated and have manually installed applications, printers etc.</p>