Problem
You’ve noticed that some desktops are in Agent Unreachable state in the VMware Horizon 7 Administrator console and clicking on the elipsis displays the following detail:
Status:Agent unreachable
Pairing state:Paired and secured
Configured by:vcs.domain.com
Attempted theft by:
Reviewing the debug logs on the virtual desktop in the directory C:\programdata\VMware\VDM\Logs:
… displays the following entries:
2018-12-17T16:48:34.881-04:00 INFO (0EAC-0D78) <Thread-4> [AgentJmsConfig] Attempting to securely pair agent for JMS communication
2018-12-17T16:48:34.881-04:00 DEBUG (0EAC-0D78) <Thread-4> [AgentJmsConfig] Using paired signing key
2018-12-17T16:48:34.881-04:00 DEBUG (0EAC-0D78) <Thread-4> [JmsManager] Unable to connect to JMS server conto-vcs.contoso.com com.vmware.vdi.logger.Logger.debug(Logger.java:44)
com.vmware.vdi.messagesecurity.MessageSecurityException: Paired key does not exist
at com.vmware.vdi.agent.messageserver.AgentJmsConfig.pairOverJms(SourceFile:318)
at com.vmware.vdi.agent.messageserver.JmsManager.connect(SourceFile:288)
at com.vmware.vdi.agent.messageserver.Main.Start(SourceFile:1265)
2018-12-17T16:48:44.975-04:00 ERROR (0EAC-09C4) <Main Thread> [wsnm_jms] JavaBridge reader thread init: timed out
2018-12-17T16:48:44.975-04:00 DEBUG (0EAC-0400) <1024> [wsnm_jms] AddressCache::ThreadEntry: Stopping
2018-12-17T16:48:45.177-04:00 DEBUG (0644-0958) <SharedMemReaderThread> [MessageFrameWork] SharedMem reader opt, reader 0x02EA21E8 for channel 0x00311280 detaching from thread 0x02E62168 handleCount 7
2018-12-17T16:48:45.177-04:00 DEBUG (0644-0958) <SharedMemReaderThread> [MessageFrameWork] SharedMem reader opt, peer is dead - reader 0x02EA21E8 in thread 0x02E62168
2018-12-17T16:48:45.177-04:00 DEBUG (0644-0958) <SharedMemReaderThread> [MessageFrameWork] MessageFrameWork Worker Shutdown OnChannelDelete, Name=EventLoggerService
2018-12-17T16:48:45.177-04:00 DEBUG (0644-0958) <SharedMemReaderThread> [MessageFrameWork] MessageFrameWork Worker Shutdown OnChannelDelete, Name=JavaView-3756
2018-12-17T16:48:45.177-04:00 DEBUG (0644-0958) <SharedMemReaderThread> [MessageFrameWork] MessageFrameWork Worker Shutdown OnChannelDelete, Name=JMSBridge
2018-12-17T16:48:45.177-04:00 DEBUG (0644-0958) <SharedMemReaderThread> [MessageFrameWork] MessageFrameWork Worker Shutdown OnChannelDelete, Name=TimingProfilerService
2018-12-17T16:48:45.177-04:00 DEBUG (0644-0958) <SharedMemReaderThread> [MessageFrameWork] CORE::AuthChannelInt::~AuthChannelInt(): Closed incoming SharedMemory channel from machine conto-PLANET002.contoso.com, user contoso\conto-PLANET002$, channel 00311280
2018-12-17T16:48:45.177-04:00 INFO (0644-08F4) <JavaBridge> [wsnm_jmsbridge] wsnm_jms died, restarting in a minute
Further review of the logs displays the following entries:
2018-12-18T11:08:00.715-04:00 DEBUG (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] coregate 'KeyVault' locking
2018-12-18T11:08:00.715-04:00 DEBUG (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] coregate 'KeyVault' locked
2018-12-18T11:08:00.715-04:00 INFO (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVaultCAPI upgrade rekey start for wsnm_jms
2018-12-18T11:08:00.716-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVaultCAPI encipher key 'local storage cipher key#1' MISSING, error:
2018-12-18T11:08:00.716-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'DataRef-9IQloB7D' from CAPI
2018-12-18T11:08:00.718-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 0x80090006 (Invalid Signature.)
2018-12-18T11:08:00.718-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.719-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'DataRef-AmH0fBxY' from CAPI
2018-12-18T11:08:00.719-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVaultCAPI encipher key 'local storage cipher key#1' MISSING, error:
2018-12-18T11:08:00.720-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'DataRef-hfBerq31' from CAPI
2018-12-18T11:08:00.720-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 0x80090006 (Invalid Signature.)
2018-12-18T11:08:00.720-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.721-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'DataRef-I4nSYR9Q' from CAPI
2018-12-18T11:08:00.721-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 0x80090006 (Invalid Signature.)
2018-12-18T11:08:00.722-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.722-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'DataRef-IYonrwPb' from CAPI
2018-12-18T11:08:00.722-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 87 (The parameter is incorrect.)
2018-12-18T11:08:00.722-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.722-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'local master encryption key' from CAPI
2018-12-18T11:08:00.723-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 0x80090006 (Invalid Signature.)
2018-12-18T11:08:00.723-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.723-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'local master encryption key' from CAPI
2018-12-18T11:08:00.724-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 87 (The parameter is incorrect.)
2018-12-18T11:08:00.724-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.724-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'local master encryption key' from CAPI
2018-12-18T11:08:00.725-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 0x80090006 (Invalid Signature.)
2018-12-18T11:08:00.725-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.725-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'local storage context' from CAPI
2018-12-18T11:08:00.726-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 0x80090006 (Invalid Signature.)
2018-12-18T11:08:00.726-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.726-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'local storage context' from CAPI
2018-12-18T11:08:00.766-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault cannot verify signature, error = 0x80090006 (Invalid Signature.)
2018-12-18T11:08:00.766-04:00 ERROR (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault registry data BAD SIGNATURE
2018-12-18T11:08:00.767-04:00 WARN (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVault failed to convert 'local storage context' from CAPI
2018-12-18T11:08:00.767-04:00 INFO (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] KeyVaultCAPI upgrade rekey completed for wsnm_jms
2018-12-18T11:08:00.767-04:00 DEBUG (12A0-0E88) <delayedRekeyCAPI> [MessageFrameWork] coregate 'KeyVault' un-locked
Solution
This issue had me stumped for a while as I could not find any results from the internet with the error messages above. What ended up correcting the problem was a solution to another issue I had at another client that displayed a different error. The solution can be found in my previous blog post here:
Troubleshooting VMware Horizon View 7.5.1 Virtual Desktop in “Agent unreachable” status
http://terenceluk.blogspot.com/2018/10/troubleshooting-vmware-horizon-view-751.html
Executing the vdmadmin command to reset the key pair resolved the issue:
vdmadmin -A -d desktop-pool-name -m name-of-machine-in-pool -resetkey
C:\Program Files\VMware\VMware View\Server\tools\bin>vdmadmin -A -d desktop-pool
-name -m planet002 -resetkey
Agent Public Key
================
MIHwMIGoBgcqhkjOOAQBMIGcAkEA/KaCzo4Syrom78z3EQ5SbbB4sF7ey80etKII864WF64B81uRpH5t
9jQTxeEu0ImbzRMqzVDZkVG9xD7nN1kuFwIVAJYu3cw2nLqOuyYO5rahJtk0bjjFAkBnhHGyepz0Tuka
ScUUfbGpqvJE8FpDTWSGkx0tFCcbnjUDC3H9c9oXkGmzLik1Yw4cIGI1TQ2iCmxBblC+eUykA0MAAkBA
lgbnFPWs2bOZJHQqyOtFVDf5IndS2riwKqaJTTUxCDutBJg4AsJqRVVa/Ktfc2Nq1joL+FF6AbAOxMtG
tyHT
C:\Program Files\VMware\VMware View\Server\tools\bin>
