On-prem Skype for Business Server 2015 federation with Skype for Business Online logs the event: "Unable to resolve DNS SRV record";domain="fabrikam.com";dns-srv-result="NegativeResult";dns-source="WireQuery";source="sip.contoso.com"

Problem

You’ve configured your Skype for Business Server 2015 SIP FEDERATED PROVIDERS to federate with Skype for Business Online domains:

… but notice that one of the domains you are trying to reach is listing the presence information of the contact as Presence unknown:

Attempting to manually configure their domain in the SIP FEDERATED DOMAINS with the Edge Server configured as sipdir.online.lync.com:

… allows you to view their presence and trade messages but they are unable to view yours:

Using the Skype for Business Server 2015 Logging Tool to capture the IMAndPresence scenario:

Shows the following SIP/2.0 404 Not Found log entries:

TL_INFO(TF_PROTOCOL) [bmlyncedge01\bmlyncedge01]0608.10A4::03/02/2018-20:08:58.210.00002EA8 (SIPStack,SIPAdminLog::ProtocolRecord::Flush:ProtocolRecord.cpp(261)) [1193230895] Trace-Correlation-Id: 1193230895

Instance-Id: 9BD4F

Direction: outgoing;source="local";destination="internal edge"

Peer: bmlyncstd01.contoso.com:65371

Message-Type: response

Start-Line: SIP/2.0 404 Not Found

From: "Terence Luk"<sip:tluk@contoso.com>;tag=b57c0d9a53;epid=de538b165f

To: <sip:kseidl@fabrikam.com>;tag=61AAA34DF2202DB35CF394AA3FF1E420

Call-ID: ed355df33e6e40e8afe98664b3abe01b

CSeq: 1 SUBSCRIBE

Via: SIP/2.0/TLS 10.21.1.106:65371;branch=z9hG4bKBD5B17D0.7277A46B0013221F;branched=FALSE;ms-received-port=65371;ms-received-cid=9BD600

Via: SIP/2.0/TLS 10.23.0.28:58599;ms-received-port=58599;ms-received-cid=817000

Content-Length: 0

ms-diagnostics: 1008;reason="Unable to resolve DNS SRV record";domain="fabrikam.com";dns-srv-result="NegativeResult";dns-source="WireQuery";source="sip.contoso.com"

ms-edge-proxy-message-trust: ms-source-type=EdgeProxyGenerated;ms-ep-fqdn=bmlyncedge01.contoso.com;ms-source-verified-user=verified

$$end_record

Solution

One of the common causes of symptoms described above is if the domain hosted in Skype for Business Online does not have the appropriate SRV setup to allow the on-prem SfB to discover the company’s SfB. To ensure that the appropriate SRV is configured, simply attempt to look up the record with the following commands:

nslookup

set q=srv

_sipfederationtls.tcp.<domainFQDN>

A domain properly configured with the required DNS record should look like the output circled in green while a domain with the missing record should look like the output circled in red:

Creating the missing record will correct the issue.

Extra Troubleshooting Step

I went through numerous troubleshooting steps prior to looking at the obvious issue as outlined above because the problematic company had insisted that they did not have any issues with other partners so my assumption was that the domain I was working with wasn’t configured properly. The following may not have been the solution for this problem but I feel is worth checking as well.

One of the other possible causes to the issue above is if your domain is also configured for Office 365 services and you had inadvertently configured the domain to provide Skype for Business Online services. If this was the case then you would either have to configure interoperability between your on-premises deployment and Skype for Business Online as outlined in the following TechNet article:

Configure federation with Skype for Business Online

https://technet.microsoft.com/en-us/library/jj205126.aspx

If there is no intention on having Skype for Business Online accounts in Office 365 then ensure that the service is turned off. To check, simply launch the Azure Active Directory Module for Windows PowerShell and connect to the domain’s Office 365 account:

Use commands such as:

Get-MsolDomain -DomainName <domain>.com | FL

… or:

Get-MSolDomain | Select Name,Capabilities

… to determine whether Skype for Business Online is enabled for the domain:

vSphere Replication replicated virtual machines in SRM protected groups have the “Protection Status” as “Replication Error”

I’ve been working with VMware Site Recover Manager (SRM) sparsely over the past few years and had thought I’ve seen most of the error messages but came across one today that I had not seen before so I thought it would be worth while to write a quite post for it.

Problem

Logging into the SRM section of a vSphere Web Client shows all of the Protection Groups listed as Not Configured:

Navigating into the list of Virtual Machines show the Protection Status as Replication Error:

Solution

One of the possible causes of this status being displayed in SRM is if the VMware vCenter Site Recovery Manager service on the target site is not started.  Starting the service will correct the issue.

Add features to an existing VMware Horizon View 7.x Agent install

Those who are familiar with the VMware Horizon View agent install may have noticed that there does not appear to be an easy way to add features to an existing agent:

… because attempting to use the Programs and Features or re-running the installation file only presents the option of removing the agent:

This may not be an issue if you only had to update a master image but certainly not practical you had to repeat the process multiple times for a full-clone deployment.  I’ve come across this in the past multiple times and thought it would be worth while to write this blog post to demonstrate the process for anyone who may be looking for this information.

Determining Installed Horizon View Agent Components

To begin, the way to determine what components are installed currently can be found in the agent install logs located in the folder:

C:\ProgramData\VMware\logs

The log named vmmsi.log_<date>.log with the latest timestamp is the file with this information.  Opening the logs and navigating down to the section beginning with:

Action start 21:20:57: InstallValidate.

MSI (s) (8C:C4) [21:20:57:927]: Feature: URLRedirection; Installed: Absent; Request: Null; Action: Null

… is where you will find all the features of the agent listed and whether they were installed.

Feature: this is the feature of the agent

Installed: this is the installation status of the feature during the install
The values are Absent and Local

Request: this is whether a request to install it was made
The values are Null, Local and Reinstall

Action: this is the action taken for the feature
The values are Null, Local and Reinstall

Adding Horizon View Agent Components (non agent upgrade)

The only way to add Horizon View agent components to an existing installation is to use the command line and features matrix as described in the following documentation:

Silent Installation Properties for Horizon Agent
https://docs.vmware.com/en/VMware-Horizon-7/7.4/horizon-virtual-desktops/GUID-3096DA8B-034B-435B-877E-5D2B18672A95.html

The command would look as such:

VMware-viewagent-y.y.y-xxxxxx.exe /s /v"/qn VDM_VC_MANAGED_AGENT=1 ADDLOCAL=<featureToAdd>

To demonstrate the process, let’s begin by performing a fresh install of the View agent with only the Core, USB Redirection and Real-Time Audio-Video components:

Note the agent install log indicating the RTAV (Real-Time Audio-Video) component is installed (along with USB and other core components):

MSI (s) (D0:B0) [19:11:50:601]: Feature: RTAV; Installed: Absent; Request: Local; Action: Local

To add a component such as the Client Drive Redirection simply run the silent install command with the following switch:

VMware-viewagent-x86_64-7.0.2-4368292.exe /s /v"/qn VDM_VC_MANAGED_AGENT=1 ADDLOCAL=ClientDriveRedirection

**Note that if you’re only specifying one feature with the ADDLOCAL switch then you do not see a quote at the end but if you specify more than one then you will:

Browsing the agent install log once the process has completed will show the following output for the feature indicating it was absent but now installed:

MSI (s) (28:D4) [20:28:45:916]: Feature: ClientDriveRedirection; Installed: Absent; Request: Local; Action: Local

Note that the RTAV component being already installed has a different output than when it was being installed:

MSI (s) (28:D4) [20:28:45:916]: Feature: RTAV; Installed: Local; Request: Null; Action: Null

Upgrading Horizon View Agent and Adding Components

One of the common mistakes I’ve seen administrators in the past make is that they are upgrading the Horizon View Agent and inadvertently using the silent install with the additional component they want to add to the agent.  This will actually install the newer agent with the single or multiple components specified in the ADDLOCAL switch because an upgrade of the agent actually removes the previous agent and reinstalls the new one thus not take the previous installed components into consideration.  If you are upgrading the agent silently then you’ll need to ensure ADDLOCAL=ALL for all components or ADDLOCAL=<features>, to ensure the features that are required would be installed.

Extra Information

The following are for those who may want to reference what a default installation of the Horizon View Agent 7.4.0 components look like:

The log out put would be as such:

Action start 21:20:57: InstallValidate.

MSI (s) (8C:C4) [21:20:57:927]: Feature: URLRedirection; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: PSG; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: VmVideo; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: VmwVdisplay; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: FlashURLRedirection; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: RDSH3D; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: SVIAgent; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: NGVC; Installed: Absent; Request: Absent; Action: Absent

MSI (s) (8C:C4) [21:20:57:927]: Feature: ScannerRedirection; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: SerialPortRedirection; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: SmartCard; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: TSMMR; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: ThinPrint; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: USB; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: V4V; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: VPA; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: VmwVaudio; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: DeviceBridgeBAS; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: CIT; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: RDP; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: BlastUDP; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: Core; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: VMWMediaProviderProxy; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: ClientDriveRedirection; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: RTAV; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Feature: FLASHMMR; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Feature: HTML5MMR; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_agentAutoUpdate_BuildNumber.reg; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_pcoipAudioFwd.reg; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _Registry_FipsEnabled; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _Registry_IpProtocolUsage; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _Registry_SetDefaultMgmtPort; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _Registry_EnableRDSH3D; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _Registry_bEnumerateHWBeforeSW; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _Registry_disable_NLA; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _arp.ico; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula.rtf; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula_de.rtf; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula_es.rtf; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula_fr.rtf; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula_ja.rtf; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula_ko.rtf; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula_zh_CN.rtf; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _eula_zh_TW.rtf; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _open_source_licenses.txt; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _pcoip_server_fips_mode.reg; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _pcoip_tcpport.reg; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _pcoip_udpport.reg; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _audio_enumerator_def.reg; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _nodeManagerSecuredQueues.reg; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _ws_perfMon.dll_reg; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _vmware_url_protocol_intercept.dll; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _vmware_url_protocol_intercept.dll_64; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _vmware_url_filtering_plugin.dll; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _vmware_url_protocol_launch_helper.exe; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _vmware_url_filtering_plugin.dll_64; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _abctrl.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _bin_wsnm_scredir.dll; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_pcoip_vmwsci32.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _smartcard_vmwsci64.dll; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:927]: Component: _pcoip_vmwsci64.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _vmkbd.sys; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_VMwareResolutionSet.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_VMwareResolutionSet.exe; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_audiodevtap.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_inputdevtap.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_pcoip_agent_win64.dll_reg; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_libeay32.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_ssleay32.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_pcoip_glib_2.0.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_pcoip_pcre.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_pcoip_iconv.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:927]: Component: _32_pcoip_intl.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_pcoip_libidn.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_pcoip_server_win32.exe; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_pcoip_perf_installer.exe; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_pcoip_vchan.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_pcoip_vmtools.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_pthreadVC2.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_svgadevtap.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _32_tera2800_accel.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _NvFBCEnable.exe; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _ICuiSDK32.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _libmfxsw32.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _pcoip_agent_win64.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _pcoip_perf_provider64.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _pcoip_server_win32.reg; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _psg.reg; Installed: Absent; Request: Null; Action: Null

MSI (s) (8C:C4) [21:20:57:928]: Component: _vmwicpdr.reg; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _mfxplugin32_screen_capture.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _plugin.cfg; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _64_VMwareResolutionSet.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _64_VMwareResolutionSet.exe; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _64_pcoip_glib_2.0.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _64_pcoip_pcre.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _64_pcoip_iconv.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _64_pcoip_intl.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _64_pcoip_vmtools.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _pcoip_vchan64.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _vdp_rdpvcbridge.dll; Installed: Absent; Request: Local; Action: Local

MSI (s) (8C:C4) [21:20:57:928]: Component: _vmwsci.dll; Installed: Absent; Request: Local; Action: Null

MSI (s) (8C:C4) [21:20:57:928]: Component: _wsnotify.dll; Installed: Absent; Request: Local; Action: Local

Now if you were to add the VMware Virtualization Pack for Skype for Business feature:

… by executing:

VMware-viewagent-x86_64-7.4.0-7400533.exe /s /v"/qn VDM_VC_MANAGED_AGENT=1 ADDLOCAL=VMWMediaProviderProxy

**Note make sure the Skype for Business client is is closed or it won’t install!

… then you’ll see the following output in the logs:

MSI (s) (CC:04) [14:02:09:034]: PROPERTY CHANGE: Adding PRODUCTLANGUAGE property. Its value is '1033'.

MSI (s) (CC:04) [14:02:09:034]: PROPERTY CHANGE: Adding VDM_VC_MANAGED_AGENT property. Its value is '1'.

MSI (s) (CC:04) [14:02:09:035]: PROPERTY CHANGE: Adding ADDLOCAL property. Its value is 'VMWMediaProviderProxy'.

MSI (s) (CC:04) [14:02:09:035]: PROPERTY CHANGE: Adding CURRENTDIRECTORY property. Its value is 'C:\Users\a-tluk\AppData\Local\Temp\{0159A3BB-1690-4203-9383-4900B9E34B0B}~setup'.

MSI (s) (CC:04) [14:02:09:035]: PROPERTY CHANGE: Adding CLIENTUILEVEL property. Its value is '3'.

MSI (s) (CC:04) [14:02:09:035]: PROPERTY CHANGE: Adding CLIENTPROCESSID property. Its value is '6732'.

MSI (s) (CC:04) [14:02:09:035]: Machine policy value 'DisableAutomaticApplicationShutdown' is 0

MSI (s) (68:EC) [13:38:11:820]: Feature: VMWMediaProviderProxy; Installed: Absent; Request: Local; Action: Local

Voicemails left with Skype for Business Server 2015 on Exchange Server 2016 does not get delivered to user’s inbox

Problem

You’ve just completed integrating Skype for Business Server 2015 with Exchange Server 2016 Unified Messaging to provide voicemail services but noticed that voicemails left for users are not being delivered and the following errors are being logged on the Exchange 2016 server:

Log Name: Application

Source: MSExchange Unified Messaging

Event ID: 1423

Level: Error

The Microsoft Exchange Unified Messaging service on the Mailbox server encountered an error while trying to process the message with header file "E:\Program Files\Microsoft\Exchange Server\V15\UnifiedMessaging\voicemail\c7d3e3ee-0212-4d27-99bd-f1960eca0698.txt". Error details: "Microsoft.Exchange.UM.UMCore.SmtpSubmissionException: Submission to the Hub Transport server failed. The operation will be retried. ---> Microsoft.Exchange.Net.ExSmtpClient.UnexpectedSmtpServerResponseException: Unexpected SMTP server response. Expected: 220, actual: 421, whole response: 421 4.3.2 Service not available

at Microsoft.Exchange.Net.ExSmtpClient.SmtpTalk.CheckResponse(ServerResponseInfo response, Int32 expectedCode)

at Microsoft.Exchange.Net.ExSmtpClient.SmtpTalk.Connect(String server, Int32 port, Boolean isSslPort)

at Microsoft.Exchange.Net.ExSmtpClient.SmtpClient.Submit()

at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.SubmitMessage(MessageItem message, String senderAddress, Guid senderOrgGuid, String recipientAddress, OutboundConversionOptions submissionConversionOptions, InternalExchangeServer smtpServer)

at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.SubmitMessage(MessageItem message, String senderAddress, Guid senderOrgGuid, String recipientAddress, OutboundConversionOptions submissionConversionOptions, String requestId)

--- End of inner exception stack trace ---

Server stack trace:

at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.HandleTransientSmtpFailure(Exception e, InternalExchangeServer smtpServer, String recipientAddress)

at Microsoft.Exchange.UM.UMCore.SmtpSubmissionHelper.SubmitMessage(MessageItem message, String senderAddress, Guid senderOrgGuid, String recipientAddress, OutboundConversionOptions submissionConversionOptions, String requestId)

at Microsoft.Exchange.UM.UMCore.SmtpSubmitStage.InternalDoSynchronousWork()

at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(IntPtr md, Object[] args, Object server, Object[]& outArgs)

at System.Runtime.Remoting.Messaging.StackBuilderSink.AsyncProcessMessage(IMessage msg, IMessageSink replySink)

Exception rethrown at [0]:

at System.Runtime.Remoting.Proxies.RealProxy.EndInvokeHelper(Message reqMsg, Boolean bProxyCase)

at System.Runtime.Remoting.Proxies.RemotingProxy.Invoke(Object NotUsed, MessageData& msgData)

at Microsoft.Exchange.UM.UMCore.SynchronousPipelineStageBase.SynchronousWorkDelegate.EndInvoke(IAsyncResult result)

at Microsoft.Exchange.UM.UMCore.SynchronousPipelineStageBase.EndSynchronousWork(IAsyncResult r)"

Log Name: Application

Source: MSExchange Unified Messaging

Event ID: 1446

Level: Error

The Microsoft Exchange Unified Messaging service on the Mailbox server failed to process the message with header file "E:\Program Files\Microsoft\Exchange Server\V15\UnifiedMessaging\voicemail\1fbf0f79-adeb-4b6c-a7e2-792ec27a5351.txt" within "11" minutes. The server will continue to process and deliver the message, but the "MSExchangeUMAvailability: % of Messages Successfully Processed Over the Last Hour" performance counter will be decreased.

Solution

Similar to one of my previous posts from 7 years ago:

Voicemails left on Exchange UM does not get delivered to inbox – Event ID 1082 and 1035 logged
http://terenceluk.blogspot.com/2010/07/voicemails-left-on-exchange-um-does-not.html

… the issue was caused by the receiving connector on the Exchange server which hosts the Unified Messaging role (note that Exchange 2016 no longer splits roles into other servers).  The following snippet from the event ID 1423 error logged displays what response the Exchange server is receiving when an attempt to deliver the voice is made:

The Microsoft Exchange Unified Messaging service on the Mailbox server encountered an error while trying to process the message with header file "E:\Program Files\Microsoft\Exchange Server\V15\UnifiedMessaging\voicemail\c7d3e3ee-0212-4d27-99bd-f1960eca0698.txt". Error details: "Microsoft.Exchange.UM.UMCore.SmtpSubmissionException: Submission to the Hub Transport server failed. The operation will be retried. ---> Microsoft.Exchange.Net.ExSmtpClient.UnexpectedSmtpServerResponseException: Unexpected SMTP server response. Expected: 220, actual: 421, whole response: 421 4.3.2 Service not available

You can verify this by logging onto one of the Exchange servers and initiate a telnet to port 25 to another as shown here:

421 4.3.2 Service not available

Connection to host lost.

C:\>

If the environment you’re working in has many receive connectors configured:

… my suggestion is to not to attempt figuring out which routing connector may be receiving the request and rejecting the connection but rather create new ones specifically for UM with the the scope configured with the Exchange server IP address and the following security configuration:

You should see the queued files stored in the folder:

… begin to clear out when the Exchange server begins accepting the voicemail delivery connections.

VMware Horizon View 7.4.0 virtual desktops does not automatically power on after upgrade

I recently upgraded a client’s VMware Horizon View 7.0.2 infrastructure to the latest 7.4.0 version Build 7400497:

… and received reports the next day that virtual desktops were not being started up when the user had shut them down even though the Remote Machine Power Policy was configured as Ensure machines are always powered on:

After reviewing the event logs and performing various troubleshooting steps without any luck, I gave the recommendation VMware support always suggested:

Shut down all View servers then power them on one after another to allow the database to synchronize properly

… a try and was able to correct the issue.  Hope this post will help anyone who may encounter this problem.  Please refer to the following KB for a more detailed outline of the steps for restarting the servers:

Restart order of the View environment to clear ADLDS (ADAM) synchronization in Horizon View (2068381)
https://kb.vmware.com/s/article/2068381

Upgrading VMware Horizon View from 7.0.2 to 7.4.0 causes VMware Horizon Composers to display a Red status in the View Administrator with "The service is not working properly" and SSL as "Unknown"

Problem

You have just successfully upgraded your VMware Horizon View 7.0.2 environment to 7.4.0 but noticed that the System Health of the View Composer Servers now displays a Red status with The service is not working properly and SSL as Unknown with no option of accepting the self signed certificate that the Composer service is using as you would normally expect to see.

The View 7.4.0 environment is currently using an older vSphere 5.5 environment and you have confirmed that TLS 1.0 has been enabled as per the following document:

Enable TLSv1.0 on vCenter and ESXi Connections from View Compose

https://docs.vmware.com/en/VMware-Horizon-7/7.4/horizon-upgrades/GUID-1845E4AD-E84A-4304-A7DD-83170B8D21C5.html

Reviewing the event logs on the View Connection server show that the following error is logged:

Log Name: Application
Source: VMware

Event ID: 105

Level: Error

User: System

BROKER_SVI_CERT_INVALID

Certificate is invalid for Composer at address https://contosodrvc01.contoso.com:18443

Attributes:

            Node=contosoUKVV01.contoso.com

            ComposerId=https://contosodrvc01.contoso.com:18443

            Severity=ERROR

            Time=Fri Feb 02 22:01:58 GMT 2018

            Module=Broker

            Source=com.vmware.vdi.desktopcontroller.PublishVcCertToSviFederatedTask

            Acknowledged=true

The Events in the View Administrator console display the following message:

Certificate is invalid for Composer at address https://<vcenter>.FQDN.com:18443

Attempting to view the properties of a linked-clone pool displays the following error:

The certificate configured on View Comopser Server is invalid, blocking communication with this server. To resume communication, replace the certificate with a valid certificate signed by a CA. Alternatively, accept the certificate thumbprint by clicking Verify in the View Administrator dashboard.

You’ve confirmed that the correct self-signed certificate is assigned to the VMware Horizon View Composer service by executing the command:

sviconfig -operation=ReplaceCertificate -delete=false

You’ve tried changing the pae-SVIURL to the short NetBIOS name instead of the FQDN:

Solution

If you do not include on issuing a certificate from a trusted Certificate Authority and would like to use the self-signed certificate, the way to accept it is to navigate to the View Configuration > Servers menu, select the vCenter instance then click Edit to bring up the vCenter Server settings, then click Edit under the View Composer Server Settings section:

Getting into the configuration settings of the View Composer will now display the following invalid certificate detected prompt:

Click on the View Certificate… button will display the option to accept the self-signed certificate:

Accepting the certificate should now display the View Composer Servers health as green: