Load Balancing Exchange Server 2019 with Citrix ADC / NetScaler

One of the most common questions I am asked by colleagues and clients is in regards to how they can publish Exchange Servers behind a Citrix ADC / NetScaler to provide load balancing for all services such as OWA, ActiveSync, RPC, EWS, AutoDiscover, MAPI, and ECP BUT NOT SMTP. I would like to further emphasize that SMTP (mail traffic) is excluded for this because I never liked suggesting load balancers to be used for this service and the reason is that the true originating IP of the traffic will be masked by the Citrix ADC / Netscaler. This effectively means that if you won’t be able to restrict IPs allowed for relaying via the Exchange receive connectors or troubleshoot an issue for a relaying client. An workaround for this solution could be to use a DSR (Direct Server Reply) configuration but limitations such as having the Exchange server and Citrix ADC on the same subnet is likely not something that will be possible for more environments. Lastly, a misconfiguration of the Citrix ADC / NetScaler (lack of filtering based on source IP address) could potentially create a open relay on the Exchange server because from the Exchange server perspective, it would relay any SMTP traffic sent by the Citrix ADC / NetScaler.

With the above said, this blog post will serve to provide the CLI commands to configure the Citrix ADC / NetScaler with the appropriate server objects, monitors, service groups, load balancing virtual server, and content switching server. Note that it is possible to not use a content switching server and simply use a load balancing virtual server to publish Exchange servers but you would not be able to independently redirect requests to specific services to another server if those a service is not available. An example to this is if ExchangeServer1 just has /owa unavailable, the content switching server will redirect requests to ExchangeServer2 but will continue to send /ews to ExchangeServer1 if it is available. Using just a load balancing server will deem ExchangeServer1 as down and redirect all traffic to another server.

The version of Exchange Server 2019 and Citrix ADC / NetScaler used for this post are as follows:

Exchange Server 2019: Version 15.2 ‎(Build 721.2)‎

Citrix ADC / NetScaler: NS13.0 67.39.nc

Note that certain version of Citrix ADC / NetScaler appear to exhibit issues with monitoring Exchange services as outlined one of my previous posts:

Citrix ADC / NetScaler monitors for Exchange 2019 fails with: "Failure - Time out during SSL handshake stage"
http://terenceluk.blogspot.com/2020/12/citrix-adc-netscaler-monitors-for.html

Prerequisites

• One private IP addresses allocated for the VIP of the Content Switching server
• Imported SSL certificate to secure traffic
• Firewall rule that allows the Citrix NSIP to the Exchange servers on port 443
• Firewall rule that allows internet traffic via the firewall to the VIP of the Content Switching server on port 443 and 80

Step #1 – Enable Features

Enable the features below by executing the following command: enable ns feature CS,RESPONDER,LB,SSL

• Content Switching
• Responder
• Load Balancing
• SSL

Step #2 – Import SSL Certificate

Either import the SSL certificate currently used on the Exchange servers to secure 443 traffic:

Importing PFX certificate from Microsoft Windows Server into Citrix NetScaler VPX
http://terenceluk.blogspot.com/2016/05/importing-pfx-certificate-from.html

Or request a new one if this is a greenfield deployment. We will need to assign the SSL certificate to the load balancing virtual server and the content switching server.

Make sure the issuing Root and Intermediate certificate chain is imported and linked as well.

Step #3 – Create the Server Objects

Create the server objects with the following command and replace the name and IP as it reflects your environment:

add server ExchServer1.contoso.com 172.16.1.81

add server ExchServer2.contoso.com 172.16.1.82

Step #4 – Create Exchange Service Monitors

The monitors we’ll be creating for are as follows:

1. OWA
2. ActiveSync
3. RPC
4. EWS
5. AutoDiscover
6. OAB
7. MAPI
8. ECP

Execute the following CLI commands to create the monitors:

add lb monitor mon_exch_owa HTTP-ECV -send "GET /owa/healthcheck.htm" -recv 200 -LRTM DISABLED -destPort 443 -secure YES

add lb monitor mon_exch_activesync HTTP-ECV -send "GET /Microsoft-Server-ActiveSync/healthcheck.htm" -recv 200 -LRTM DISABLED -secure YES

add lb monitor mon_exch_rpc HTTP-ECV -send "GET /rpc/healthcheck.htm" -recv 200 -LRTM DISABLED -secure YES

add lb monitor mon_exch_ews HTTP-ECV -send "GET /ews/healthcheck.htm" -recv 200 -LRTM DISABLED -secure YES

add lb monitor mon_exch_autodiscover HTTP-ECV -send "GET /Autodiscover/healthcheck.htm" -recv 200 -LRTM DISABLED -secure YES

add lb monitor mon_exch_oab HTTP-ECV -send "GET /oab/healthcheck.htm" -recv 200 -LRTM DISABLED -secure YES

add lb monitor mon_exch_mapi HTTP-ECV -send "GET /mapi/healthcheck.htm" -recv 200 -LRTM DISABLED -secure YES

add lb monitor mon_exch_ecp HTTP-ECV -send "GET /ecp/healthcheck.htm" -recv 200 -LRTM DISABLED -secure YES

Step #5 – Create Service Groups and Bind Servers

We’ll need to create a service group for each of the services we created a monitor for with the following CLI commands:

add serviceGroup SVG_EX2019_owa SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

add serviceGroup SVG_EX2019_activesync SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

add serviceGroup SVG_EX2019_rpc SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

add serviceGroup SVG_EX2019_ews SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

add serviceGroup SVG_EX2019_autodisover SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

add serviceGroup SVG_EX2019_oab SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

add serviceGroup SVG_EX2019_mapi SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

add serviceGroup SVG_EX2019_ecp SSL -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO

With the service groups created, proceed to bind the Exchange servers and the monitors into them with the following CLI command:

bind serviceGroup SVG_EX2019_owa ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_owa ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_owa -monitorName mon_exch_owa

bind serviceGroup SVG_EX2019_activesync ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_activesync ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_activesync -monitorName mon_exch_activesync

bind serviceGroup SVG_EX2019_rpc ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_rpc ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_rpc -monitorName mon_exch_rpc

bind serviceGroup SVG_EX2019_ews ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_ews ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_ews -monitorName mon_exch_ews

bind serviceGroup SVG_EX2019_autodisover ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_autodisover ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_autodisover -monitorName mon_exch_autodiscover

bind serviceGroup SVG_EX2019_oab ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_oab ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_oab -monitorName mon_exch_oab

bind serviceGroup SVG_EX2019_mapi ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_mapi ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_mapi -monitorName mon_exch_mapi

bind serviceGroup SVG_EX2019_ecp ExchServer2.contoso.com 443

bind serviceGroup SVG_EX2019_ecp ExchServer1.contoso.com 443

bind serviceGroup SVG_EX2019_ecp -monitorName mon_exch_ecp

Step #6 – Create Load Balancing Virtual Server, Bind Service Groups and Bind SSL Certificate

A Load Balancing Virtual Server object with an IP Address Type configured as Non Addressable is required to be created for each service:

Proceed to create these load balancing virtual servers with the following CLI commands:

add lb vserver mail.contoso.com_owa-LBVS SSL 0.0.0.0 0 -persistenceType NONE -cltTimeout 180

add lb vserver mail.contoso.com_activesync-LBVS SSL 0.0.0.0 0 -persistenceType SRCIPDESTIP -cltTimeout 180

add lb vserver mail.contoso.com_rpc-LBVS SSL 0.0.0.0 0 -persistenceType SOURCEIP -timeout 30 -cltTimeout 180

add lb vserver mail.contoso.com_ews-LBVS SSL 0.0.0.0 0 -persistenceType NONE -cltTimeout 180

add lb vserver mail.contoso.com_autodiscover-LBVS SSL 0.0.0.0 0 -persistenceType SOURCEIP -timeout 30 -cltTimeout 180

add lb vserver mail.contoso.com_oab-LBVS SSL 0.0.0.0 0 -persistenceType NONE -cltTimeout 180

add lb vserver mail.contoso.com_mapi-LBVS SSL 0.0.0.0 0 -persistenceType SOURCEIP -timeout 30 -cltTimeout 180

add lb vserver mail.contoso.com_ecp-LBVS SSL 0.0.0.0 0 -persistenceType NONE -cltTimeout 180

With the load balancing virtual servers created, proceed to bind the service groups containing the Exchange Servers to them:

bind lb vserver mail.contoso.com_owa-LBVS SVG_EX2019_owa

bind lb vserver mail.contoso.com_activesync-LBVS SVG_EX2019_activesync

bind lb vserver mail.contoso.com_rpc-LBVS SVG_EX2019_rpc

bind lb vserver mail.contoso.com_ews-LBVS SVG_EX2019_ews

bind lb vserver mail.contoso.com_autodiscover-LBVS SVG_EX2019_autodisover

bind lb vserver mail.contoso.com_oab-LBVS SVG_EX2019_oab

bind lb vserver mail.contoso.com_mapi-LBVS SVG_EX2019_mapi

bind lb vserver mail.contoso.com_ecp-LBVS SVG_EX2019_ecp

Lastly, bind the imported SSL certificate (Step #2) to the load balancing virtual server with the following commands:

bind ssl vserver mail.contoso.com_owa-LBVS -certkeyName wildcard-contoso-exp-mar-2021

bind ssl vserver mail.contoso.com_activesync-LBVS -certkeyName wildcard-contoso-exp-mar-2021

bind ssl vserver mail.contoso.com_rpc-LBVS -certkeyName wildcard-contoso-exp-mar-2021

bind ssl vserver mail.contoso.com_ews-LBVS -certkeyName wildcard-contoso-exp-mar-2021

bind ssl vserver mail.contoso.com_autodiscover-LBVS -certkeyName wildcard-contoso-exp-mar-2021

bind ssl vserver mail.contoso.com_oab-LBVS -certkeyName wildcard-contoso-exp-mar-2021

bind ssl vserver mail.contoso.com_mapi-LBVS -certkeyName wildcard-contoso-exp-mar-2021

bind ssl vserver mail.contoso.com_ecp-LBVS -certkeyName wildcard-contoso-exp-mar-2021

Step #7 – Create a Content Switching Server, Bind SSL Certificate, Create the Content Switching Policies, Bind the Policies to the Content Switching Server

With all the dependent components created and configured, proceed to create the content switching server, which will have the pre-allocated VIP assigned and configured to direct requests to the Exchange services to a server that has a healthy status:

add cs vserver mail.contoso.com_http-CS HTTP 172.16.5.62 80 -cltTimeout 180 -persistenceType NONE

add cs vserver mail.contoso.com_ssl-CS SSL 172.16.5.62 443 -cltTimeout 180 -persistenceType NONE

Bind the same SSL certificate used for the load balancing virtual server to the SSL content switching server:

bind ssl vserver mail.contoso.com_ssl-CS -certkeyName wildcard-contoso-exp-mar-2021

Create the content switching policies that represent each of the services being published by the load balancing virtual servers:

add cs action cs_act_exch2019_owa -targetLBVserver mail.contoso.com_owa-LBVS

add cs action cs_act_exch2019_ews -targetLBVserver mail.contoso.com_ews-LBVS

add cs action cs_act_exch2019_autodiscover -targetLBVserver mail.contoso.com_autodiscover-LBVS

add cs action cs_act_exch2019_activesync -targetLBVserver mail.contoso.com_activesync-LBVS

add cs action cs_act_exch2019_oab -targetLBVserver mail.contoso.com_oab-LBVS

add cs action cs_act_exch2019_mapi -targetLBVserver mail.contoso.com_mapi-LBVS

add cs action cs_act_exch2019_rpc -targetLBVserver mail.contoso.com_rpc-LBVS

add cs action cs_act_exch2019_ecp -targetLBVserver mail.contoso.com_ecp-LBVS

add cs policy cs_pol_exch2019_owa -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/owa\")" -action cs_act_exch2019_owa

add cs policy cs_pol_exch2019_ews -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/ews\")" -action cs_act_exch2019_ews

add cs policy cs_pol_exch2019_autodiscover -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/autodiscover\")" -action cs_act_exch2019_autodiscover

add cs policy cs_pol_exch2019_activesync -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"Microsoft\")" -action cs_act_exch2019_activesync

add cs policy cs_pol_exch2019_oab -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/oab\")" -action cs_act_exch2019_oab

add cs policy cs_pol_exch2019_mapi -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/mapi\")" -action cs_act_exch2019_mapi

add cs policy cs_pol_exch2019_rpc -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/rpc\")" -action cs_act_exch2019_rpc

add cs policy cs_pol_exch2019_ecp -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/ecp\")" -action cs_act_exch2019_ecp

An additional OWA policy will also need to be added to address a but identified in the following KB:

OWA access via NetScaler is getting stuck at /cgi/selfauth?params= after authentication
https://support.citrix.com/article/CTX209060

add cs policy cs_pol_exch2019_cgi -rule "HTTP.REQ.URL.SET_TEXT_MODE(IGNORECASE).CONTAINS(\"/cgi\")" -action cs_act_exch2019_owa

The following policy will redirect requests going to the mail.contoso.com address to mail.contoso.com/owa:

add cs policy cs_pol_exch2019_owa_redirect -rule "HTTP.REQ.HOSTNAME.EQ(\"mail.contoso.com\")" -action cs_act_exch2019_owa

Lastly, proceed to bind the policies created above to the content switching server:

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_owa -priority 110

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_ews -priority 120

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_autodiscover -priority 130

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_activesync -priority 140

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_oab -priority 150

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_mapi -priority 160

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_rpc -priority 170

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_ecp -priority 180

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_cgi -priority 190

bind cs vserver mail.contoso.com_ssl-CS -policyName cs_pol_exch2019_owa_redirect -priority 200

Step #8 – Create a Responder to redirect http to https

The final step for the configuration is to create a responder to redirect http to https so users typing in http://mail.contoso.com will automatically be redirected to https://mail.contoso.com:

add responder action redirect_http_https_act_owa redirect "\"https://\"+HTTP.REQ.HOSTNAME+\"/owa/\""

add responder policy redirect_http_https_pol_owa "HTTP.REQ.HOSTNAME.CONTAINS(\"mail.contoso.com\")" redirect_http_https_act_owa

bind cs vserver mail.contoso.com_http-CS -policyName redirect_http_https_pol_owa -priority 100 -gotoPriorityExpression END -type REQUEST

The above steps should configuring the required components for load balancing Exchange servers with the Citrix ADC NetScaler. I will include some screenshots of the configuration in case anyone wants to see the configuration in the GUI.

Screenshots

Load Balancing Virtual Servers:

Service Groups:

Content Switching Server: