NetScaler VPX 10.5 build 54.9.nc loses network connectivity shortly after restarting the appliance

I recently ran into an issue that had me troubleshooting a NetScaler VPX issue for a good two weeks and while there’s an actual VMware and Citrix KB article about the issue, I hope that this blog post will reach out to anyone who actually follows my blog or twitter so that they are aware of it.

Problem

I was informed by a client that the NetScaler HA Pair I deployed over a year ago that the Citrix portal published behind it no longer worked and when I logged on, I noticed that I was unable to log into the management GUI of the NetScaler.  What I would basically get is the login page and once I type in the credentials and attempt to log in, I would see a white page indicating it’s loading and eventually fail after a very long time.

Attempting to ping the device would either flat out no longer respond or exhibit drop packets shown in the following screenshot:

Logging onto the console and attempting to ping other servers on the same subnet would fail but pinging the NetScaler VPX’s own IP address would work.  A quick restart of the NetScaler would return the appliance to what appear to be a regular state for various window of time but will be short lived.

The NetScaler VPX in the environment is version is 10.5 build 54.9.nc:

Solution

After troubleshooting over the week, I finally found the following VMware article:

After applying patches to an ESXi 5.1/5.5 host, Citrix NetScaler virtual machine with e1000 vNIC loses network connectivity (2092809)
http://kb.vmware.com/selfservice/microsites/search.do?language=en_US&cmd=displayKC&externalId=2092809

… and the following Citrix article:

NetScaler VPX Loses Network Connectivity on VMware ESXi 5.1.0 2191751 and VMware ESXi 5.5 2143827
http://support.citrix.com/article/CTX200278

What I found out shortly after asking was that the ESXi hosts were recently updated to ESXi 5.1 Update 3 which supersedes the Patch 5 build that VMware and Citrix indicates would cause the NetScaler to lose its network connectivity.

I can confirm that the workaround as outlined in the Citrix KB as well as updating the appliance to 10.5 build 55.8 resolves the issue.