Pages

Monday, June 21, 2021

Attempting to upgrade ESXi on a Nutanix node manually fails with: "OSError: [Errno 39 Directory not empty: ‘/vmfs/volumes/e2be1277-f8844313-564e-dbe0fc3821c4/Nutanix’"

Problem

You’re attempting to manually upgrade a Nutanix node from VMware ESXi, 6.5.0, 17167537 to VMware ESXi, 6.7.0, 15160138 with the ISO VMware-VMvisor-Installer-201912001-15160138.x86_64.iso then patch it to VMware ESXi, 6.7.0, 17700523 with ESXi670-202103001.zip:

image

Upgrade ESXi, preserve VMFS datastore

image

image

… but the upgrade from 6.5 to 6.7 would fail with the following error after you select the upgrade option:

------ An unexpected error occurred ------

See logs for details

OSError: [Errno 39 Directory not empty: ‘/vmfs/volumes/e2be1277-f8844313-564e-dbe0fc3821c4/Nutanix’

image

The upgrade does not proceed further until you restart the host, which will boot into ESXi 6.5 normally.

Subsequent attempts to re-run the upgrade will now present the option of installing a fresh ESXi installation and not upgrade.

Install ESXi, preserve VMFS datastore

image

Solution

I could not find any Nutanix KB but I did find quite a few forum and blog posts indicating that the files in /bootbank need to be copied to /altbootbank (copy not move) so I performed the following with WinSCP:

  1. Backed up single file in /altbootbank named boot.cfg.backup and then deleted it
  2. Backed up the Nutanix folder in bootbank folder and then deleted it
  3. Copied the files in /bootbank to /altbootbank

Note that removing the Nutanix folder is important as failure to do so will not allow you to upgrade.

bootbank directory:

image

image

altbookbank directory:

image

I then rebooted the ESXi host with the 6.7 ISO mounted and was able to proceed with the upgrade and proceed to patch 6.7.

image

To be safe, I performed the following after the upgrade:

  1. Copy the file named boot.cfg.backup back into in /altbootbank
  2. Copy the the Nutanix folder back into the bootbank folder

Proceeded to upgrade

esxcli software vib install -d /vmfs/volumes/ntnx-lun03/ISO/ESXi670-202103001.zip

image

Hope this helps anyone who may encounter this issue.

1 comment:

Luis Bracamontes said...

Hi Terence - This was a live saver! We had the exact same issue and your instructions were perfect.