So I had quite the “kick off” on my first day back at work for the year 2011. To cut a long story short, I was asked to figure out why 3 of our UCS blades were knocked offline from vCenter and the rest of the network even though accessing them from UCSM’s KVM console showed that they were up. Aside from spending over an hour reviewing logs and finally realizing that someone managed to delete the the service profiles for the UCS blades (I’ll write a post about the troubleshooting process and resolution at a later time), I found myself at the point where I had to shuffle around the 2 vmnics (this UCS has a menlo card) from a Nexus 1000v switch back to a regular vSwitch all through the CLI. I’d have to say I’m not much of an expert with the commands and while I do know the basics, I can never remember the switches. What I ended up using the following Cisco document that includes all the commands I ended up using to:
- Remove a vmnic from a Nexus 1000v.
- Delete a vswif interface mapped to a port on the Nexus 1000v.
- Create a regular vSwitch.
- Recreate a vswif interface on a regular vSwitch.
- Add the vswif to a Service Console port on the regular vSwitch.
I’ve also pasted the section I used at the bottom in case I ever need to reference this again and the link is no longer available.
Update: The information provided below doesn’t show you how to remove a vmkernel port from a Nexus 1000v or distributed switch so I have followed up with another blog post to show how to do this: http://terenceluk.blogspot.com/2011/01/quick-note-about-reconfiguring-esx-4x_05.html
VMware ESX and ESXi Commands to Troubleshoot Cisco Nexus 1000V Series Switches in the Cisco Unified Computing System
In VMware ESX with a service console OS, these commands are in the /usr/sbin directory. Press ALT-F1 to display the service console login prompt, and press ALT-F11 to leave the prompt. To get to a misconfigured Cisco UCS blade for recovery, click the KVM item under the server view. This procedure assumes that the KVM IP addresses have been set up and assigned.
In VMware ESXi, the esxcfg-vswitch functions are all present as in VMware ESX. Since VMware ESXi does have a service console OS, use ALT-F2 and ALT-F11 key presses to go to and from the VMware ESXi command line. All the commands listed here for esxcfg-vswif will now be replaced with esxcfg-vmknic, and you can update specific details as required.
To List Information About the Current Configuration
esxcfg-vswif -l To list information about the current vswif interface
esxcfg-vswitch -l To list information about the current vSwitches and DVSs
To Manage the Existence of a vSwitch
esxcfg-vswitch -a vSwitch0 To add a vSwitch named vSwitch0 to the server
esxcfg-vswitch -d vSwitch0 To delete a vSwitch named vSwitch0 from the server
To Manage the Port Groups on a vSwitch
esxcfg-vswitch -A "Service Console" vSwitch0 To add a port group named Service Console to vSwitch0
esxcfg-vswitch -D "Service Console" vSwitch0 To remove a port group named Service Console from vSwitch0
To Assign VLAN Membership to a Port Group on a vSwitch
esxcfg-vswitch -p "Service Console" -v 102 vSwitch0 To set a port group named Service Console to user VLAN 102 on vSwitch0
To View the Available NICs That VMware ESX Can See
esxcfg-nics -l To see a list of VMNICs
To Manage Uplinks from the DVS or Cisco Nexus 1000V Series Switch
First run the list commands (esxcfg-vswif -l and esxcfg-vswitch -l ) to get the port numbers for the existing uplinks (vmnic0, vmnic1, etc.); this will be port#.
esxcfg-vswitch -Q vmnic0 -V [port#] myN1kDVS To remove the vmnic0 uplink from the DVS named myN1kDVS
esxcfg-vswitch -P vmnic0 -V [port#] myN1kDVS To add the vmnic0 uplink to the DVS named myN1kDVS
To add the VMNICs back into a DVS, the recommended approach is to use VMware vCenter and not the command line.
To Manage Uplinks to a vSwitch
esxcfg-vswitch -L vmnic0 vSwitch0 To add vmnic0 as an uplink to vSwitch0
esxcfg-vswitch -U vmnic0 vSwitch0 To remove vmnic0 as an uplink to vSwitch0
To remove a VMNIC from a vSwitch, the recommended approach is to use VMware vCenter and not the command line while migrating from a vSwitch to a Cisco Nexus 1000V Series DVS.
To Delete a vswif Interface from a VMware ESX Server Port Group
esxcfg-vswif -d vswif0 -p "Service Console" vSwitch0 To delete a vswif interface from the port group on vSwitch0
Esxcfg-vswif -d vswif0 -p "Service Console" -P xxx -V myN1kDVS To delete a vswif interface from the port group on the Cisco Nexus 1000V Series Switch
This deletion is a common task if an error situation arises and you lose connectivity to the VMware ESX server when migrating to the Cisco Nexus 1000V Series. You will have to add the port group to vSwitch0 and then add the vswif interface on it.
To Add a vswif Interface to the VMware ESX Server
esxcfg-vswif -a vswif0 -p "Service Console" -i 10.1.1.10 -n 255.255.255.0 To add vswif0 with IP 10.1.1.10/24 to port group Service Console
esxcfg-vswif -a vswif0 -p "Service Console" -i 10.1.1.10 -n 255.255.255.0 -P [port#]-V myN1kDVS To add vswif0 with IP 10.1.1.10/24 to port group Service Console to port = port# on DVS myN1kDVS. The port# value is usually just the last unused entry in the list of available port numbers when the DVS configuration is listed with esxcfg-vswitch -l.
To Manage a Default Gateway for the VMware ESX Server
On the VMware vCenter GUI, you can change the gateway directly, but on the command-line interface (CLI), you must edit the /etc/sysconfig/network file and update the vswif interface's default gateway. The file will look similar to this:
After saving the file, you must reboot the VMware ESX server to implement the change.
To Ping Test a VMware VMkernel Interface
esxcfg-route -l To get the routes on the VMware VMkernel network
vmkping X.X.X.X To ping an IP address on the VMware VMkernel network
To Test for the VEM Installation on a VMware ESX Host
vmkload_mod -l grep vem To verify that the appropriate VEM is loaded (the command should return the modules)