For those who have undertaken the task or project to move resources from one Azure subscription to another will know that the operation within the Azure Portal is a matter of a few clicks but the complexity can significantly increase depending on the type, the amount, and the dependencies of the resources that are being moved. While I feel that Microsoft will eventually make this potentially painful process something of the pass, it can be a daunting task today as I write this post. To help demystify a common operation that is made, I would like to demonstrate how to move an Azure virtual machine and the Recovery Services Vault with its backups from one subscription to another.
Microsoft Documentation
To begin, I would like to provide the following three links that should be reviewed to understand the operations this post will be demonstrating:
Move resources to a new resource group or subscription
https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/move-resource-group-and-subscription
Move a Recovery Services vault across Azure Subscriptions and Resource Groups
https://docs.microsoft.com/en-us/azure/backup/backup-azure-move-recovery-services-vault?toc=/azure/azure-resource-manager/toc.json
Move operation support for resources
https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/move-support-resources
Move guidance for virtual machines
https://docs.microsoft.com/en-us/azure/azure-resource-manager/management/move-limitations/virtual-machines-move-limitations
Test Environment
The following are the two Pay-as-you-go subscriptions I’ll be using:
The following is the Test-VM I will be moving from Azure Source Sub to Azure Destination Sub (note that tags are configured for this VM):
The following are the components all placed into the resource group RG-VMs and their respective Resource IDs, which will change after the subscription move:
Virtual Machine:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/virtualMachines/Test-VM
VM NIC:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/networkInterfaces/Test-VM-nic-9190f27cc6cc40a3bede740db168b0d3
VM Public IP with Basic SKU (basic SKU IP addresses can be moved but standard cannot):
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/publicIPAddresses/Test-VM-pip-90462821bc164f12a624afccf204b93c
VNet:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet
VM Disk:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611
The following is the Recovery Services Vault that is configured to backup the Test-VM:
The following is the backup status for the Test-VM (note that there are backups configured and a few restore points):
Different behavior during Subscription to Subscription Move
I’ve found that the subscription to subscription move operation changes depending on when you run it.
One of them will run a validation check prior then stopping to allow you to initiate it. The second will run the validation and upon successfully completing it will automatically initiate the move. There does not appear to be any consistency in which is presented but the GUI will be different so it is best to assume that you will not get the option to initiate the move.
Subscription to Subscription Move Blockers (Validation errors)
The subscription to subscription move operation will run a validation check prior to allow you to initiate it and the following are the type of errors that would be presented.
You cannot move a VM with backups enabled:
{"code":"DiskHasRestorePoints","target":"Microsoft.Compute/disks","message":"The move resources request contains resources like /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611 that are being backed up as part of a Azure Backup job. Browse the link https://aka.ms/vmbackupmove for information.","details":[{"code":"DiskHasRestorePoints","target":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611","message":"The move resources request contains resources like /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611 that are being backed up as part of a Azure Backup job. Browse the link https://aka.ms/vmbackupmove for information."}]}
You cannot move a VM without deleting its restore collection points even if the backup is disabled:
The same message is displayed as having backups configured but the component that isn’t passing the validation is DiskHasRestorePoints:
{"code":"DiskHasRestorePoints","target":"Microsoft.Compute/disks","message":"The move resources request contains resources like /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611 that are being backed up as part of a Azure Backup job. Browse the link https://aka.ms/vmbackupmove for information.","details":[{"code":"DiskHasRestorePoints","target":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611","message":"The move resources request contains resources like /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611 that are being backed up as part of a Azure Backup job. Browse the link https://aka.ms/vmbackupmove for information."}]}
You cannot move a VM attached to a VNET that has another VM attached to it that is not being moved:
In this example, the Test-VM to be migrated is attached to the Test-VNet VNet. This VNet also has another VM named Test-VM2 attached to it, which isn’t being moved.
{"code":"MissingMoveDependentResources","target":"Microsoft.Network/virtualNetworks","message":"The move resources request does not contain all the dependent resources. Please check details for missing resource Ids.","details":[{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/virtualMachines/Test-VM2"},{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/publicIPAddresses/Test-VM2-ip"}]
You cannot move a VM without the VNet it is attached to:
{"code":"MissingMoveDependentResources","target":"Microsoft.Network/networkInterfaces","message":"The move resources request does not contain all the dependent resources. Please check details for missing resource Ids.","details":[{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet"}]}
You cannot move a VM with a public IP address without moving the VNet it is attached to:
{"code":"MissingMoveDependentResources","target":"Microsoft.Network/publicIPAddresses","message":"The move resources request does not contain all the dependent resources. Please check details for missing resource Ids.","details":[{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet"}]}
You cannot move a VM without the attached public IP address that is assigned to it (you can move a Basic SKU IP address but not Standard):
{"code":"MissingMoveDependentResources","target":"Microsoft.Network/networkInterfaces","message":"The move resources request does not contain all the dependent resources. Please check details for missing resource Ids.","details":[{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/publicIPAddresses/Test-VM-pip-90462821bc164f12a624afccf204b93c"},{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet"}]}
You cannot move a VM with a dependent VNet that has a VNet Peering configured with another VNet regardless of whether the peered VNet is being moved as well:
{"code":"CannotMoveResource","target":"Microsoft.Network/virtualNetworks","message":"Cannot move one or more resources in the request. Please check details for information about each resource.","details":[{"code":"CannotMoveVnetDueToPeering","message":"Cannot move virtual network /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet because it's peered with other virtual networks: /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-NewVNet/providers/Microsoft.Network/virtualNetworks/Test-VNet2."}]}
You cannot move a VNet with App Service attached to it (Regional VNet Integration):
{"code":"CannotMoveResource","target":"Microsoft.Network/virtualNetworks","message":"Cannot move one or more resources in the request. Please check details for information about each resource.","details":[{"code":"CannotMoveResourceDueToReference","message":"Cannot move resource /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet since it references resource /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Web/serverfarms/ASP-RGVMs-9e72, which does not support move or updating references after the move."}]}
The validation process will not throw and error if you attempt to move a recovery services vault with backups enabled. However, you will need to move the virtual machines along with it and use the same target VM resource group name (as it was in old subscription) to continue backups.
It is important to note that regardless of whether the virtual machine is configured to be backed up by the Recovery Services Vault again, you will be able to restore the backups carried over from the time when the RSV was in the old subscription.
Moving virtual machine from one subscription to another
Step #1 – Identify Dependencies for all components
Before starting any configuration, it is important to spend time to identify every component of the virtual machine that are dependent on a resource in which other components that are not being moved are dependent on. An example of this is if you have to virtual machines:
- Test-VM1
- Test-VM2
… and both of these VMs have NICs connected to the same VNet. As mentioned earlier, you cannot move a virtual machine without the VNet it is connected to and attempting to move a VNet means you would also need to move other machines in that same VNet. When you need to move all VMs attached to the same VNet, you will also need to consolidate those VM resources into the same resource group. Finally, then there are other component such as App Services, which can use Regional VNet Integration, which attaches to a VNet for egress traffic that would be routed directly to the VNet instead of the internet but is not supported to be moved when attached to a VNet. Leverage information and diagrams provided by the Azure portal for planning (e.g. VNet > Monitoring > Diagram to identify resources connected to the VNet). Only proceed to the next step after all the dependent components have been identified.
Step #2 – Disable backup for virtual machine
Navigate to the backup configuration for the virtual machine and click on Stop backup to stop the backup:
There are two options for how the backup data is handled. Retain Backup Data will retain the backups until the retention period is reached, in which the backup will expire. Delete Backup Data will remove the backups immediately:
Successfully stopping the backup will disable the Stop backup button and enable the Resume backup button will be available as well as state the Last backup status as Warning(Backup disabled):
Step #3 – Delete restore point collections
With the backup disabled for the virtual machine, proceed to delete the delete the restore points from the vault by locating the resource group Azure creates when a Recovery Services Vault is created. The naming convention is:
AzureBackupRG_<RSV-region>_1
You won’t see the restore point collections until you select Show hidden types:
Proceed to select the restore point collection and delete them (this will not delete the existing backups OR prevent the restore of them):
Step #4 – Delete any VNet peerings
As the VM’s NIC will be associated with a VNET, any peerings configured between the VNet will need to be deleted. Note that you will need to do this even if the VNet was peered with another VNet that is being moved at the same time.
Step #5 – Consolidate the virtual machine and all of its dependent resources into one resource group
Ensure that all the components of the virtual machine as well as dependent components are placed into one resource group so they can be selected to together and be moved.
Step #6 – Move virtual machine and machine and all of its dependent resources to the destination subscription
Proceed to initiate the move of the resources to the destination subscription (you can move the RSV together with the resources as well), which will not cause an outage as the resource will continue to be reachable. This particular VM also has a Basic SKU public IP address which allows for subscription transfers so attempting to constantly PING the IP address during the move will show that no replies are missed:
You will need to select the checkbox indicating the following to proceed with the move:
I understand that tools and scripts associated with moved resources will not work until I update them to use new resource IDs
Step #7 – Move Recovery Services Vault to destination subscription
If the Recovery Services Vault of the virtual machine was not moved with the VM, proceed to move it into the destination subscription:
Step #8 – Enable backups for virtual machine
If you place the virtual machine into a resource group in the target subscription that has the same as the resource group in the source subscription then you can simply click on the Resume backup button to re-enable backups with the same backup item object. If the migrated VM is in a resource group that is a different name then a new backup item will be configured.
Note that if you’ve placed the VM in the target subscription that has the same as the resource group in the source subscription then attempting to add a VM to be backed up will display:
There are on virtual machines that can be backed up in this vault.
With the resources migrated to the new subscription, remember to update any applications and services that reference the resource ID as the URL will have been changed. The following lists the original and the updated resource IDs of the moved VM:
Virtual Machine:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/virtualMachines/Test-VM
/subscriptions/22d77ef8-89c3-40c2-8730-e786114770a0/resourceGroups/RG-VMs/providers/Microsoft.Compute/virtualMachines/Test-VM
VM NIC:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/networkInterfaces/Test-VM-nic-9190f27cc6cc40a3bede740db168b0d3
/subscriptions/22d77ef8-89c3-40c2-8730-e786114770a0/resourceGroups/RG-VMs/providers/Microsoft.Network/networkInterfaces/Test-VM-nic-9190f27cc6cc40a3bede740db168b0d3
VM Public IP with Basic SKU (basic SKU IP addresses can be moved but standard cannot):
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/publicIPAddresses/Test-VM-pip-90462821bc164f12a624afccf204b93c
/subscriptions/22d77ef8-89c3-40c2-8730-e786114770a0/resourceGroups/RG-VMs/providers/Microsoft.Network/publicIPAddresses/Test-VM-pip-90462821bc164f12a624afccf204b93c
VNet:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet
/subscriptions/22d77ef8-89c3-40c2-8730-e786114770a0/resourceGroups/RG-VMs/providers/Microsoft.Network/virtualNetworks/Test-VNet
VM Disk:
/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611
/subscriptions/22d77ef8-89c3-40c2-8730-e786114770a0/resourceGroups/RG-VMs/providers/Microsoft.Compute/disks/testvm-osdisk-20210424-103611
Resource tags will be retained across subscriptions moves:
Source Subscription:
Destination Subscription:
Other Resource Moves
The following are a few other resources not related to virtual machines that I would like to add to this post.
Managed Identities
As per the documentation:
Move operation support for resources - Microsoft.ManagedIdentity
Cross subscription moves of Managed Identities are not supported. Attempting to move a managed identity will fail with the following message:
{"code":"ResourceMoveProviderValidationFailed","message":"Resource move validation failed. Please see details. Diagnostic information: timestamp '20210426T221659Z', subscription id 'b0957dbf-1cad-4e5e-9360-341ccc197953', tracking id 'c5bcd133-aea2-4481-8b4d-b59a94844056', request correlation id '1e7ed70c-1ba5-458e-a59f-f4fe15179740'.","details":[{"code":"BadRequest","target":"Microsoft.ManagedIdentity/userAssignedIdentities","message":"The identity type is invalid for the requested operation."}]}
----------------------------------------------------------------------------------------------------------------------------
Alert Rules
As per the documentation, Alert rules (microsoft.insights/metricalerts) cannot be move across subscriptions:
{"code":"ResourceMoveValidationFailed","message":"The resource batch move request has '2' validation errors. Diagnostic information: timestamp '20210430T153645Z', tracking Id is '471b94fd-ee3a-4198-84bf-8960aa1ef13e'."},{"code":"ResourceMoveNotSupported","target":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/microsoft.insights/metricalerts/Alert CPU Consumed","message":"Resource move is not supported for resource types 'microsoft.insights/metricalerts'."}]}
One of the ways to migrate the alert rules over is to redeploy alerts to target subscription from source subscription using the Deploy option:
----------------------------------------------------------------------------------------------------------------------------
Private Endpoints
Cross subscription migration for Private Endpoints are not supported and the following validation error would be presented if an attempt to move it is made:
{"code":"ResourceMoveProviderValidationFailed","message":"Resource move validation failed. Please see details. Diagnostic information: timestamp '20210426T225744Z', subscription id 'b0957dbf-1cad-4e5e-9360-341ccc197953', tracking id '63d9dac6-70bb-41ce-ac36-b354700f810a', request correlation id '9f8792df-9c8d-4a95-b63a-c7def04b1b84'.","details":[{"code":"CannotMoveResource","target":"Microsoft.Network/privateEndpoints","message":"Cannot move one or more resources in the request. Please check details for information about each resource.","details":[{"code":"MoveNotSupported","message":"Move for resource type privateEndpoints is not supported. Move request contains resource /subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/privateEndpoints/PrivateEndpoint of that type."}]}]}
The same will be presented for any VNets with a Private Endpoint attached to it:
{"code":"ResourceMoveProviderValidationFailed","message":"Resource move validation failed. Please see details. Diagnostic information: timestamp '20210426T230301Z', subscription id 'b0957dbf-1cad-4e5e-9360-341ccc197953', tracking id 'e507a53e-7793-4c3c-8dda-68829f8988ed', request correlation id '9d032a97-e55d-41e7-9688-bc6085437836'.","details":[{"code":"MissingMoveDependentResources","target":"Microsoft.Network/virtualNetworks","message":"The move resources request does not contain all the dependent resources. Please check details for missing resource Ids.","details":[{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/networkInterfaces/PrivateEndpoint.nic.c35a3d15-6f0e-4f7a-bf2a-caa7297a2152"},{"code":"0","message":"/subscriptions/b0957dbf-1cad-4e5e-9360-341ccc197953/resourceGroups/RG-VMs/providers/Microsoft.Network/privateEndpoints/PrivateEndpoint"}]}]}
Hope this helps anyone who may be looking for more information about what the migration of VMs from one subscription to another would look like.
1 comment:
Terence, Thank you for the detailed step; it helped me understand the details behind the scenes.
Post a Comment