One of the common questions I am asked about accessing Azure services is the concept of service endpoints and/or private endpoints in Azure. To put it simply, Azure is a public cloud service, which means it is inherently designed to allow its services to be publicly accessed through the internet. This often instills quite a bit of fear to security experts (or anyone in IT for that matter) as they decide to move on-premises workloads to the cloud. Microsoft is aware of this potential road block for organizations and thus provides service endpoints and private endpoints to limit public access to Azure services.
In this blog post, I will attempt to provide a summarized explanation of Service Endpoints and Private Endpoints, then demonstrate the configuration. For more in-depth information on these two services, please see the official Microsoft documentation. I would highly recommend reading the documentation if you decide to read through this blog post.
Virtual Network service endpoints
https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-service-endpoints-overview
What is Azure Private Endpoint?
https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-overview
Service Endpoints
Service endpoints allows an administrator to lock down the Azure resource such as a storage account to a VNet (all of its subnets), a specific VNet’s subnet, or a public IP. The locking down via public IP is an easy to grasp concept so we’ll look at how locking down with a VNet’s subnet works. When a VNet’s subnet is selected, the routing table on the subnet is updated to route traffic to the service endpoint before routing to the internet. The traffic from the subnet still accesses the storage account by its public IP address but the traffic will flow through the faster Azure backbone network rather than the internet thus enjoying better performance and arguably better security.
The following are some key items to note about Service Endpoints:
- The resource will continue to have a public IP address (it does not get a private IP address)
- The public IP will continue to be resolved by the DNS provided by Microsoft
- It is not accessible from the on-premise network through, say, a site-to-site VPN because the service endpoint does not actually have a private IP so if an on-premise device needs to access the resource then the on-premise public IP will need to be added to the service endpoint (the on-premise device will use the internet to reach the resource)
- If there is an ExpressRoute connecting an on-premise network to Azure, then the NAT IP addresses can be added to allow access: https://docs.microsoft.com/en-us/azure/virtual-network/virtual-network-service-endpoints-overview#secure-azure-service-access-from-on-premises
The following is a diagram demonstrating the traffic flow of a service endpoint configured for a storage account:
**Note that the device attempting to access the storage account will continue to reach it via its public IP address but the originating IP address seen by the storage account will be the device’s private IP address. This is the important difference to highlight for service endpoints. Also, as the public IP address of the storage account is being used as the destination, any restrictive NSGs applied to the traffic flowing outbound will need to be configured to allow traffic to reach the public IP address of the storage account.
**Note that Service Endpoints is not available for all services. The current services it support are as follows:
Generally available
- Azure Storage (Microsoft.Storage): Generally available in all Azure regions.
- Azure SQL Database (Microsoft.Sql): Generally available in all Azure regions.
- Azure Synapse Analytics (Microsoft.Sql): Generally available in all Azure regions for dedicated SQL pools (formerly SQL DW).
- Azure Database for PostgreSQL server (Microsoft.Sql): Generally available in Azure regions where database service is available.
- Azure Database for MySQL server (Microsoft.Sql): Generally available in Azure regions where database service is available.
- Azure Database for MariaDB (Microsoft.Sql): Generally available in Azure regions where database service is available.
- Azure Cosmos DB (Microsoft.AzureCosmosDB): Generally available in all Azure regions.
- Azure Key Vault (Microsoft.KeyVault): Generally available in all Azure regions.
- Azure Service Bus (Microsoft.ServiceBus): Generally available in all Azure regions.
- Azure Event Hubs (Microsoft.EventHub): Generally available in all Azure regions.
- Azure Data Lake Store Gen 1 (Microsoft.AzureActiveDirectory): Generally available in all Azure regions where ADLS Gen1 is available.
- Azure App Service (Microsoft.Web): Generally available in all Azure regions where App service is available.
- Azure Cognitive Services (Microsoft.CognitiveServices): Generally available in all Azure regions where Cognitive services are available.
Public Preview
- Azure Container Registry (Microsoft.ContainerRegistry): Preview available in limited Azure regions where Azure Container Registry is available.
Private Endpoints
Private Endpoints adds a virtual network interface to the resource that connects to the VNet. The network interface will then have a private IP address and behaves like a device on the network. Having a private IP presence on the VNet will mean that on-premise devices traversing through a site-to-site VPN or express route can now access the resource via its private IP address through the private connection.
The following are some key items to note about Private Endpoints:
- All public IP address access can be blocked because a private IP is available to access the resource
- Azure internal DNS will now resolve the resource’s hostname to its private IP address
- Network Security Groups (NSG) will not be applied to the private endpoint’s network interface
· If there is a desire to block other network resources to the private endpoint, then outbound NSG security groups assigned to the source can be used to block access
The following is a diagram demonstrating the traffic flow of a private endpoint configured for a storage account:
**Note that this example places the endpoint directly into the subnet where the virtual machines are located but it is also possible to place the endpoint into its own subnet within the same VNET as the other subnets that will access the endpoint. It is important to note that subnets within the same VNET as the subnet containing the endpoint will have Azure DNS resolve the internal IP address of the private endpoint. The public IP will be returned when resolving the DNS name outside of the VNet.
As with Service Endpoints, Private Endpoints are only available for a set of services as listed here: https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-overview#private-link-resource
With a brief overview of what a Service Endpoints and Private Endpoints are, let’s have a look at what the configuration would look like.
I’ve configured two storage accounts to demonstrate the Service Endpoint and Private Endpoint.
Configuring a Service Endpoint
The storage account we’ll be using for the service endpoint is named sgserviceendpoint:
I’ve created a container and uploaded a text file:
Without a service endpoint configured, access to the container is available from anywhere on the internet with authentication because of the access level I’ve configured:
I can access the container with Azure Storage Explorer from my home computer over internet and connect to a Blob container with a generated shared access signature:
To create a service endpoint simply navigating to the Networking configuration for the storage account and select Firewalls and virtual networks:
Change Allow access from the configuration All networks to Selected networks, select Add existing virtual network, then in the right blade, add the subnet or subnets that you want to access the storage group via the service endpoint:
Notice that the following information message is displayed:
The following networks don’t have service endpoints enabled for 'Microsoft.Storage'. Enabling access will take up to 15 minutes to complete. After starting this operation, it is safe to leave and return later if you do not wish to wait.
This is because the Microsoft.Storage service needs to be enabled for the service endpoint as we are accessing a storage account. Azure noticed that it is not enabled and therefore will be enable it for you.
Click on the Save button to apply the changes once the subnet has been added but note that saving the configuration will update the routing table on the subnet and may cause a disruption on any connections for the subnet (best to make this change on a scheduled maintenance window):
Now when you navigate to the VNet containing the subnet that was enabled for the service endpoint, you’ll be able to see the configuration added to the Service endpoints setting:
Navigating into the Subnets configuration and clicking on the subnet configured for the service endpoint will also show that Microsoft.Storage is listed:
Access to the container should no longer accessible via the internet as this is what is displayed when I hit the refresh button from my laptop:
This request is not authorized to perform this operation. RequestId:44c96890-501e-0053-310c-254d92000000 Time:2021-03-30T02:31:40.6499686Z
Attempting to use Azure Storage Explorer from a server in the subnet defined in the service endpoint will be continued to be allowed:
You will also notice that attempting to resolve the sgserviceendpoint.blob.core.windows.net address representing the container resolves to a public IP address and not a private IP address:
Configuring a Private Endpoint
The storage account we’ll be using for the private endpoint is named sgprivateendpoint. To create a private endpoint representing this storage account, navigate to the Networking configuration under Settings, click on the Private endpoint connections tab, then Private endpoint:
Select the subscription, resource group, provide a name for the logical private endpoint (what the private endpoint will be named in Azure), and the region it should reside in:
Select the appropriate resource for the Target sub-resource field:
**Note that if you want to provide blob and another resource such as file access then you’ll need to add the additional resource(s) afterwards one at a time.
Proceed to select the appropriate subnet you want to place the private endpoint in:
**Note the Private DNS integration remark below the Networking configuration. We will discuss this in more depth after the initial configuration
Add additional tags if necessary:
Proceed to create the private endpoint:
Clicking into the Private Endpoint resource once it has been created will display the following information:
Clicking into the DNS configuration under Settings will display the FQDN and the private IP address assigned to the private endpoint:
FQDN: sgprivateendpoint.privatelink.blob.core.windows.net
IP address: 10.248.1.7
Navigating into the VNet where the private endpoint was created will display the private endpoint that was just created:
Navigating into the resource group containing the private endpoint will display the private endpoint object as well as a NIC created:
Clicking into the NIC representing the private endpoint will show that the settings are the same as any other NIC adapter:
This private endpoint configuration is depicted in the diagram I included earlier in the blog post and pasted again here:
Private Endpoints and DNS
With the private endpoint created, the next important configuration is to ensure that DNS resolution resolves as it is supposed to in various places. The following are 3 common places you may want to access the private endpoint:
#1 - Within the same subnet where the private endpoint resides:
#2 - Within a subnet residing in the same VNet where the private endpoint resides:
#3 - From an on-premise network connected to Azure via ExpressRoute or VPN:
The URL to access the storage account’s blog service is:
sgprivateendpoint.blob.core.windows.net
Attempting to perform an nslookup from a machine using internet DNS will resolve the public IP along with two aliases where one represented the private endpoint. Attempting to lookup the public or private endpoint alias will return the public IP:
> sgprivateendpoint.blob.core.windows.net
Server: dns.google
Address: 8.8.8.8
Non-authoritative answer:
Name: blob.blz22prdstr18a.store.core.windows.net
Address: 20.60.7.100
Aliases: sgprivateendpoint.blob.core.windows.net
sgprivateendpoint.privatelink.blob.core.windows.net
> sgprivateendpoint.privatelink.blob.core.windows.net
Server: dns.google
Address: 8.8.8.8
Non-authoritative answer:
Name: blob.blz22prdstr18a.store.core.windows.net
Address: 20.60.7.100
Aliases: sgprivateendpoint.privatelink.blob.core.windows.net
Alternatively, if an nslookup is used with an Azure DNS server (168.63.129.16) within the same VNET, the link will return the private link alias with the private IP:
C:\>nslookup
Default Server: UnKnown
Address: ::1
> server 168.63.129.16
Default Server: [168.63.129.16]
Address: 168.63.129.16
> sgprivateendpoint.blob.core.windows.net
Server: [168.63.129.16]
Address: 168.63.129.16
Non-authoritative answer:
Name: sgprivateendpoint.privatelink.blob.core.windows.net
Address: 10.248.1.7
Aliases: sgprivateendpoint.blob.core.windows.net
As most administrators will immediately know, many of the environments in Azure does not use Azure DNS because they would typically use Active Directory domain DNS. To get around this issue, we can simply add the private link as a zone in the internal DNS and a corresponding A record representing the storage account. The following is a demonstration of how this would be configured.
Launch DNS Manager on your DNS server (usually a domain controller) and create a new zone:
Select Primary zone:
Whether to replicate to all the DCs in the forest or domain would be your choice:
Create a zone representing the private link without the storage account name:
Select the desired dynamic update configuration and create the zone:
With the zone configured, create a new A record to represent the storage account (the private link alias):
Add the appropriate private IP:
Now when you initiate a nslookup against the internal Active Directory domain controller, the private IP will be returned:
C:\>nslookup
Default Server: UnKnown
Address: ::1
> server 10.248.1.250
Default Server: [10.248.1.250]
Address: 10.248.1.250
> sgprivateendpoint.blob.core.windows.net
Server: [10.248.1.250]
Address: 10.248.1.250
Non-authoritative answer:
Name: sgprivateendpoint.privatelink.blob.core.windows.net
Address: 10.248.1.7
Aliases: sgprivateendpoint.blob.core.windows.net
Note that this example configured blob storage so if you require, say, file storage access then an additional forward lookup zone will need to be created.
More information about private link DNS configuration can be found here:
Azure Private Endpoint DNS configuration
https://docs.microsoft.com/en-us/azure/private-link/private-endpoint-dns
Preventing Access to Storage Account with Private Endpoint
Unlike a service endpoint, configuring a private endpoint does not prevent access to the storage account so if that is a requirement then you will need to mimic what you would normally do to configure a service endpoint by selecting Selected networks but do not add any virtual networks or IP ranges:
Hope this helps anyone who may be looking for information on service endpoints and private endpoints. Note that service endpoints have no extra cost while private endpoints do (https://azure.microsoft.com/en-ca/pricing/details/private-link/)
No comments:
Post a Comment