Pages

Monday, May 31, 2021

Configuring App-only authentication with certificates for unattended scripts with EXO V2 module

Those who have worked with Exchange Online PowerShell for a while will know how much a challenge it was to move from basic to modern authentication with MFA for unattended scripts that are scheduled to run non-interactively. The introduction and enforcement of modern authentication and MFA meant attempting to run unattended scripts would no longer work because it would require an administrator to interactively enter credentials and authenticate with the 2nd authentication. As such, many administrators continued to use basic authentication or modern authentication with accounts without MFA enforced to work around the issue, which leads to vulnerability issues. Microsoft originally had plans to disable basic authentication in 2020 but delayed it indefinitely.

Given the challenge as described above, I was extremely excited when Microsoft released the EXO V2 2.0.3 module for public preview in July 2020 where it introduced certificate based authentication. This meant it was now possible to connect to Exchange Online with unattended scripts that no longer passed a username and password, and therefore eliminating the issue where MFA is required upon authentication. I haven’t been working with Exchange Online for sometime due to my new role but was asked by an ex-colleague about setting up unattended scripts so I took the opportunity to capture the process while demonstrating the setup, which I will now use for this blog.

The Scenario

An organization wants to use a PowerShell script to export Office 365 audit logs of users’ events from Exchange Online, SharePoint Online, OneDrive for Business, Azure Active Directory, Microsoft Teams, Power BI, and other Microsoft 365 services with the Audit Log search feature in the following two Microsoft 365 consoles:

Office 365 Security & Compliance
https://protection.office.com/unifiedauditlog

Microsoft 365 compliance
https://compliance.microsoft.com/auditlogsearch

The log will then get emailed for review at the end of the month with a scheduled task.

The Challenge

As this will be an automated process that runs at the end of each month, attempting to setup a PowerShell script that requires using Connect-ExchangeOnline with basic or modern authentication means a username and password will be used with an account without MFA.

The Solution

With the release of EXO V2 2.0.3, unattended scripts (automation) scenarios can now authenticate using Azure AD applications and self-signed certificates.

How does it work?

The EXO V2 module uses the Active Directory Authentication Library to fetch an app-only token using the application Id, tenant Id (organization), and certificate thumbprint. The application object provisioned inside Azure AD has a Directory Role assigned to it, which is returned in the access token. Exchange Online configures the session RBAC using the directory role information that's available in the token.

image

The Tools

To accomplish the task above, we will require the following components:

  1. EXO V2 2.0.3 or higher module
  2. PowerShell Version 7 and higher
  3. Self-signed certificate
  4. PowerShell Script using the Search-UnifiedAuditLog (https://docs.microsoft.com/en-us/powershell/module/exchange/search-unifiedauditlog?view=exchange-ps)

This post will focus on setting up the components required for certificate based authentication but for those who are interested in #4, I’ve written another blog post and will provide the following link to that post:

Script to export audit logs for the current month from Office 365 using Search-UnifiedAuditLog
http://terenceluk.blogspot.com/2021/05/script-to-export-audit-logs-for-current.html

Official Microsoft Documentation

As always, I’d like to provide the official Microsoft documentation and other useful reference documents here:

App-only authentication for unattended scripts in the EXO V2 module
https://docs.microsoft.com/en-us/powershell/exchange/app-only-auth-powershell-v2?view=exchange-ps

Modern Auth and Unattended Scripts in Exchange Online PowerShell V2
https://techcommunity.microsoft.com/t5/exchange-team-blog/modern-auth-and-unattended-scripts-in-exchange-online-powershell/ba-p/1497387

About the Exchange Online PowerShell V2 module
https://docs.microsoft.com/en-us/powershell/exchange/exchange-online-powershell-v2?view=exchange-ps

Basic Authentication and Exchange Online – July Update
https://techcommunity.microsoft.com/t5/exchange-team-blog/basic-authentication-and-exchange-online-july-update/ba-p/1530163

UPDATE: Exchange Online deprecating Basic Authentication (Basic Auth)
https://docs.microsoft.com/en-us/lifecycle/announcements/exchange-online-basic-auth-deprecated

Step #1 – Required Module and PowerShell

Begin by obtaining the required PowerShell and EXO V2 modules as using the incorrect version of say, PowerShell, will cause the certificate based authentication to fail. Download and install the latest versions of the following:

ExchangeOnlineManagement 2.0.5 (the latest at the time of this writing)
https://www.powershellgallery.com/packages/ExchangeOnlineManagement

v7.1.3 Release of PowerShell (the latest at the time of this writing)
https://github.com/PowerShell/PowerShell/releases/tag/v7.1.3

While not a requirement, I highly recommend using Visual Studio Code and the PowerShell extension to write and test PowerShell scripts as the Windows built-in ISE only supports version 5:

Visual Studio Code
https://code.visualstudio.com/download

Using Visual Studio Code for PowerShell Development
https://docs.microsoft.com/en-us/powershell/scripting/dev-cross-plat/vscode/using-vscode?view=powershell-7.1

image

Step #2 – Register an application in Azure AD

In order to authenticate with the EXO V2 module’s Connect-ExchangeOnline with a certificate, you must register the an application in Azure AD, which will represent the unattended script.

Begin by logging into https://portal.azure.com, navigate to Azure Active Directory > App registrations:

image

Click on the New Registration button to register a new app that will represent the identity of the unattended script:

image

Provide a name for the application and select the appropriate account types option. For the purpose of this demonstration, we’re limiting to the single tenant so the following is selected:

Accounts in this organizational directory only (<YourOrganizationName> only - Single tenant)

The Redirect URI (optional) field is not required for what we’re trying to accomplish so leave Web as the selection and the URI empty.

image

The newly created registration of the application should now be displayed:

image

With the application created, we’ll need to assign the appropriate permissions to the application that will represent our unattended script. Proceed to navigate into the configuration of the registered application:

image

----------------------------------------------------------------------------------------------------------------------------

There are two ways to configure the appropriate permissions:

Option #1 – Use the Manifest configuration

This is the easiest as you simply edit the manifest properties, which will configure the permissions and remove the default Microsoft Graph > User.Read permissions as shown here in the API permissions:

image

Navigate to the Manifest configuration and locate requiredResourceAccess entry around line 44:

image

"requiredResourceAccess": [

{

"resourceAppId": "00000002-0000-0ff1-ce00-000000000000",

"resourceAccess": [

{

"id": "dc50a0fb-09a3-484d-be87-e023b12c6440",

"type": "Role"

}

]

}

],

image

Click Save to apply the changes:

image

Navigate to API permissions should now display the Exchange.ManageAsApp permissions configured:

image

Proceed to grant admin consent for the configured permission:

image

image

image

Option #2 – Use the API permissions configuration

The second option is to use the API permissions to manually add the preproperate permissions for the app:

image

Search for Office 365 Exchange Online:

image

Select Application permissions:

image

Under Exchange (1), select Exchange.ManageAsApp:

image

Grant admin consent for the Exchange.ManageAsApp permissions that was just assigned:

image

image

Remove the Microsoft Graph permissions:

image

image

----------------------------------------------------------------------------------------------------------------------------

Step #3 – Generate a self-signed certificate for the application that will be authenticating

The next step is to generate a self-signed X.509 certificate that the application will use to authenticate against Azure AD to request the app-only access token. It is also possible to use an internal or public PKI infrastructure for the certificate but this demonstration will use a self-signed certificate that can be generated locally on a Windows Server with PowerShell version 7.

Note that next Generation (CNG) certificates are not supported for app-only authentication with Exchange. CNG certificates are created by default in modern Windows versions. You must use a certificate from a CSP key provider.

To generate a self-signed certificate for authentication, log onto any Windows Server or desktop with PowerShell version 7 or newer and execute the following:

# Create certificate
$mycert = New-SelfSignedCertificate -DnsName "contoso.org" -CertStoreLocation "cert:\LocalMachine\My" -NotAfter (Get-Date).AddYears(1) -KeySpec KeyExchange

image

A certificate with the private key will be created in the local computer store:

image

The certificate with the private key located on the computer will be used to authenticate the identity of an unattended script. A corresponding certificate with the public key (without the private key) will need to be attached to the Azure AD application. Execute the following cmdlet to create a .cer file with the public key:

# Export certificate to .cer file
$mycert | Export-Certificate -FilePath mycert.cer

image

image

If the intention is to execute Connect-ExchangeOnline from the server where this self-signed certificate is generated then we can simply reference it with its thumbprint when we authenticate. If the script will reside on another server that may not be Windows or have the certificate imported into the local computer store then it is possible to export the certificate with the private key to a .pfx file, which can then be used to authenticate. Scenarios such as having an App Service that needs to authenticate against Azure AD can have the .pfx file stored in the Azure Key Vault and retrieved during the authentication process. Use the following cmdlet to export the certificate to a pfx file:

# Export certificate to .pfx file
$mycert | Export-PfxCertificate -FilePath mycert.pfx -Password $(ConvertTo-SecureString -String "P@ssw0Rd1234" -AsPlainText -Force)

Step #4 – Attach the self-signed certificate to the Azure AD application

The next step is to upload the certificate with the public key (without the private key) to the Azure AD application. Proceed to navigate to the App registrations configuration in the Azure portal and click into the application:

image

Navigate to Certificates & secrets and click on the Upload certificate button:

image

Upload the .cer export of the certificate:

image

The uploaded certificate will be displayed:

image

Step #5 – Assign the required Azure AD roles to the application

The last step for the configuration is the RBAC roles with the required permissions for the registered application so it is able to execute the required cmdlets with the EXO V2 module because authenticating with the certificate will not be in the context of a user. Not all of the Azure AD roles are currently supported but the following ones are:

  • Global administrator
  • Compliance administrator
  • Security reader
  • Security administrator
  • Helpdesk administrator
  • Exchange administrator
  • Global Reader

Begin by navigating to Azure Active Directory > Roles and administrators, search for the Exchange administrator role and open the properties:

image

Click on Add assignments:

image

Search for the app registration we created earlier to grant the service principal permissions:

image

Note how the registered app is now assigned Exchange administrator permissions:

image

Step #6 – Authenticate against Azure AD with Connect-ExchangeOnline using the certificate

We can now use the certificate to authenticate against Azure AD with the Connect-ExchangeOnline cmdlet. Proceed to obtain the Application (client) ID from the Overview properties of the registered application:

image

And the thumbprint of the certificate from the Certificates & secrets of the registered application or the certificate on the Windows operating system’s local computer store:

image

image

With the variable properties above, the following cmdlet can be used to authenticate.

Authenticating with a certificate stored on the Windows server or desktop’s Local Computer > Personal Certificates store

Ensure that the certificate is located in the appropriate store:

image

Connect-ExchangeOnline -CertificateThumbPrint "3D057B3299A75B824F326F1A8A64262F12C60958" -AppID "b6925809-be8c-441b-8915-1bbcc2f2b6fc" -Organization "contoso.onmicrosoft.com"

image

Authenticating with an exported PFX file

Alternatively, you can also use an exported PFX to connect via the following cmdlet:

Connect-ExchangeOnline -CertificateFilePath "C:\scripts\mycert.pfx" -CertificatePassword (ConvertTo-SecureString -String "<MyPassword>" -AsPlainText -Force) -AppID "36ee4c6c-0812-40a2-b820-b22ebd02bce3" -Organization "contoso.onmicrosoft.com"

Authenticating with a certificate object (e.g. retrieved from Azure Key Vault)

The last method to provide a certificate is to use a certificate object via the following cmdlet:

Connect-ExchangeOnline -Certificate <%X509Certificate2 Object%> -AppID "36ee4c6c-0812-40a2-b820-b22ebd02bce3" -Organization "contoso.onmicrosoft.com"

When the Certificate parameter is used, the certificate does not need to be installed on the computer where the command is executed. This parameter is applicable for scenarios where the certificate object is stored remotely and fetched at runtime during script execution. An example of this is when the certificate is stored in the Azure Key Vault.

6 comments:

Anand Sunka said...

Hi Terence,

Thanks for this article which is saved my time and given the detailed explanation.
Really appreciated.

When I was tried to connect EXO by using below command, it's throwing the below error:
Connect-ExchangeOnline -CertificateThumbPrint "MyCertThumbprint" -AppID "b6925809-be8c-441b-8915-1bbcc2f2b6fc" -Organization "mydomain.onmicrosoft.com"

Error as : New-ExoPSSession : Create Powershell Session is failed using OAuth
At C:\Program
Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\2.0.5\netFramework\ExchangeOnlineManagement.psm1:475 char:30
+ ... PSSession = New-ExoPSSession -ExchangeEnvironmentName $ExchangeEnviro ...
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ CategoryInfo : NotSpecified: (:) [New-ExoPSSession], Exception
+ FullyQualifiedErrorId : System.Exception,Microsoft.Exchange.Management.ExoPowershellSnapin.NewExoPSSession


FYI in my laptop BASIC AUTHENTICATION IS DISABLED.

But when I enabled the BASIC AUTHENTICATION above commands runs fine.
My requirement is to automate the custom scripts by using this command which is not meeting my requirement. My company has GPO which is forcefully disabled the BASIC AUth but am doing manually by enabling the BASIC Auth whenever I want to run the script manually & after sometime it's get disabled.



Please can you shed some light on it.

Regards
Anand Sunka

Adam said...

Anand,
Some of the cmdlets in the current general version of exo module v2 (2.0.5) still rely on basic auth. Install the latest prerelease version from PSGallery 2.0.6 and this should be resolved.
Regards
Adam

Anonymous said...

Hi there,

For the cert, we have a certificate authority. I wonder, do you have any experience with generating a cert through a CA for this? I tried to follow your instructions to create a self signed cert & mimic it by creating a server authentication request, download the cer file from the CA, export a pfx with key exportable (with just a password no users selected & with just users selected) but can't seem to get this to work.

Unknown said...

Hi Terence,

I already followed other descriptions to tackle this problem, but couldn't get it done. Your straightforward approach helped me to get through it. Thanks a lot for this brilliant blog post, I really appreciate it!

Best regards
Jens

Melvin said...

Hello dears,

I have a problem with step number 5 because azure has changed the screen, does anyone know how this azure can work on the last azure page?

Melvin said...

Como segunda pregunta, tengo un error de acceso restringido cuando ejecuto el comando y no se a que se debe .

PS C:\WINDOWS\system32> Connect-ExchangeOnline -CertificateThumbPrint "R39DC6340648BAF741A61179A077D7B7B98B2783" -AppID "dv861cc4-5984-4d56-b60c-3e79405841c4" -Organization "ins-cr.com"
UnAuthorized
At C:\Program Files\WindowsPowerShell\Modules\ExchangeOnlineManagement\3.1.0\netFramework\ExchangeOnlineManagement.psm1:733 char:21
+ throw $_.Exception;
+ ~~~~~~~~~~~~~~~~~~
+ CategoryInfo : OperationStopped: (:) [], UnauthorizedAccessException
+ FullyQualifiedErrorId : UnAuthorized