Troubleshooting traffic blocked by Azure Front Door WAF Policy in Prevention mode

I recently had to troubleshoot an issue with an Azure Front Door WAF policy we had just changed from Detection to Prevention and thought I’d share some steps I used to troubleshoot and have this blog post for me to reference in the future as I’m bound to forget some of the steps.

Before you begin, have a look at the following Microsoft documentation to understand how to use Log Analytics to review WAF logs in a Log Analytics workspace:

Use Log Analytics to examine Application Gateway Web Application Firewall (WAF) Logs
https://learn.microsoft.com/en-us/azure/application-gateway/log-analytics

With that, let’s have a look at the process I went through.

Scenario

The developer notified me after the Front Door WAF Policy was switched from Prevention from Detection:

To troubleshoot, we used a browser’s Developer Tools to review the Network traffic and noticed the following error:

Request Method: Post
Status Code: 403
Referrer Policy: strict-origin-when-cross-origin

The above error was thrown when we opened up the website and navigated to sit.domain.com that presented a login page, entered our credentials into the form, clicked login, which would then send us to sitapp.domain.com. Clicking on the login button did not do anything as we would receive the 403 error as shown above.

Below is a screenshot of the log showing the origin and referrer URL, which is the login site we tried to log into:

Troubleshooting 

The first thought I had was that the referrer policy did not allow sit.domain.com to send a successful login to sitapp.domain.com so what we needed to do was whitelist the domain but this actually the wasn’t the case when we look into the logs.

There are no custom rules configured so we needed to determine which one of the managed rules has blocked the traffic:

In order to troubleshoot, Diagnostics must be turned on to capture the WebApplicationFirewall logs:

Verify that you are sending the FrontDoor WebApplicationFirewall Log to a log analytics workspace as shown here:

Once logging is turned on, wait 10 to 15 minutes for the log capturing to begin then proceed to replicate the issue. Note that you should see AzureDiagnostics displayed when logging has come into effect:

We should see entries with the following query when the WAF logs are captured:

AzureDiagnostics
| where Category == "FrontDoorWebApplicationFirewallLog"

Use the following query to retrieve all the entries representing blocks by a WAF rule:

AzureDiagnostics
| where action_s == "Block"

**Note that Block is case sensitive so “block” will not return any results.

Since we know have the information we collected from the browser network trace, we can use the information in there to look for the entry representing the block we’re experiencing. The values I usually use are requestUri_s or host_s but note that when using the requestUri_s, the URL will need to have a colon and port number after the domain as shown in the screenshot below:

https://sitapp.domain.com:443/webapi/api/account/login

Here is an example of adding the requestUri_s in the query:

AzureDiagnostics

| where action_s == "Block" and requestUri_s == https://sitapp.domain.com:443/webapi/api/account/login

With the entry located, review the ruleName_s value and note the ending number:

In this example, the rule that is blocking the traffic is 949110. However, this may not be locatable in the Managed rules of the WAF policy:

Although this is the rule that results in the traffic being blocked, it could be triggered by another rule. To determine the other related rules, we can locate the trackingReference_s value to look up the other logs related to this:

Use the following query to retrieve the entries:

AzureDiagnostics

| where trackingReference_s contains "0viJPYwAAAAD2qxqM/l3+QJ3xvYLEO0hYQ0hJMzBFREdFMDUxMwA0MzRlN2MwNC02M2NmLTRjMjMtYjFhOS00NDNjYmViZTJmNTg="

Expand the results and you should see another rule with the number 200002 that contains the same trackingReference_s as shown in the screenshot below:

Scrolling further down in the log will display more details about the reason why the traffic is blocked and in this case they are:

details_msg_s: Failed to parse request body.

details_data_s: %{reqbody_error_msg]

Searching for the 200002 number in the Managed rules will display the following rule with the description Failed to parse request body:

Unsure of what to make of the details, a case was opened with Microsoft but the engineer was unable to find any additional information for this error but was able to suggest that the following is what was happening:

When a request to sign into the https://sit.domain.com/ portal, the webpage directing the traffic to sitapp.domain.com is sending some strange or malformed data in the request body which causes the Front Door’s WAF policy to think there may be an attack. Their suggestion was to check the request body for any formatting error that would be triggering front door.

Workaround

I’ve requested the developer to look into this and for the time being changed the action for rule 200002 to log rather than block on anomaly so the portal login would start working:

Not the best solution but serves as a workaround. Hope this helps anyone looking for information on how to troubleshoot Azure Front Door WAF policy prevention blocks.