Pages

Wednesday, February 23, 2022

Using Azure Change Tracking and Inventory to monitor Windows Services

In my previous post:

Monitor and Alerting for an Azure Virtual Machine with Azure Monitor
Terence Luk: Monitor and Alerting for an Azure Virtual Machine with Azure Monitor

I demonstrated how to set up Log Analytics to monitor the event log for a system event ID 7031 with a specific string that represents a Windows Service we wanted to monitor for and detect if it stops. While this method is certainly a viable option, it isn’t very straight forward if you’re not familiar with Windows and do not know what system events are triggered when a service stops. Case in point, when a service is abruptly terminated, an event ID 7031 error is logged but if the service is gracefully stopped, the event logged will be a 7036 information. Having to capture all types of events with a query leaves a lot of room for error so I would like to demonstrate a different method for monitoring Windows or Linux services.

The Azure feature I typically use to monitor services from within a virtual machine is Change Tracking and Inventory in Azure Automation. This feature tracks changes in virtual machines hosted in Azure, on-premises, and other cloud environments. Items that are tracked by Change Tracking and Inventory include:

  • Windows software
  • Linux software (packages)
  • Windows and Linux files
  • Windows registry keys
  • Windows services
  • Linux daemons

Change Tracking and Inventory overview

https://docs.microsoft.com/en-us/azure/automation/change-tracking/overview

I find this feature extremely powerful and opens up many monitoring opportunities for all sorts of use cases. For the purpose of this example, we’ll use it to monitor Windows service status.

Creating an Automation Account

The Change Tracking and Inventory is a feature of Azure Automation and you’ll therefore need to create an Automation account that is linked to a Log Analytics workspace. Begin by navigate to Automation Accounts:

image

Then create an automation account:

image

Enabling Change Tracking for the Automation Account

Navigate to Configuration Management > Change Tracking, select a supported Log Analytics workspace, then click Enable:

image

The following console will be displayed once the deployment has successfully completed:

image

Adding VMs for Change Tracking

With Change Tracking ready in the Automation Account, proceed to add the VMs:

image

Select the virtual machine(s) you would like to enable Change Tracking and then click Enable:

image

With the virtual machine added, proceed to adjust the settings by clicking on Edit Settings:

image

Navigate to Windows Services and note how the frequency is set to 30 minutes:

image

This is likely not frequent enough and for the purpose of this example, we’ll use the lowest frequency of 10 seconds to collect the Windows Services changes:

image

With the change tracking configured, proceed to stop the service you intend to test with:

image

Refresh the Change tracking console and you should see the Windows Services change logged:

image

Note the details for the Advanced Monitoring Agent service we stopped:

image

Proceed to click on the Log Analytics button:

image

The query window will automatically execute the ConfigurationChange query without any other requirements, which will provide all results for any configuration change. We’ll refine it to only list the service we want to track:

ConfigurationData
| where SvcName =~ "Advanced Monitoring Agent"
| project SvcName, SvcDisplayName, SvcState, TimeGenerated
| where SvcState != "Running"

image

Now that we have a query to search for a specific service, we can create a new alert by clicking the New alert rule button:

image

Proceed to configure the Condition settings:

image

Update the Threshold value to 0 and Frequency of evaluation to 1 minute to capture any service status that is not “Running”:

image

Select an action group for the notification:

image

Fill in the details for the rule:

image

Complete creating the rule:

image

As with all rules, it may take a bit of time before it shows up in the console:

image

Note that although the rule was created within the Automation Account, it is actually configured and associated with the Log Analytics Workspace of the Automation Account:

image

Proceed to test stopping the monitored service and you should see an email notification similar to the one below:

image

Hope this provides a good overview of how to use Change Tracking and Inventory to monitor Windows Services. What’s great about this feature is that it allows you to track other changes such as file and registry, which opens up so many possibilities for monitoring.

No comments: