Managing VM configuration with Azure native tools

This post was written for the Azure Spring Clean 2024 event. Visit the official website to see the schedule and links to great content.

Introduction

It is not a secret that majority of enterprises have been betting on Active Directory, Group Policies, and eventually System Center Configuration Manager to keep control over the configuration of their Windows-based IT assets. These are proven technologies with many features, allowing to apply security baselines and tune the configuration in a scalable way: making the configuration objects (GPOs) 'stackable' and apply it to any domain member in the domain (forest).

There are, however, some important disadvantages too:

• single platform - you are only targeting Windows servers and clients, requiring you to treat your Linux machines differently and use different tools like Ansible.

• GUI-based management - making a transition from ClickOps to 'Configuration as Code' hard.

• dependency on 'legacy' identity provider - your Active Directory.

Now, when many organisations are migrating to the cloud, they are asking themselves questions like:

• Should I keep using the same tools to manage my hybrid estate?

• Are there better tools I could adopt and embrace modern engineering practices?

• Are these tools mature, scalable, and flexible enough, so they can cover my needs?

• How painful will this adoption be? Do I need to learn a ton of new things or reuse something I already know?

Requirements

I'm sure you agree with me that important decisions tend to take some time and effort and require clarity in defining 'what you want'. This is of course very individual, so your list might look differently than mine. I am still betting on a high degree of 'corelation' though:

What can Azure offer?

It started with Automation DSC

If you worked with Azure long enough, you have used Azure Automation as a central place to manage VMs updates, inventory, check changes, automate processes with PowerShell and Python runbooks... and use Desired State Configuration to manage their configuration.

This feature was then rebranded to Azure Automation State Configuration (AADSC) but its core capabilities remained the same:

• write, manage, and compile PowerShell DSC configuration for nodes in any cloud or on-premises datacenter. Once uploaded, these configurations had to be compiled to MOF files and hosted on a Pull Server.

• register (onboard) nodes.

• import DSC Resources and assigns configurations to target nodes.

• support both Windows and Linux OSes.

• get a hosted DSC Pull server.

Fair enough, this service is ticking quite a few boxes from our requirements list: cross-platform, hybrid, declarative config, enforcement mode (you could one of these modes: ApplyOnly, ApplyAndMonitor, ApplyAndAutocorrect)... but not all of them! The biggest limitation was that only one configuration can be assigned to a node per automation account, so it doesn't scale that well.

In addition, if you visit the documentation page, you will be greeted with the following info box:

For those of you who have been using AADSC, there is a comprehensive migration guide for moving to the new service. There are compelling reasons for making such switch:

• Increased size limit for configurations to 100 MB

• Advanced reporting through Azure Resource Graph including resource ID and state.

• Manage multiple configurations for the same machine.

• When machines drift from the desired state, you control when remediation occurs.

• Linux and Windows both consume PowerShell-based DSC resources.

Moving on to Machine Configuration

So, let's move on and explore Guest Configuration. Well, kinda... :) You know how Microsoft marketing machine work, right? This service was meanwhile rebranded to Azure Machine Configuration. And for a brief period, it was called Azure Automanage Machine Configuration. Not confusing at all 🤣.

Microsoft describes this service as:

Azure Machine Configuration (formerly called Azure Policy Guest Configuration) provides native capability to audit or configure operating system settings as code, both for machines running in Azure and hybrid Arc-enabled machines.
Configuration resources in Azure are designed as an extension resource. You can imagine each configuration as an extra set of properties for the machine. Configurations can include settings such as:

• Operating system settings

• Application configuration or presence

• Environment settings

Let's dive into the innerworkings by looking at the overview of key components and their relations:

• VM Extension

  • This is a new extension you can either manually deploy to your Azure VMs or you could use 'Deploy prerequisites to enable Guest Configuration policies on virtual machines' built-in policy initiative to enable it at scale. Note: it is different from the DSC extension!

  • The extension downloads applicable machine configuration assignments and the corresponding dependencies.

  • The extension isn't required for Arc-enabled servers because it's included in the Arc Connected Machine agent (CMA).

• System-assigned identity - used to authenticate the machine as it reads and writes to the machine configuration service.

• Machine configuration package - a zip file stored typically in Azure Blob with an HTTPS endpoint, containing a MOF file, dependencies, and metadata.

• Machine configuration assignment (guest assignment)

  • Resource type: Microsoft.GuestConfiguration/guestConfigurationAssignments

  • It binds configuration packages with VMs.

  • It is a child resource to Azure VMs and Arc-enabled servers, so when you are assigning it manually, you need to refer the 'parent resource', for example using Bicep.

  • Ensure that Microsoft.GuestConfiguration RP is registered in subscription you are targeting! It can get registered automatically if your subscription is enrolled in Microsoft Defender for Cloud.

  • Configurations are distinct from policy definitions!

  • You can view existing assignments in the Portal in Guest Assignments or on individual VM (Arc machine) level, there is a Configuration Management blade.

• Custom policy definition and assignment

  • When using Azure Policy engine to dynamically assign machine configuration at scale (e.g., on a Management Group level), you need to craft a new policy definition - with Guest Configuration category - and include metadata to describe a guest assignment.

  • Alternatively, you can also assign configurations by using other Azure services such as Automanage. Be aware, that Automanage service has something called configuration profiles, which is different from machine configurations.

Here is the list of officially supported OS types and versions:

Built-in vs. DIY

It would be hard if we all had to craft our configuration packages ourselves. Luckily, Microsoft is providing a solid collection of built-in policies for Guest Configuration. You can explore them in the Portal or in their raw format in GitHub.

Once you identify a need to build your custom configuration, you won't avoid tapping into PowerShell DSC, so if you haven't worked with this technology before, better buckle up. It will be a steep learning curve. On the positive side, DSC has been maturing over years and has a vibrant and helpful community with a lot of ready-made DSC resources and modules, published mainly in the PowerShell Gallery.

Creation of a DSC configuration script is only one step in the authoring process. The following diagram can give you a full overview:

Fortunately, Microsoft is kind enough to provide good instructions for getting your authoring environment ready (with all prerequisites) and using cmdlets from GuestConfiguration PowerShell module to produce all artifacts all the way to deploying and assigning custom policies.

Ready for the Prime time?

When I think about the original requirements, Machine Configuration 'ticks all the boxes' and it comes with good reporting (ability to query Resource Graph and visualise compliance through Azure Policy compliance view).

I can see some challenges here:

• A migration tool helping organisations to move from GPO to Machine Configuration is missing.

• Building custom configurations requires an authoring environment and deep PowerShell DSC competence.

I can image that many companies will begin with assessing the built-in configurations, applying them in 'AuditOnly' mode, gradually switching to the enforcement mode and later creating custom configurations. I don't predict to see massive adoption in a short time though.

Honorable mention

There is another interesting technology that partly fits into 'configuration management' space: WinGet Configuration.

It primarily focuses on application deployment through WinGet sources, but it can be also used for configuration. The main use case is to bootstrap a machine (server or client) with all the apps included and environment configured. It works great with Microsoft Dev Box or getting a dev environment in general.

In the context of our original requirements, it has a few caveats:

• Since WinGet is a package manager for Windows, it is not cross-platform.

• It isn't easy to apply the configuration at scale.

• It doesn't do drift detection and auto-remediation (as far as I know).