# BenchPress: a must-have tool to test your Bicep muscles *This post was created for the*[*Azure Back to School 2023*](https://azurebacktoschool.github.io/)*event.There is a lot of great content published every day, I would encourage you to check it out.* Testing is hard. Testing your Bicep code, doubly so. Until now. At least that's what [BenchPress](https://github.com/Azure/benchpress), a new Azure testing framework promises.
đź’ˇ
This article was originally published on my Wordpress blog: BenchPress: a must-have tool to test your Bicep muscles – David Pazdera (pazdedav.blog)
Let's explore together this interesting open-source project to understand: * how it works * how you can test locally * how it can be integrated in our CI/CD pipeline * and how it can complement existing validation options like linting, pre-flight and what-if deployment for Bicep templates and modules # How it works The central idea behind BenchPress is about adding a **verification after you deploy your Bicep template** that would confirm that the resources you declared exist (were deployed successfully) and assert if the actual values match with expected ones. > Compared to other tools and linters that would inspect your code and run some checks before you hit deploy, BenchPress needs some Azure environment to run its validation! Apart from adding such verification to your regular flow when working with various environments, you could use it with ephemeral environments, where you could **deploy, validate, and destroy in a single flow**. This could work great when building a library of Bicep modules, you want to test and validate before publishing to your private registry. BenchPress is technically an **extension of**[**Pester**](https://pester.dev/docs/quick-start), an exceedingly popular testing framework for PowerShell, and it has several [prerequisites](https://github.com/Azure/benchpress/blob/main/docs/getting_started.md#requirements) to function properly: * PowerShell 7+ * Az PowerShell module * Pester module * Bicep CLI Since it uses Pester as an underlying engine, it won't surprise you that BenchPress itself is published as a module in the [PowerShell Gallery](https://www.powershellgallery.com/packages/Az.InfrastructureTesting/0.1) and it can be easily downloaded and enabled in your local machine using the following commands: ```powershell Install-Module -Name Az.InfrastructureTesting Import-Module -Name Az.InfrastructureTesting ``` Apart from the obvious need to have an Azure subscription (account), you will also need to create a Service Principal and inject its key properties to your environment variables. In other words, BenchPress doesn't support the signed-in user credentials yet (only Service Principal and Managed Identities). What BenchPress adds on top of Pester is a collection of cmdlets that simplify your assertions, for example: ```powershell Confirm-AzBPResourceGroup -ResourceGroupName $rgName | Should -BeSuccessful Confirm-AzBPResourceGroup -ResourceGroupName $rgName | Should -BeInLocation $location ``` *Of course, you could craft your own tests with "pure" Pester syntax and Azure PowerShell modules instead, but it is easier with BenchPress.* ## A small hiccup If you [search for "benchpress" in the PowerShell Gallery](https://www.powershellgallery.com/packages?q=Benchpress), you will get the following result: ![](https://pazdedav.files.wordpress.com/2023/09/image.png?w=1024 align="left") **Why am I pointing this out?** When I was experimenting with BenchPress and the library of [examples](https://github.com/Azure/benchpress/tree/main/examples) a couple of months ago, it worked without problems. When I was re-running the same tests while authoring this article, it was throwing this error: ```plaintext ParameterBindingException: Parameter set cannot be resolved using the specified named parameters. One or more parameters issued cannot be used together or an insufficient number of parameters were provided. ``` I tried to figure out what was causing it but what helped me as a workaround was to import "the other BenchPress" module using the `Import-Module BenchPress.Azure` command, rather than `Az.InfrastructureTesting`. # How to test locally ### Install tools Before you can test anything, you need to make sure your box has all the dependencies installed. Your setup will vary depending on what OS you're using but for Windows, you could use `winget` or Chocolatey: ```bash winget install --id Microsoft.Powershell --source winget winget install --id Microsoft.Bicep --source winget winget install --id Microsoft.VisualStudioCode --source winget ``` Then you would install the following PowerShell modules and VS Code extensions: ```powershell Install-Module -Name Az Install-Module -Name Pester Install-Module -Name Az.InfrastructureTesting code --install-extension ms-vscode.powershell code --install-extension ms-azuretools.vscode-bicep code --install-extension pspester.pester-test ``` \> TIP: It would make sense to create a [Dev Container](https://code.visualstudio.com/docs/devcontainers/create-dev-container) definition for BenchPress, so you don’t need to install any of these tools directly. ### Workload identity and environment Remember that you need to create a new Service Principal or re-use an existing one for BenchPress to work. This script will create one for you and populate environment variables (Check the [Getting started](https://github.com/Azure/benchpress/blob/main/docs/getting_started.md) guide for more details): ```powershell Connect-AzAccount $sp = New-AzADServicePrincipal -DisplayName BenchPressSPN New-AzRoleAssignment -ApplicationId $sp.Id -RoleDefinitionName 'Contributor' $encryptedPass = $sp.PasswordCredentials.SecretText | ConvertTo-SecureString -AsPlainText -Force | ConvertFrom-SecureString $Env:AZ_APPLICATION_ID= $sp.Id $Env:AZ_TENANT_ID= (Get-AzSubscription).TenantId $Env:AZ_SUBSCRIPTION_ID= (Get-AzSubscription).id $Env:AZ_ENCRYPTED_PASSWORD= $encryptedPass ``` \> Note the special form in which the SPN password needs to be stored (`$encryptedPass` variable). ### First local test Now when your dev box has everything it needs, you could begin with the simplest available example: [Resource Group](https://github.com/Azure/benchpress/tree/main/examples/ResourceGroup). *You can either clone the entire BenchPress repo or copy specific examples you'd like to test.* The instructions for the Resource Group can be found in [ResourceGroup\_Example.md](https://github.com/Azure/benchpress/blob/main/examples/ResourceGroup/ResourceGroup_Example.md): Run the deployment with this `resourceGroup.bicep` template: ```plaintext targetScope = 'subscription' param name string = 'rg${take(uniqueString(subscription().id), 5)}' param location string = deployment().location // https://docs.microsoft.com/en-us/azure/templates/microsoft.resources/resourcegroups?tabs=bicep resource resourceGroup 'Microsoft.Resources/resourceGroups@2022-09-01' = { name: name location: location } output name string = resourceGroup.name output id string = resourceGroup.id ``` Update the `ResourceGroup.Tests.ps1` file with correct values (name and location): ```powershell BeforeAll { Import-Module Az.InfrastructureTesting $Script:rgName = 'rg-test' $Script:noRgName = 'notestrg' $Script:location = 'westus3' } Describe 'Verify Resource Group Exists' { It "Should contain a Resource Group named $rgName - Confirm-AzBPResource" { # arrange $params = @{ ResourceType = "ResourceGroup" ResourceName = $rgName } # act and assert Confirm-AzBPResource @params | Should -BeSuccessful } It "Should contain a Resource Group named $rgName - Confirm-AzBPResource" { # arrange $params = @{ ResourceType = "ResourceGroup" ResourceName = $rgName PropertyKey = 'ResourceGroupName' PropertyValue = $rgName } # act and assert Confirm-AzBPResource @params | Should -BeSuccessful } It "Should contain a Resource Group named $rgName" { Confirm-AzBPResourceGroup -ResourceGroupName $rgName | Should -BeSuccessful } It "Should not contain a Resource Group named $noRgName" { # The '-ErrorAction SilentlyContinue' command suppresses all errors. # In this test, it will suppress the error message when a resource cannot be found. # Remove this field to see all errors. Confirm-AzBPResourceGroup -ResourceGroupName $noRgName -ErrorAction SilentlyContinue | Should -Not -BeSuccessful } It "Should contain a Resource Group named $rgName in $location" { Confirm-AzBPResourceGroup -ResourceGroupName $rgName | Should -BeInLocation $location } } AfterAll { Get-Module Az.InfrastructureTesting | Remove-Module Get-Module BenchPress.Azure | Remove-Module } ``` Run `Invoke-Pester -Path .\ResourceGroup.Tests.ps1` command and ideally get the following output: ```plaintext Tests completed in 952ms Tests Passed: 2, Failed: 0, Skipped: 0 NotRun: 0 ``` This was quite a simple test, I must admit. You should try several examples or even write your own tests from scratch to assess the true potential this tool can provide you. ## VS Code extension If you are a Visual Studio Code user, there is a special treat available. Simply install the [Pester Tests](https://marketplace.visualstudio.com/items?itemName=pspester.pester-test) extension with e.g., `code --install-extension pspester.pester-test` ![](https://pazdedav.files.wordpress.com/2023/09/image-5.png?w=1024 align="left") You will get a nice **side panel** that will show all \*.Tests.ps1 files in your workspace and allow you to run and re-run all tests or just individual ones or debug them if your tests don't do what you expect. ![](https://pazdedav.files.wordpress.com/2023/09/image-1.png?w=1024 align="left") ![](https://pazdedav.files.wordpress.com/2023/09/image-3.png?w=1024 align="left") \> *Note: The extension is in preview, but it worked great for me, and I haven’t encountered any issues.* # Integration with GitHub Actions workflow Although the BenchPress repo doesn't contain any guidance on how to integrate it with various CI/CD tools, since it is powered by Pester, I used the '[Test Results](https://pester.dev/docs/usage/test-results)' article from Pester documentation as a starting point and added what BenchPress required on top. If you want to check my GitHub workflow definition, you can find it here: [https://github.com/pazdedav/bicep-pester-validation](https://github.com/pazdedav/bicep-pester-validation) My demo setup is using several key "ingredients": * a workflow with `workflow_dispatch:` trigger, so I could run it manually and inject a few inputs * a set of variables - `AZ_SUBSCRIPTION_ID, AZ_TENANT_ID, AZ_APPLICATION_ID` - and a secret (`AZ_ENCRYPTED_PASSWORD`) exactly like BenchPress requires. Those are then made available as environment variables The key step in the workflow looks like this: ```yaml - name: BenchPress tests id: pester uses: azure/powershell@v1 with: inlineScript: | Set-PSRepository psgallery -InstallationPolicy trusted Install-Module -Name Pester -RequiredVersion 5.5.0 -Confirm:$false -Force -SkipPublisherCheck Install-Module -Name BenchPress.Azure -RequiredVersion 0.2.1 -Confirm:$false -Force -SkipPublisherCheck Import-Module Pester -Force Import-Module BenchPress.Azure -Force $configuration = [PesterConfiguration]::Default $configuration.TestResult.Enabled = $true $configuration.Output.Verbosity = 'Detailed' $configuration.TestResult.OutputFormat = 'NUnitXml' $configuration.TestResult.OutputPath = 'Test.xml' $configuration.Output.CIFormat = 'GithubActions' $container = New-PesterContainer -Path "./infrastructure/resourceGroup/resourceGroup.Tests.ps1" -Data @{ ResourceGroupName = "${{ env.RESOURCE_GROUP }}"; location = "${{ env.LOCATION }}" } $configuration.Run.Container = $container $configuration.Run.PassThru = $true $result = Invoke-Pester -Configuration $configuration exit $result.FailedCount azPSVersion: "latest" env: RESOURCE_GROUP: ${{ inputs.resourceGroupName }} LOCATION: ${{ inputs.location }} ``` After importing those two PowerShell modules, the script builds a Pester configuration object, where you can specify many properties based on your preference. It also creates a `$container` object, where you provide path to test files and inject variables those test use. After a few trial-and-error attempts, I finally got this result: ![](https://pazdedav.files.wordpress.com/2023/09/image-4.png?w=1024 align="left") # How it fits into existing tools I believe that BenchPress tests can nicely complement other tests and validations that have been around for Bicep / JSON ARM for a while: * Bicep linter - `az bicep build --file main.bicep` * ARM pre-flight validation - `azure/arm-deploy@v1` action with `deploymentMode: Validate` * what-if deployment - running `azure/arm-deploy@v1` action with `additionalArguments: --what-if` * PSRules for Azure - [docs](https://azure.github.io/PSRule.Rules.Azure/about/) The significant difference though is the fact that all the other tools listed above are checking your Bicep templates and modules before you deploy them. BenchPress tests are asserting against a "real" Azure environment, so the resources must exist. You could think of it as a post-deployment smoke test. # Going forward One slightly concerning thing is [the frequency of commits](https://github.com/Azure/benchpress/commits/main/) to the BenchPress repo and the fact that some issues that are still open were created eight months ago. I couldn't find any roadmap (e.g., a GitHub Project or a similar publicly available view), so the future of this project is uncertain. It currently has fourteen maintainers. The list of cmdlets that are available in the `Az.InfrastructureTesting` module is also quite limited: ![](https://pazdedav.files.wordpress.com/2023/09/image-2.png?w=1024 align="left") ### Native testing framework in Bicep? In addition to this project, the Bicep core team announced a new experimental testing framework during their [Community call in July 2023](https://youtu.be/UuwhLi-Xe2U) that would bring native support directly in the language. ![](https://pazdedav.files.wordpress.com/2023/09/image-6.png?w=1024 align="left")