Test Coverage For PowerShell Scripts With Pester

Before we start, the complete code used for this article is available here - https://github.com/santoshkaranam/PSUnitTestFramework

If you are familiar with powershell, perster and PSSCriptAnalyzer skip to the section- “Generating the Report”.

Writing powershell scripts has been the easiest way for devops to automate things if they are from an azure .net background, as powershell works on all platforms. But we often forget that these scripts also need to be maintainable. For this we need to have all the coding guidelines, test cases and test coverage done to keep the scripts maintainable. Here I am discussing my solution to achieve the same using Pester, PSSCriptAnalyzer ReportUnit.

PowerShell

PowerShell runs on Windows, Linux, and macOS. PowerShell is a cross-platform task automation solution made up of a command-line shell, a scripting language, and a configuration management framework.

Command-line Shell

The shell includes the following features:

Robust command-line history
Tab completion and command prediction (See about_PSReadLine)
Supports command and parameter aliases
Pipeline for chaining commands
In-console help system, similar to Unix man pages

You can open powershell editor using “powershell ise” in run.

Note: - To develop and run powershell scripts in your machine, you need to change the execution policy to unrestricted.

Set-ExecutionPolicy Unrestricted

Automation platform

The extensible nature of PowerShell has enabled an ecosystem of PowerShell modules to deploy and manage almost any technology you work with. For example:

Microsoft

Azure Windows Exchange SQL

Third-party

AWS VMWare Google Cloud

As a result of these features, powershell is used extensively to automate, but these scripts are not tested as once automation is completed these scripts are never modified till something failes.

To have maintainable scripts one needs to write test cases to cover all the scenarios and cover the entire scripts. In this article, I am going to explain one of the ways to write test cases for scripts and completely automate the generation of coverage reports.

For this I am using Perter and PSScriptAnalyzer which are available as module in powershell. The end result will generate a HTML file which can be visualized as below

With this HTML generated using ReportUnit.exe you can check the test cases which have passed and check if some test cases have failed. For the failed test cases exact test cases and lines are available in the HTML report. This report can be published in a static server to be visualized.

What is Pester?

Pester is a testing and mocking framework for PowerShell.

Pester provides a framework for writing and running tests. Pester is most commonly used for writing unit and integration tests, but it is not limited to just that. It is also a base for tools that validate whole environments, computer deployments, database configurations and so on.

Creating a Pester Test

BeforeAll {
    function Get-Planet ([string]$Name = '*') {
        $planets = @(
            @{ Name = 'Mercury' }
            @{ Name = 'Venus'   }
            @{ Name = 'Earth'   }
            @{ Name = 'Mars'    }
            @{ Name = 'Jupiter' }
            @{ Name = 'Saturn'  }
            @{ Name = 'Uranus'  }
            @{ Name = 'Neptune' }
        ) | ForEach-Object { [PSCustomObject] $_ }
        $planets | Where-Object { $_.Name -like $Name }
    }
}
Describe 'Get-Planet' {
    It 'Given no parameters, it lists all 8 planets' {
        $allPlanets = Get-Planet
        $allPlanets.Count | Should -Be 8
    }
}

You can read more about how to write powershell script test cases using pester here - https://pester-docs.netlify.app/docs/quick-start

Perster also generates code coverage stats for Hit Commands (lines hit), Missed comments(not covered in tests) this is a useful feature when you want to analyses your code.

Once we have test cases written next is to check the code coverage and script analysis for rule violations.

PSScriptAnalyzer For Static Script analysis

PSScriptAnalyzer is a static code checker for PowerShell modules and scripts. PSScriptAnalyzer checks the quality of PowerShell code by running a set of rules. The rules are based on PowerShell best practices identified by PowerShell Team and the community. It generates DiagnosticResults (errors and warnings) to inform users about potential code defects and suggests possible solutions for improvements.

PSScriptAnalyzer is shipped with a collection of built-in rules that checks various aspects of PowerShell code such as presence of uninitialized variables, usage of PSCredential Type, usage of Invoke-Expression etc. Additional functionalities such as exclude/include specific rules are also supported.

More can be read here - https://github.com/PowerShell/PSScriptAnalyzer

Now the Meat!!!!!!!!

Generating the Report

Now that we know how to write test cases with perter, PSSCriptAnalyzer we will dig deep into the .\TestExecutor.ps1

Import-Module Pester

# paths to include for code coverage report
$filesCovered = @("$PSScriptRoot\scripts\*.psm1","$PSScriptRoot\scripts\*.ps1")

#check and create artifacts folder
$artifactFolder = "$PSScriptRoot\artifacts"
if (!(test-path -path $artifactFolder)) {new-item -path $artifactFolder -itemtype directory}

# run pester test cases
$testOutputFile = "$PSScriptRoot\artifacts\TestResults.xml"
$results = Invoke-Pester -EnableExit -OutputFile $testOutputFile -OutputFormat NUnitXml -PassThru -CodeCoverage $filesCovered

#run reportunit.exe to generate test report using input '$testOutputFile' in same folder
.\reportgenerator\ReportUnit.exe "$testOutputFile"


#output coverage result to different files from '$results.CodeCoverage' variable
$results.CodeCoverage.HitCommands | ConvertTo-Html | Out-File -FilePath "$PSScriptRoot\artifacts\HitCommands.html"
if($results.CodeCoverage.MissedCommands -ne $null){
$results.CodeCoverage.MissedCommands | ConvertTo-Html | Out-File -FilePath "$PSScriptRoot\artifacts\MissedCommands.html"}
$results.CodeCoverage.AnalyzedFiles | Out-File -FilePath "$PSScriptRoot\artifacts\AnalyzedFiles.txt"

#calculate coverage percentage
$Percentage = ($results.CodeCoverage.NumberOfCommandsExecuted / $results.CodeCoverage.NumberOfCommandsAnalyzed)*100
$results.CodeCoverage | Add-Member Percentage ($Percentage | % { '{0:0.##}' -f $_ })
$results.CodeCoverage | ConvertTo-Html  | Out-File -FilePath "$PSScriptRoot\artifacts\CoverageReport.html"

In the above script we are first getting all the files that needs to be tested and running the perster module to test the scripts using the testcases written in separate scripts.

The results are then stored to XML file for report generator to use.

The report generator is fed with this file to produce a static HTML to visualize.

Error Report sample

You can check that the page actually shows which line in the script has the error.

GitHub: https://github.com/santoshkaranam/PSUnitTestFramework

If you like this framework, you can clone this repository and use it with your team. This code is licensed under MIT License. You can improve the framework by contributing to the code.

Conclusion

Writing powershell scripts has been the easiest way for devops to automate things if they are from an azure .net background as powershell works on all platforms. But we often forget that these scripts also need to be maintainable. For this we need to have all the coding guidelines, test cases and test coverage done to keep the scripts maintainable.

I have shown here simple ways which I followed to maintain the scripts I have written using Pester and PSSCriptAnalyzer and ReportGenerator using which we can not only write test cases but also generate coverage reports and test cases execution status in a beautiful HTML provided by ReportUnit.Exe which can be published in build servers for team access.

Please download the code from github repo and execute locally and let me know the feedback.

References