PowerShell benchmarking module built around Measure-Command

From Svendsen Tech PowerShell Wiki
Jump to: navigation, search

Quite often I've found myself wondering what will work faster, given two or more ways of solving a problem. A way to get a clue is to benchmark it. This will allow you to optimize scripts in some cases. Presented here is a PowerShell module called Benchmark which exports the function Measure-These, which will work like a cmdlet, and can be used to measure how long one or more script blocks take to execute the specified number of times.

The module works with PowerShell version 2 and up.

I use Measure-Object to collect data about the processed script blocks, and apparently there is quite a bit of overhead involved in this, so if the measured commands take about 1.1 seconds with 10,000 iterations, you might find the benchmark script itself to take maybe 12 seconds, as in one of my test cases. The overhead seems to increase fast with the number of iterations. I tried rolling my own algorithm instead of using Measure-Object, but failed miserably at improving the performance.

If you prefer to have it as a function in your profile rather than a module, simply copy and paste the entire content of the .psm1 file into your PowerShell profile (external Microsoft site link). You can also dot-source the script file containing the Measure-These function on demand when you need it, but you need to rename the file to have a .ps1 extension as modules pop up in notepad (for "security" reasons) when dot-sourced:



Dot-sourcing it on demand:

PS C:\> copy .\Benchmark.psm1 Benchmark.ps1
PS C:\> . .\Benchmark.ps1
PS C:\> Get-Help Measure-These

It's based on Measure-Command, which works like this:

PS E:\> Measure-Command { dir -r e:\temp } | Select -ExpandProperty TotalMilliseconds
223,0757

PS E:\> Measure-Command { dir -r e:\temp }


Days              : 0
Hours             : 0
Minutes           : 0
Seconds           : 0
Milliseconds      : 51
Ticks             : 512956
TotalDays         : 5,93699074074074E-07
TotalHours        : 1,42487777777778E-05
TotalMinutes      : 0,000854926666666667
TotalSeconds      : 0,0512956
TotalMilliseconds : 51,2956

Download

Benchmark.zip - download, remember to unblock before extracting and unzip into a PowerShell module directory (tip: see $env:PSModulePath - and create a "Modules" directory if necessary). Or install from the PowerShell gallery (see below).

If you have Windows Management Framework 5 or higher (WMF 5 is available for Windows 7 and up), you can install my "Benchmark" module from the PowerShell gallery, a Microsoft-driven project and online repository for scripts.

To install with WMF 5 and up (to get the latest Benchmark module version available), simply run this command (requires an internet connection):

Install-Module -Name Benchmark

or for your user only (no elevation required):

Install-Module -Name Benchmark -Scope CurrentUser
  • 2017-12-02: Uploaded v1.2.2. Minor doc fixes. Make $Precision a Byte. Use [CmdletBinding()] and add scaffolding.
  • 2017-12-01: Uploaded v1.2.1. Return numerical types for easier use with comparison operators (such as -gt and -lt). Add optional -Precision parameter (default 5). Trailing zeroes are removed (by [Math]::Round()).



Example Screenhots

Benchmark-module-example-psv5.png


One with multiple counts specified:


PowerShell-Benchmark-Module-Example-Multiple-Counts2.png

Examples As Text

Let's see how accurate Start-Sleep and the measurement is. Not too bad.

PS C:\> Import-Module Benchmark
PS C:\> Measure-These -Count 5 -ScriptBlock { sleep 1 }, { sleep 2 }, { sleep 5 }
-Title '1 second', '2 seconds', '5 seconds' | Format-Table -AutoSize

Title/no. Average (ms) Count Sum (ms)     Maximum (ms) Minimum (ms)
--------- ------------ ----- --------     ------------ ------------
1 second  1,000.06218      5 5,000.31090  1,002.43900  999.41950
2 seconds 1,999.58470      5 9,997.92350  1,999.66930  1,999.53590
5 seconds 4,999.75984      5 24,998.79920 4,999.95170  4,999.67560

Compare some methods of determining if a string is contained within another string. 50,000 iterations. This time around on PSv4, where they appear to have made some optimizations to "-like".

PS C:\temp> $String = 'some target string'

PS C:\temp> Measure-These -Count 50000 -ScriptBlock { $String -match 'target' }, { $String.Contains('target') },
{ $String -like '*target*' } -Title '-match', '.contains()', '-like' | ft -auto

Title/no.   Average (ms) Count Sum (ms)    Maximum (ms) Minimum (ms)
---------   ------------ ----- --------    ------------ ------------
-match      0.03024      50000 1,512.10510 2.80810      0.02630     
.contains() 0.02386      50000 1,192.81570 6.16960      0.01950     
-like       0.02893      50000 1,446.25670 2.93130      0.02380     

Default output with Format-List.

PS C:\> Measure-These -Count 1 { sleep 3 }

Title/no.    : 1
Average (ms) : 3,000.46310
Count        : 1
Sum (ms)     : 3,000.46310
Maximum (ms) : 3,000.46310
Minimum (ms) : 3,000.46310

PS C:\>

Assigning the resulting object to a variable.

PS C:\> $TimedSleep = Measure-These -Count 1 { sleep 3 }
PS C:\> $TimedSleep


Title/no.    : 1
Average (ms) : 2,999.98120
Count        : 1
Sum (ms)     : 2,999.98120
Maximum (ms) : 2,999.98120
Minimum (ms) : 2,999.98120

Source Code

function Measure-These {
    <#
    .SYNOPSIS
        Svendsen Tech's Benchmarking Module for PowerShell.

        Benchmark PowerShell script blocks and (virtually) any "DOS"/cmd.exe command
        using this module built around PowerShell's Measure-Command cmdlet. It is designed
        to give a quick, convenient overview of how code performs when doing for instance
        the same thing in different ways.

        See the comprehensive online documentation at:
        http://www.powershelladmin.com/wiki/PowerShell_benchmarking_module_built_around_Measure-Command

        MIT license.

    .DESCRIPTION
        This is a benchmarking module for PowerShell. Get objects containing data about
        the execution time of script blocks. Pipe to Format-Table -AutoSize
        for a direct report. You can also assign the resulting objects to a variable (do not
        use Format-Table if assigning to a variable).

        See the comprehensive online documentation at:
        http://www.powershelladmin.com/wiki/PowerShell_benchmarking_module_built_around_Measure-Command

        Copyright (c) 2012-2017, Joakim Borger Svendsen.
        All rights reserved.
        Author: Joakim Borger Svendsen

        MIT license.

    .PARAMETER Count
        Number of times to execute the code in each specified script block. Pass in
        multiple counts separated by commas.
    .PARAMETER ScriptBlock
        Script block(s) to time the execution time of and collect data about.
    .PARAMETER Title
        Optional titles for each script block. Title 1 goes with block 1,
        2 with 2, and so on. If you omit titles, you will get numbered
        script blocks (from left to right). If you have fewer titles than
        script blocks, you will get numbers when you "run out of titles".
    .PARAMETER Precision
        Specify number of digits after the decimal separator. Default 5. Maximum 15.
        Trailing zeroes are removed by the [Math]::Round() static function.
    #>
    [CmdletBinding()]
    param(
        [Parameter(Mandatory = $true)] [Int32[]] $Count,
        [Parameter(Mandatory = $true)] [ScriptBlock[]] $ScriptBlock,
        [String[]] $Title = @(''),
        [ValidateRange(1, 15)][Byte] $Precision = 5)
    begin {
    }
    process {
        $Times = @()
        $BlockNumber = 0
        foreach ($LoopCount in $Count) {
            $ScriptBlock | ForEach-Object {
                $Block = $_
                $BlockNumber++
                $Times += 1..$LoopCount | ForEach-Object {
                    Measure-Command -Expression $Block | # Process the current block once for each specified $Count.
                        Select-Object -ExpandProperty TotalMilliSeconds # Get only the milliseconds.
                    } | # End of 1..$Count ForEach-Object which will be passed to Measure-Object.
                    Measure-Object -Sum -Maximum -Minimum | # Gather results using Measure-Object.
                    ForEach-Object {
                        # Send results down the pipeline in the form of custom PS objects.
                        New-Object -TypeName PSObject -Property @{
                            'Average (ms)' = $_.Sum / $_.Count
                            'Maximum (ms)' = $_.Maximum
                            'Minimum (ms)' = $_.Minimum
                            'Count' = $_.Count
                            'Sum (ms)' = $_.Sum
                            'BlockNumber' = $BlockNumber
                        }
                    }
            } # End of $ScriptBlock ForEach-Object
        } # End of $Count foreach loop
    }
    end {
        # Since this is a _benchmarking_ module, it seems in the right spirit to
        # cache this for a performance gain. :)
        $NumBlocks = $ScriptBlock.Count
        # This is used to keep track of the relative position in the block count.
        # Had to script scope it or else I think it behaved sort of like a closure
        # inside the Select-Object below.
        $Script:Counter = 0
        $Times |
            Select-Object @{n='Title/no.'; e={
                    ++$script:Counter
                    $Index = $script:Counter - 1
                    if ($script:Counter -ge $NumBlocks) {
                        $script:Counter = 0
                    }
                    if ($Title[$Index]) {
                        $Title[$Index]
                    }
                    else {
                        $_.BlockNumber
                    }
                   }},
                   @{ Name = 'Average (ms)'; Expression = { [Math]::Round($_.'Average (ms)', $Precision)} }, Count,
                   @{ Name = 'Sum (ms)'; Expression = { [Math]::Round($_.'Sum (ms)', $Precision) } },
                   @{ Name = 'Maximum (ms)'; Expression = { [Math]::Round($_.'Maximum (ms)', $Precision) } },
                   @{ Name = 'Minimum (ms)'; Expression = { [Math]::Round($_.'Minimum (ms)', $Precision) } }
        
    }
}