A client approached me with a need for a temporary workaround for an issue they had where CPU usage would suddenly spike - and the workaround was (possibly at 3 AM for someone on call) to run "iisreset", as dirty as that may seem. Users were then happy again for a while.
While working on isolating the root cause, they wanted a script to automatically run iisreset when CPU usage reached 70 % or higher based on a set of sample intervals over the last five minutes. I spent some hours writing this and later decided to write a more generic version to share with the world.The sample code below will allow you to perform an action, such as run "iisreset", run any cmd/batch script, executable file, any PowerShell code, and really do anything a computer running the script/service is capable of doing.
In my sample code I just use "cmd /c echo 1" and check the exit code. To run PowerShell code, just replace it, and you probably want a try/catch statement to catch errors and warn you about them, like I do myself in the sample code, with Send-MailMessage. You can add other Send-MailMessage statements to send errors on the various conditions, such as when the condition/code/script is triggered. Or you could Splunk / log (beyond the built-in logging) / whatever.This is meant to be adapted to suit your needs.
It should work with PowerShell version 2 and up (default on Server 2008 R2 / Windows 7). Tested on 2012 R2 and Windows 10.As text:
PS C:\temp> .\testservice.ps1 VERBOSE: [2018-03-17 02:39:31] Starting PowerShell Service. VERBOSE: [2018-03-17 02:39:38] Only 1 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:39:45] Only 2 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:39:52] Only 3 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:39:59] Only 4 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:40:06] Only 5 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:40:13] Only 6 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:40:20] Only 7 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:40:27] Only 8 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:40:34] Only 9 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:40:41] CPU usage alert triggered! CPU usage for the samples was 6.69877037857689 % (trigger value: 3). Running command and emptying counter array. VERBOSE: [2018-03-17 02:40:43] Successfully ran cmd. VERBOSE: [2018-03-17 02:40:43] Memory usage alert triggered! Memory usage for the sampl es was 8.2791 GB (trigger value: 10000). Running command and emptying counter array. VERBOSE: [2018-03-17 02:40:44] Successfully ran cmd. VERBOSE: [2018-03-17 02:40:44] Average CPU usage percent for 10 samples was 6.70 %. VERBOSE: [2018-03-17 02:40:44] Average free memory for 10 samples was 8,477.81 MB. VERBOSE: [2018-03-17 02:40:55] Only 1 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:02] Only 2 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:09] Only 3 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:16] Only 4 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:23] Only 5 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:30] Only 6 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:37] Only 7 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:44] Only 8 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:51] Only 9 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:41:58] CPU usage alert triggered! CPU usage for the samples was 5.61043619484128 % (trigger value: 3). Running command and emptying counter array. VERBOSE: [2018-03-17 02:41:59] Successfully ran cmd. VERBOSE: [2018-03-17 02:41:59] Memory usage alert triggered! Memory usage for the sampl es was 8.2763 GB (trigger value: 10000). Running command and emptying counter array. VERBOSE: [2018-03-17 02:42:00] Successfully ran cmd. VERBOSE: [2018-03-17 02:42:00] Average CPU usage percent for 10 samples was 5.61 %. VERBOSE: [2018-03-17 02:42:00] Average free memory for 10 samples was 8,474.98 MB. VERBOSE: [2018-03-17 02:42:11] Only 1 samples so far. Need 10. Collect more values. VERBOSE: [2018-03-17 02:42:18] Only 2 samples so far. Need 10. Collect more values.
First I change this line in the sample code script, so I get a new log I can check to see if anything happens and gets logged.
[String] $LogFile = "C:\temp\TestingThePowerShellService.txt"This does not currently exist.
Then I take the command from the commented-out code at the top and adapt it a bit. This is on Windows 10 with the latest Win 10 Creators Edition build of nssm.exe as per 2018-03-17.This screenshot demonstrates the full process and verification that it works (as text below).
The same as text:
PS C:\temp> $PowerShellServicePath = "c:\temp\testservice.ps1"PS C:\temp> $ServiceName = "PowerShellTestService"
PS C:\temp> Start-Process -FilePath "C:\temp\nssm_x64.exe" -NoNewWindow -Wait ` >> -ArgumentList "install $ServiceName ""C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe"" ""-NoProfile"" ""-ExecutionPolicy Bypass"" ""-NonInteractive"" ""-File $PowerShellServicePath"" " Service "PowerShellTestService" installed successfully!PS C:\temp> Get-Service PowerShellTestService | Start-Service
PS C:\temp> get-dateSaturday, March 17, 2018 3:11:16 AM
PS C:\temp> Get-Content .\TestingThePowerShellService.txt [2018-03-17 03:11:08] Starting PowerShell Service.PS C:\temp> # wait a while ...
PS C:\temp> Get-Content .\TestingThePowerShellService.txt [2018-03-17 03:11:08] Starting PowerShell Service. [2018-03-17 03:12:19] CPU usage alert triggered! CPU usage for the samples was 7.20894380917347 % (trigger value: 3). Running command and emptying counter array. [2018-03-17 03:12:20] Successfully ran cmd. [2018-03-17 03:12:20] Memory usage alert triggered! Memory usage for the samples was 7.9962 GB (trigger value: 10000). Running command and emptying counter array. [2018-03-17 03:12:21] Successfully ran cmd.PS C:\temp> Get-Date
Saturday, March 17, 2018 3:12:25 AMPS C:\temp>
PS C:\temp> Get-Service PowerShellTestService | Stop-Service PS C:\temp> .\nssm_x64.exe remove PowerShellTestService confirm Service "PowerShellTestService" removed successfully! PS C:\temp> Get-Service PowerShellTestService Get-Service : Cannot find any service with service name 'PowerShellTestService'. At line:1 char:1 + Get-Service PowerShellTestService + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ + CategoryInfo : ObjectNotFound: (PowerShellTestService:String) [Get-Ser vice], ServiceCommandException + FullyQualifiedErrorId : NoServiceFoundForGivenName,Microsoft.PowerShell.Command s.GetServiceCommand PS C:\temp>
Change them as necessary and experiment to see how long the overhead is, this is good to do in the console with the frequent, verbose-timestamped messages before setting it up as a service, if that's what you choose to do.
# Set according to needs... adapt further, etc. # Set one to false to not measure that. # It makes no sense to set both to false, then you have a very strange sleep NOOP # script... [Bool] $MeasureCpu = $True [Bool] $MeasureMemory = $True [String] $SmtpServer = "smtp.internal.example.com" [String] $LogFile = "C:\temp\TestingThePowerShellService.txt" [Decimal] $CpuTriggerValue = 80.0 # in percent [Decimal] $MinimumFreeMemoryTriggerValueMB = 900 # The same intervals are used for both CPU and memory measurements, rename the # script and run two in parallel for different intervals... or rewrite yourself [Int32] $SleepSeconds = 20 [Int32] $SampleCount = 15 [Int32] $CatchCount = 0 [Int32] $HistoryMinutes = 5 # [Math]::Floor($SleepSeconds * $SampleCount / 60) + 3
Source code:
#requires -version 2 [CmdletBinding()] Param() # MIT license. # Copyright (c) 2018. Svendsen Tech. Joakim Borger Svendsen, # 2018, March. # Compatible with PowerShell version 2. <# To install as a service with nssm.exe (non-sucking service manager...) running as the SYSTEM account, adapt the command below (paths, other). $PowerShellServicePath = "c:\temp\testservice.ps1" $ServiceName = "PowerShellTestService" Start-Process -FilePath "C:\temp\nssm_x64.exe" -NoNewWindow -Wait ` -ArgumentList "install $ServiceName ""C:\Windows\system32\WindowsPowerShell\v1.0\powershell.exe"" ""-NoProfile"" ""-ExecutionPolicy Bypass"" ""-NonInteractive"" ""-File $PowerShellServicePath"" "Windows Powershell .NET CPU usage Memory usage All Categories#>
$Host.CurrentCulture.NumberFormat.NumberDecimalSeparator = '.' # Set according to needs... adapt further, etc. [Bool] $MeasureCpu = $True [Bool] $MeasureMemory = $True [String] $SmtpServer = "smtp.internal.example.com" [String] $LogFile = "C:\temp\TestingThePowerShellService.txt" [Decimal] $CpuTriggerValue = 80.0 # in percent [Decimal] $MinimumFreeMemoryTriggerValueMB = 900 # The same intervals are used for both CPU and memory measurements, rename the # script and run two in parallel for different intervals... or rewrite yourself [Int32] $SleepSeconds = 20 [Int32] $SampleCount = 15 [Int32] $CatchCount = 0 [Int32] $HistoryMinutes = 5 # [Math]::Floor($SleepSeconds * $SampleCount / 60) + 3 function Write-Log { param( [string] $Message, [switch] $Verbose = $true) "[$([DateTime]::Now.ToString('yyyy\-MM\-dd HH\:mm\:ss'))] $Message" | Add-Content -LiteralPath $LogFile -Encoding UTF8 Write-Verbose -Message "[$([DateTime]::Now.ToString('yyyy\-MM\-dd HH\:mm\:ss'))] $Message" -Verbose:$Verbose } $CpuData = @() $MemoryData = @() Write-Log -Message "Starting PowerShell Service." # Simple service... while ($True) { $DoBreak = $False while ($True) { if ($MeasureCpu) { try { $CpuUsageData = Get-Counter -Counter '\Processor(_Total)\% Processor Time' -ErrorAction Stop $DoBreak = $True } catch { $DoBreak = $False Write-Log -Message "Get-Counter failed to retrieve a cooked CPU usage value. Retrying in 5 seconds" } } if ($MeasureMemory) { try { $MemoryUsageData = Get-Counter -Counter '\Memory\Available Bytes' -ErrorAction Stop $DoBreak = $True } catch { $DoBreak = $False Write-Log -Message "Get-Counter failed to retrieve a cooked memory usage value. Retrying in 5 seconds" } } Start-Sleep -Seconds 5 if ($DoBreak) { break } <#else { Send-MailMessage ... or increase a counter and then mail, or whatever }#> } # Now we have a cooked value or two that we will add to an array or two that contain(s) values for the last $HistoryMinutes # in custom PSObjects with a Timestamp from the counter data itself. if ($MeasureCpu) { $CpuData += New-Object -TypeName PSObject -Property @{ DateTime = $CpuUsageData.Timestamp CookedValue = $CpuUsageData.CounterSamples.CookedValue } } if ($MeasureMemory) { $MemoryData += New-Object -TypeName PSObject -Property @{ DateTime = $MemoryUsageData.Timestamp CookedValue = $MemoryUsageData.CounterSamples.CookedValue } } # Filter out elements older than $HistoryMinutes. $CpuData = @($CpuData | Where-Object { $_.DateTime -gt [DateTime]::Now.AddMinutes(-1 * $HistoryMinutes) }) $MemoryData = @($MemoryData | Where-Object { $_.DateTime -gt [DateTime]::Now.AddMinutes(-1 * $HistoryMinutes) }) # We should have at least $SampleCount samples before we act... 5 minutes in sample version. if ($MeasureCpu) { if (($Count = $CpuData.Count) -lt $SampleCount) { Write-Verbose -Verbose -Message "[$([DateTime]::Now.ToString('yyyy\-MM\-dd HH\:mm\:ss'))] Only $Count samples so far. Need $SampleCount. Collect more values." continue } } if ($MeasureMemory) { if (($MemCount = $MemoryData.Count) -lt $SampleCount) { Write-Verbose -Verbose -Message "[$([DateTime]::Now.ToString('yyyy\-MM\-dd HH\:mm\:ss'))] Only $Count samples so far. Need $SampleCount. Collect more values." continue } } # We have $SampleCount or more records. Process these and run command # (iisreset in this example) if necessary. # Calculate average and see if it's higher than $CpuTriggerValue or $FreeMemoryTriggerValue, if # it is, run iisreset and reset the $CpuData array, so we start over. $AverageCpuUsage = $CpuData | Measure-Object -Property CookedValue -Average | Select-Object -ExpandProperty Average $AverageMemoryUsage = $MemoryData | Measure-Object -Property CookedValue -Average | Select-Object -ExpandProperty Average if ($MeasureCpu -and $AverageCpuUsage -ge $CpuTriggerValue) { Write-Log "CPU usage alert triggered! CPU usage for the samples was $AverageCpuUsage % (trigger value: $CpuTriggerValue). Running command and emptying counter array." $ErrorActionPreference = "Stop" while ($True) { $ProcessResult = Start-Process -FilePath cmd -NoNewWindow -Wait -ErrorAction Stop -PassThru -ArgumentList '/c', 'echo 1' if ($ProcessResult.ExitCode -eq 0) { Write-Log -Message "Successfully ran cmd." # You can add a Send-MailMessage here if you want. Send-MailMessage -To whatever@whatever.org -From ... $CpuData = @() break } else { # Avoid insane amounts of spam in case it hangs... $CatchCount++ if ($CatchCount -gt 1) { if ($CatchCount -gt 99) { $CatchCount = 0 } } else { Send-MailMessage -SmtpServer $SmtpServer -Cc 'joakim@example.com' -From 'PS_Service@example.com' ` -To 'DistributionList@example.com' -Subject "Failed to iisreset on $Env:ComputerName! $(Get-Date)" ` -Body @" PowerShell Service failed to run command on $Env:ComputerName! $(Get-Date). Will try again in about $SleepSeconds seconds. After 100 repeated failures, you will get a new mail..." "@ } } } $ErrorActionPreference = "Continue" } if ($MeasureMemory -and ($AverageMemoryUsage/1MB) -le $MinimumFreeMemoryTriggerValueMB) { Write-Log "Memory usage alert triggered! Memory usage for the samples was $('{0:N4}' -f ($AverageMemoryUsage / 1MB)) MB (trigger value: $MinimumFreeMemoryTriggerValueMB MB). Running command and emptying counter array." $ErrorActionPreference = "Stop" while ($True) { $ProcessResult = Start-Process -FilePath cmd -NoNewWindow -Wait -ErrorAction Stop -PassThru -ArgumentList '/c', 'echo 1' if ($ProcessResult.ExitCode -eq 0) { Write-Log -Message "Successfully ran cmd." # You can add a Send-MailMessage here if you want. Send-MailMessage -To whatever@whatever.org -From ... $MemoryData = @() break } else { # Avoid insane amounts of spam in case it hangs... $CatchCount++ if ($CatchCount -gt 1) { if ($CatchCount -gt 99) { $CatchCount = 0 } } else { Send-MailMessage -SmtpServer $SmtpServer -Cc 'joakim@example.com' -From 'PS_Service@example.com' ` -To 'DistributionList@example.com' -Subject "PowerShell service failed to iisreset on $Env:ComputerName! $(Get-Date)" ` -Body @" PowerShell Service failed to run command on $Env:ComputerName! $(Get-Date). Will try again in about $SleepSeconds seconds. After 100 repeated failures, you will get a new mail..." "@ } } } $ErrorActionPreference = "Continue" } Write-Verbose -Verbose -Message "[$([DateTime]::Now.ToString('yyyy\-MM\-dd HH\:mm\:ss'))] Average CPU usage percent for $Count samples was $('{0:N2}' -f $AverageCpuUsage) %." Write-Verbose -Verbose -Message "[$([DateTime]::Now.ToString('yyyy\-MM\-dd HH\:mm\:ss'))] Average free memory for $Count samples was $('{0:N2}' -f ($AverageMemoryUsage/1MB)) MB." Start-Sleep -Seconds $SleepSeconds # Avoid memory leaks, especially on PS versions before 4 or 5 (not sure which version (apparently) got better at it). [System.GC]::Collect() }
Minimum cookies is the standard setting. This website uses Google Analytics and Google Ads, and these products may set cookies. By continuing to use this website, you accept this.