ConvertTo-Json for PowerShell version 2

From Svendsen Tech PowerShell Wiki
Jump to: navigation, search

Bad news: No ConvertFrom-Json here. Good news: A pure-PowerShell ConvertTo-Json for PowerShell version 2 that works pretty well after some testing, polishing and experimentation.

JSON, JavaScript Object Notation is the nerdy favorite for simple serialization/exchange of data. Support was introduced in PowerShell version 3. I found myself wanting/needing PowerShell version 2 support for it to format a web request to send data to Splunk also from legacy operating systems and systems that for various reasons haven't been upgraded to PSv3 yet. So I started writing mostly to see if I could do it. A significant amount of hours of work later I have a working function. I'll try to blog my Send-SplunkMessage function later (which I called Write-SplunkMessage, but later changed my mind about). In PSv3 it's pretty easy with Invoke-RestMethod and ConvertTo-Json (built in).

Hash tables, custom PowerShell objects and most types of collections/arrays are handled by my ConvertTo-STJson. Arbitrarily deeply nested data. If you have objects of other types, pass them to | Select-Object -Property Some, Properties, Here - to select properties and conveniently convert it to a PowerShell object. Calculated properties should be great here for custom fields. It's not perfect and you're likely to run into corner cases if you use complex objects / data structures. Try using Select-Object and finding workarounds. Feedback can be given to svendsentech@gmail.com or on GitHub.

The safest thing is likely to construct a data structure within a hash table that you pass to ConvertTo-STJson ("ST" is for Svendsen Tech - a prefix to make this advanced function name presumably unique). It works best with passing in a hash or a PS object that contains the rest of the data, but I also tried to add support directly for arrays, strings and even bools and $null ($null turns into an empty string and bool types aren't preserved, but comparisons against -eq $false/true are evaluated correctly - corner cases anyway).


Screenshot example of a fairly complex data structure converted to JSON

ConvertTo-Json-PSv3-ConvertTo-STJson-example.png

Various examples

PS C:\> ConvertTo-STJson "string"
"string" 

PS C:\> ConvertTo-STJson "a", 'simple', "array"
[
        "a",
        "simple",
        "array"
]

PS C:\> ConvertTo-STJson @{ Simple = 'Hash table' }
{
    "Simple": "Hash table"
}

PS C:\> #. .\ConvertTo-STJson.ps1;
ConvertTo-STJson -InputObject @{
    a = @{ a1 = 'value1'; a2 = 'value2'; a3 = @(1, 'two', 3) }
    b = "test"
    c = [pscustomobject] @{ c1 = 'value1' }
    d = @( @{ foo = 'bar' }, @{ foo2 = 'bar2';
    foo_inner_array = @( @{ deephash = (1..4);
    deephash2 = [pscustomobject] @{ a = 1 } }  )})
}
######################################
{
    "c":
    {
        "c1": "value1"
    },
    "d":
    [
        {
            "foo": "bar"
        },
        {
            "foo_inner_array":
            [
                {
                    "deephash2":
                    {
                        "a": 1
                    },
                    "deephash":
                    [
                        1,
                        2,
                        3,
                        4
                    ]
                }
            ],
            "foo2": "bar2"
        }
    ],
    "b": "test",
    "a":
    {
        "a1": "value1",
        "a2": "value2",
        "a3":
        [
            1,
            "two",
            3
        ]
    }
}

PS C:\> ConvertTo-STJson -InputObject (Get-Service -Name bits | select Name, DisplayName, status)
{
    "DisplayName": "Background Intelligent Transfer Service",
    "Name": "bits",
    "Status": "Running"
}

PS C:\Dropbox\PowerShell\ConvertTo-Json> dir somefile* | Select Name, FullName | ConvertTo-STJson
[
            {
                "FullName": "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile1.txt",
                "Name": "somefile1.txt"
            },
            {
                "FullName": "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile2.txt",
                "Name": "somefile2.txt"
            },
            {
                "FullName": "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile3.txt",
                "Name": "somefile3.txt"
            },
            {
                "FullName": "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile4.txt",
                "Name": "somefile4.txt"
            },
            {
                "FullName": "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile5.txt",
                "Name": "somefile5.txt"
            },
            {
                "FullName": "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile6.txt",
                "Name": "somefile6.txt"
            }
]

PS C:\Dropbox\PowerShell\ConvertTo-Json> dir somefile* | ConvertTo-STJson
[
        "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile1.txt",
        "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile2.txt",
        "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile3.txt",
        "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile4.txt",
        "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile5.txt",
        "C:\\Dropbox\\PowerShell\\ConvertTo-Json\\somefile6.txt"
]

PS C:\Dropbox\PowerShell\ConvertTo-Json> 

Download and code

ConvertTo-STJson.ps1.txt - right click, download, unblock, rename to .ps1 only, dot-source (. X:\path\ConvertTo-STJson.ps1), use.

File history: File:ConvertTo-STJson.ps1.txt.

I'm starting to grow up, so I also pushed ConvertTo-STJson to GitHub here. It's likely you'll get the best overview in the version history in this wiki, but for now I'm working against GitHub mostly and keeping that up to date before posting (slightly) more "final" releases here in the wiki, so you'll be able to get the latest version there first sometimes and also track history more closely (more frequent posts).

Now that I'm practically huge, I even made some Pester 4.x tests for this function. See GitHub for a possibly more updated version, but here's a version of the file here in this wiki: File:ConvertTo-STJson.Tests.ps1.txt.

Version history

  • 2017-10-21: v0.9.4.0: Fix nested array bracket alignment issues.
  • 2017-04-19: v0.9.3.3: Comparing to the PS team's ConvertTo-Json again and they don't escape "/" alone. Undoing 0.9.2.2 change. Bug fix fix.
  • 2017-04-19: v0.9.3.2: Coerce numbers from strings only if -CoerceNumberStrings is specified (non-default), properly detect numerical types and by default omit double quotes only on these.
  • 2017-04-18: v0.9.2.2. Also escaping solidus (forward slash). Bug fix.
  • 2017-04-18: v0.9.2.1. Minor stuff.
  • 2017-04-18: v0.9.2. Returning proper value types when sending in only single values of $true and $false (passed through), but the bool type isn't preserved. $null is buggy as well and turns into an empty string.
  • 2017-04-18: v0.9. Vast improvements. Way more standards-conforming. Major rewrite. Still works on v2, of course (after a couple of bug workarounds). Now handles the types $null, $true and $false properly and the parameters -EscapeAll and -QuoteValueTypes are gone (fixed with overall better logic).
  • 2017-04-14: v0.8.2. Now calculated properties work. This greatly improves flexibility. I've noticed the PS team's ConvertTo-Json does some things I don't and vice versa. It converts datetime typed objects into "/Date(number here)/". I'll look into this a bit more.
  • 2017-04-14: v0.8.1. Fixed a bug causing x.y numbers to be quoted. Started occurring when I introduced scientific notation in v0.4.
  • 2017-04-13: v0.8. Added a -Compress parameter (save some bytes when sending data).
  • 2017-04-13: v0.7. Changed parameter name -EscapeAllowedEscapesToo to -EscapeAll (before it's too late).
  • 2017-04-13: v0.6. Added -QuoteValueTypes to also quote "null", "true" and "false".
  • 2017-04-12: v0.5. Added switch parameter "EscapeAllowedEscapesToo" (later changed to "-EscapeAll" - also see 'Quirks' section).
  • 2017-04-12: v0.4. Added support for scientific notation on numbers.
  • 2017-04-12: v0.3. Actually read up on JSON syntax. Now I'm not escaping allowed escape sequences (see http://www.json.org ). Not quoting "null", "true" and "false" as values.
  • 2017-04-12: Added pipeline support. Let's say it's v0.2. Second upload.

Source code

# Author: Joakim Borger Svendsen, 2017. http://www.json.org
# Svendsen Tech. Public domain licensed code.
# v0.3, 2017-04-12 (second release of the day, I actually read some JSON syntax this time)
#       Fixed so you don't double-whack the allowed escapes from the diagram, not quoting null, false and true as values.
# v0.4. Scientific numbers are supported (not quoted as values). 2017-04-12.
# v0.5. Adding switch parameter EscapeAllowedEscapesToo (couldn't think of anything clearer),
#       which also double-whacks (escapes with backslash) allowed escape sequences like \r, \n, \f, \b, etc.
#       Still 2017-04-12.
# v0.6: It's after midnight, so 2017-04-13 now. Added -QuoteValueTypes that makes it quote null, true and false as values.
# v0.7: Changed parameter name from EscapeAllowedEscapesToo to EscapeAll (... seems obvious now). Best to do it before it's
#       too late. 2017-04-13.
# v0.7.1: Made the +/- after "e" in numbers optional as this is apparently valid (as plus, then)
# v0.8: Added a -Compress parameter! 2017-04-13.
# v0.8.1: Fixed bug that made "x.y" be quoted (but scientific numbers and integers worked all the while). 2017-04-14.
# v0.8.2: Fixed bug with calculated properties (yay, this improves flexibility significantly). 2017-04-14.
# v0.9: Almost too many changes to mention. Now null, true and false as _value types_ are unquoted, otherwise they
#       are quoted. Comparing to the PowerShell team's ConvertTo-Json. Now escaping works better and more
#       standards-conforming. If you have a newline in the strings, it'll be replaced by "\n" (literally, not a newline),
# while if you have "\n" literally, it'll turn into \\n. Code quality improvements. Refactoring. Still some more to fix,
# but it's getting better. Datetime stuff is bothering me, not sure I like how it's handled in the PS team's cmdlet, but I
# don't have a sufficiently informed opinion.
#
# v0.9.1: Formatting fixes.
# v0.9.2: Returning proper value types when sending in only single values of $true and $false (passed through).
#         $null is buggy, but only if you pass in _nothing_ else, but $null. As a value in an array, hash or
#         anywhere else, it works fine.
# v0.9.2.1: Forgot.
# v0.9.2.2: Adding escaping of "solidus" (forward slash).
# v0.9.3: Coerce numbers from strings only if -CoerceNumberStrings is specified (non-default), properly detect numerical types and
#         by default omit double quotes only on these.
# v0.9.3.1: Respect and do not doublewhack/escape (regex) "\u[0-9a-f]{4}".
# v0.9.3.2: Undoing previous change ... (wrong logic).
# v0.9.3.3: Comparing to the PS team's ConvertTo-Json again and they don't escape "/" alone. Undoing 0.9.2.2 change.
# v0.9.3.4: Support the IA64 platform and int64 on that too.
# v0.9.4.0: Fix nested array bracket alignment issues. 2017-10-21.
######################################################################################################

# Take care of special characters in JSON (see json.org), such as newlines, backslashes
# carriage returns and tabs.
# '\\(?!["/bfnrt]|u[0-9a-f]{4})'
function FormatString {
    param(
        [String] $String)
    # removed: #-replace '/', '\/' `
    # This is returned 
    $String -replace '\\', '\\' -replace '\n', '\n' `
        -replace '\u0008', '\b' -replace '\u000C', '\f' -replace '\r', '\r' `
        -replace '\t', '\t' -replace '"', '\"'
}

# Meant to be used as the "end value". Adding coercion of strings that match numerical formats
# supported by JSON as an optional, non-default feature (could actually be useful and save a lot of
# calculated properties with casts before passing..).
# If it's a number (or the parameter -CoerceNumberStrings is passed and it 
# can be "coerced" into one), it'll be returned as a string containing the number.
# If it's not a number, it'll be surrounded by double quotes as is the JSON requirement.
function GetNumberOrString {
    param(
        $InputObject)
    if ($InputObject -is [System.Byte] -or $InputObject -is [System.Int32] -or `
        ($env:PROCESSOR_ARCHITECTURE -imatch '^(?:amd64|ia64)$' -and $InputObject -is [System.Int64]) -or `
        $InputObject -is [System.Decimal] -or $InputObject -is [System.Double] -or `
        $InputObject -is [System.Single] -or $InputObject -is [long] -or `
        ($Script:CoerceNumberStrings -and $InputObject -match $Script:NumberRegex)) {
        Write-Verbose -Message "Got a number as end value."
        "$InputObject"
    }
    else {
        Write-Verbose -Message "Got a string as end value."
        """$(FormatString -String $InputObject)"""
    }
}

function ConvertToJsonInternal {
    param(
        $InputObject, # no type for a reason
        [Int32] $WhiteSpacePad = 0)
    [String] $Json = ""
    $Keys = @()
    Write-Verbose -Message "WhiteSpacePad: $WhiteSpacePad."
    if ($null -eq $InputObject) {
        Write-Verbose -Message "Got 'null' in `$InputObject in inner function"
        $null
    }
    elseif ($InputObject -is [Bool] -and $InputObject -eq $true) {
        Write-Verbose -Message "Got 'true' in `$InputObject in inner function"
        $true
    }
    elseif ($InputObject -is [Bool] -and $InputObject -eq $false) {
        Write-Verbose -Message "Got 'false' in `$InputObject in inner function"
        $false
    }
    elseif ($InputObject -is [HashTable]) {
        $Keys = @($InputObject.Keys)
        Write-Verbose -Message "Input object is a hash table (keys: $($Keys -join ', '))."
    }
    elseif ($InputObject.GetType().FullName -eq "System.Management.Automation.PSCustomObject") {
        $Keys = @(Get-Member -InputObject $InputObject -MemberType NoteProperty |
            Select-Object -ExpandProperty Name)
        Write-Verbose -Message "Input object is a custom PowerShell object (properties: $($Keys -join ', '))."
    }
    elseif ($InputObject.GetType().Name -match '\[\]|Array') {
        Write-Verbose -Message "Input object appears to be of a collection/array type."
        Write-Verbose -Message "Building JSON for array input object."
        #$Json += " " * ((4 * ($WhiteSpacePad / 4)) + 4) + "[`n" + (($InputObject | ForEach-Object {
        $Json += "[`n" + (($InputObject | ForEach-Object {
            if ($null -eq $_) {
                Write-Verbose -Message "Got null inside array."
                " " * ((4 * ($WhiteSpacePad / 4)) + 4) + "null"
            }
            elseif ($_ -is [Bool] -and $_ -eq $true) {
                Write-Verbose -Message "Got 'true' inside array."
                " " * ((4 * ($WhiteSpacePad / 4)) + 4) + "true"
            }
            elseif ($_ -is [Bool] -and $_ -eq $false) {
                Write-Verbose -Message "Got 'false' inside array."
                " " * ((4 * ($WhiteSpacePad / 4)) + 4) + "false"
            }
            elseif ($_ -is [HashTable] -or $_.GetType().FullName -eq "System.Management.Automation.PSCustomObject" -or $_.GetType().Name -match '\[\]|Array') {
                Write-Verbose -Message "Found array, hash table or custom PowerShell object inside array."
                " " * ((4 * ($WhiteSpacePad / 4)) + 4) + (ConvertToJsonInternal -InputObject $_ -WhiteSpacePad ($WhiteSpacePad + 4)) -replace '\s*,\s*$' #-replace '\ {4}]', ']'
            }
            else {
                Write-Verbose -Message "Got a number or string inside array."
                $TempJsonString = GetNumberOrString -InputObject $_
                " " * ((4 * ($WhiteSpacePad / 4)) + 4) + $TempJsonString
            }
        #}) -join ",`n") + "`n],`n"
        }) -join ",`n") + "`n$(" " * (4 * ($WhiteSpacePad / 4)))],`n"
    }
    else {
        Write-Verbose -Message "Input object is a single element (treated as string/number)."
        GetNumberOrString -InputObject $InputObject
    }
    if ($Keys.Count) {
        Write-Verbose -Message "Building JSON for hash table or custom PowerShell object."
        $Json += "{`n"
        foreach ($Key in $Keys) {
            # -is [PSCustomObject]) { # this was buggy with calculated properties, the value was thought to be PSCustomObject
            if ($null -eq $InputObject.$Key) {
                Write-Verbose -Message "Got null as `$InputObject.`$Key in inner hash or PS object."
                $Json += " " * ((4 * ($WhiteSpacePad / 4)) + 4) + """$Key"": null,`n"
            }
            elseif ($InputObject.$Key -is [Bool] -and $InputObject.$Key -eq $true) {
                Write-Verbose -Message "Got 'true' in `$InputObject.`$Key in inner hash or PS object."
                $Json += " " * ((4 * ($WhiteSpacePad / 4)) + 4) + """$Key"": true,`n"            }
            elseif ($InputObject.$Key -is [Bool] -and $InputObject.$Key -eq $false) {
                Write-Verbose -Message "Got 'false' in `$InputObject.`$Key in inner hash or PS object."
                $Json += " " * ((4 * ($WhiteSpacePad / 4)) + 4) + """$Key"": false,`n"
            }
            elseif ($InputObject.$Key -is [HashTable] -or $InputObject.$Key.GetType().FullName -eq "System.Management.Automation.PSCustomObject") {
                Write-Verbose -Message "Input object's value for key '$Key' is a hash table or custom PowerShell object."
                $Json += " " * ($WhiteSpacePad + 4) + """$Key"":`n$(" " * ($WhiteSpacePad + 4))"
                $Json += ConvertToJsonInternal -InputObject $InputObject.$Key -WhiteSpacePad ($WhiteSpacePad + 4)
            }
            elseif ($InputObject.$Key.GetType().Name -match '\[\]|Array') {
                Write-Verbose -Message "Input object's value for key '$Key' has a type that appears to be a collection/array."
                Write-Verbose -Message "Building JSON for ${Key}'s array value."
                $Json += " " * ($WhiteSpacePad + 4) + """$Key"":`n$(" " * ((4 * ($WhiteSpacePad / 4)) + 4))[`n" + (($InputObject.$Key | ForEach-Object {
                    #Write-Verbose "Type inside array inside array/hash/PSObject: $($_.GetType().FullName)"
                    if ($null -eq $_) {
                        Write-Verbose -Message "Got null inside array inside inside array."
                        " " * ((4 * ($WhiteSpacePad / 4)) + 8) + "null"
                    }
                    elseif ($_ -is [Bool] -and $_ -eq $true) {
                        Write-Verbose -Message "Got 'true' inside array inside inside array."
                        " " * ((4 * ($WhiteSpacePad / 4)) + 8) + "true"
                    }
                    elseif ($_ -is [Bool] -and $_ -eq $false) {
                        Write-Verbose -Message "Got 'false' inside array inside inside array."
                        " " * ((4 * ($WhiteSpacePad / 4)) + 8) + "false"
                    }
                    elseif ($_ -is [HashTable] -or $_.GetType().FullName -eq "System.Management.Automation.PSCustomObject" `
                        -or $_.GetType().Name -match '\[\]|Array') {
                        Write-Verbose -Message "Found array, hash table or custom PowerShell object inside inside array."
                        " " * ((4 * ($WhiteSpacePad / 4)) + 8) + (ConvertToJsonInternal -InputObject $_ -WhiteSpacePad ($WhiteSpacePad + 8)) -replace '\s*,\s*$'
                    }
                    else {
                        Write-Verbose -Message "Got a string or number inside inside array."
                        $TempJsonString = GetNumberOrString -InputObject $_
                        " " * ((4 * ($WhiteSpacePad / 4)) + 8) + $TempJsonString
                    }
                }) -join ",`n") + "`n$(" " * (4 * ($WhiteSpacePad / 4) + 4 ))],`n"
            }
            else {
                Write-Verbose -Message "Got a string inside inside hashtable or PSObject."
                # '\\(?!["/bfnrt]|u[0-9a-f]{4})'
                $TempJsonString = GetNumberOrString -InputObject $InputObject.$Key
                $Json += " " * ((4 * ($WhiteSpacePad / 4)) + 4) + """$Key"": $TempJsonString,`n"
            }
        }
        $Json = $Json -replace '\s*,$' # remove trailing comma that'll break syntax
        $Json += "`n" + " " * $WhiteSpacePad + "},`n"
    }
    $Json
}

function ConvertTo-STJson {
    [CmdletBinding()]
    #[OutputType([Void], [Bool], [String])]
    param(
        [AllowNull()]
        [Parameter(Mandatory=$true,
                   ValueFromPipeline=$true,
                   ValueFromPipelineByPropertyName=$true)]
        $InputObject,
        [Switch] $Compress,
        [Switch] $CoerceNumberStrings = $false)
    begin{
        $JsonOutput = ""
        $Collection = @()
        # Not optimal, but the easiest now.
        [Bool] $Script:CoerceNumberStrings = $CoerceNumberStrings
        [String] $Script:NumberRegex = '^-?\d+(?:(?:\.\d+)?(?:e[+\-]?\d+)?)?$'
        #$Script:NumberAndValueRegex = '^-?\d+(?:(?:\.\d+)?(?:e[+\-]?\d+)?)?$|^(?:true|false|null)$'
    }
    process {
        # Hacking on pipeline support ...
        if ($_) {
            Write-Verbose -Message "Adding object to `$Collection. Type of object: $($_.GetType().FullName)."
            $Collection += $_
        }
    }
    end {
        if ($Collection.Count) {
            Write-Verbose -Message "Collection count: $($Collection.Count), type of first object: $($Collection[0].GetType().FullName)."
            $JsonOutput = ConvertToJsonInternal -InputObject ($Collection | ForEach-Object { $_ })
        }
        else {
            $JsonOutput = ConvertToJsonInternal -InputObject $InputObject
        }
        if ($null -eq $JsonOutput) {
            Write-Verbose -Message "Returning `$null."
            return $null # becomes an empty string :/
        }
        elseif ($JsonOutput -is [Bool] -and $JsonOutput -eq $true) {
            Write-Verbose -Message "Returning `$true."
            [Bool] $true # doesn't preserve bool type :/ but works for comparisons against $true
        }
        elseif ($JsonOutput-is [Bool] -and $JsonOutput -eq $false) {
            Write-Verbose -Message "Returning `$false."
            [Bool] $false # doesn't preserve bool type :/ but works for comparisons against $false
        }
        elseif ($Compress) {
            Write-Verbose -Message "Compress specified."
            (
                ($JsonOutput -split "\n" | Where-Object { $_ -match '\S' }) -join "`n" `
                    -replace '^\s*|\s*,\s*$' -replace '\ *\]\ *$', ']'
            ) -replace ( # these next lines compress ...
                '(?m)^\s*("(?:\\"|[^"])+"): ((?:"(?:\\"|[^"])+")|(?:null|true|false|(?:' + `
                    $Script:NumberRegex.Trim('^$') + `
                    ')))\s*(?<Comma>,)?\s*$'), "`${1}:`${2}`${Comma}`n" `
              -replace '(?m)^\s*|\s*\z|[\r\n]+'
        }
        else {
            ($JsonOutput -split "\n" | Where-Object { $_ -match '\S' }) -join "`n" `
                -replace '^\s*|\s*,\s*$' -replace '\ *\]\ *$', ']'
        }
    }
}