Powershell Errors Are Not Being Reported

[ReneGreen27]

Production environment
Windows 32

User cannot send an event via powershell project. They've tried on 3 different devices, checked that the DSN is correct. All sampling is at 1.0 and no explicit errors in their output logs other than a generic

"PS C:\WINDOWS\system32> "Something went wrong" | Out-Sentry"

Project is able to receive events successfully when sending events directly to the API rather than using powershell.

Steps to Reproduce

Send event via powershell exactly as directed in our documentation: https://docs.sentry.io/platforms/powershell/

try
{
    throw 'Test error'
}
catch
{
    $_ | Out-Sentry
}

Expected Result

Test error event to be successfully sent and received

Actual Result

No error event received

[vaind]

I can repro the issue on windows powershell 5.1. Some stack frame module path is an empty string & Test-Path throws when checking. I'll add a test case and a fix.

[vaind]

I was wrong about the cause of the issue. Instead, it looks like the event is not being sent by the HttpTransport. Still need to figure out why.

In the meantime, I suggest upgrading to PowerShell 7 if that is an option for you.

[willhcr]

After some investigation, it appears there might be an interaction between PowerShell 5.1's runtime and the .NET managed threadpool that causes background tasks to lock up or not run at all. I suspect that PowerShell 5.1 is more limited in its ability to interact with multi-threaded C# code than PowerShell 7 and that calling from C# tasks back into PowerShell code may cause deadlocks as well. I am not an expert on PowerShell internals, but here are my findings:

Workaround 1: disabling the Synchronous Worker

Commenting out the SynchronousWorker on line 55 of Start-Sentry.ps1 causes the test code to work and successfully send an event to Sentry. With debug logging enabled I can observe the HttpTransport actually transmits to Sentry after the PowerShell script has returned an exit code.

# $options.BackgroundWorker = [SynchronousWorker]::new($options)

try
{
    throw 'Test error'
}
catch
{
    $_ | Out-Sentry
}

DEBUG: [Sentry] Envelope handed off to transport (event ID: '62ef4e502d9647108fcdf2ce1fcffba8'). 1 items in queue.
PS C:\Users\user\> DEBUG: [Sentry] HttpTransport: Envelope '62ef4e502d9647108fcdf2ce1fcffba8' successfully sent. Content:

Workaround 2: Synchronous Transport

My use case is using this with Azure Automation runbooks. Sentry support would be greatly appreciated in this environment, as Azure Automation often swallows stack traces and line numbers, making debugging a challenge. The Azure Automation hybrid worker executes runbooks under a multi-threaded apartment worker process in a sandbox and it seems to terminate execution immediately upon script exit, so the previous workaround didn't work in this environment.

I have prototyped a synchronous transport class that is working under the PowerShell 5.1 Azure Automation Hybrid Worker and is compatible with the SynchronousWorker:

Add-Type -AssemblyName System.Net.Http

class SynchronousTransport : Sentry.Http.HttpTransportBase, Sentry.Extensibility.ITransport
{
    hidden [System.Net.Http.HttpClient] $httpClient

    SynchronousTransport([Sentry.SentryOptions] $options)
        : base($options)
    {
        $this.httpClient = [System.Net.Http.HttpClient]::new()
    }

    [System.Threading.Tasks.Task] SendEnvelopeAsync([Sentry.Protocol.Envelopes.Envelope] $envelope, [System.Threading.CancellationToken]$cancellationToken = [System.Threading.CancellationToken]::None)
    {
        $assembly = [Sentry.SentrySdk].Assembly
        $type = $assembly.GetType('Sentry.Http.HttpTransportBase')
        $ProcessEnvelope = $type.GetMethod('ProcessEnvelope', [System.Reflection.BindingFlags]::Instance + [System.Reflection.BindingFlags]::NonPublic + [System.Reflection.BindingFlags]::Public)
        $CreateRequest = $type.GetMethod('CreateRequest', [System.Reflection.BindingFlags]::Instance + [System.Reflection.BindingFlags]::NonPublic + [System.Reflection.BindingFlags]::Public)
        $HandleResponse = $type.GetMethod('HandleResponse', [System.Reflection.BindingFlags]::Instance + [System.Reflection.BindingFlags]::NonPublic + [System.Reflection.BindingFlags]::Public)

        $processedEnvelope = $ProcessEnvelope.Invoke($this, @($envelope))
        if ($processedEnvelope.Items.count -gt 0)
        {
            $request = $CreateRequest.Invoke($this, @($processedEnvelope))


            $headers = @{}
            foreach ($header in $request.Headers) {
                $Key = $header.Key
                $Value = $header.Value.Trim() -join ", "
                $headers[$Key] = $Value
            }

            $EnvelopeHttpContentType = $assembly.GetType('Sentry.Internal.Http.EnvelopeHttpContent')
            $SerializeToStream = $EnvelopeHttpContentType.GetMethod('SerializeToStream', [System.Reflection.BindingFlags]::Instance + [System.Reflection.BindingFlags]::NonPublic + [System.Reflection.BindingFlags]::Public)

            $memoryStream = [System.IO.MemoryStream]::new()
            $SerializeToStream.Invoke($request.Content, @($memoryStream, $null, $cancellationToken))
            $memoryStream.Position = 0

            $reader = New-Object System.IO.StreamReader($memoryStream)
            $content = $reader.ReadToEnd()
            $reader.Close()

            $psResponse = Invoke-WebRequest -Uri $request.RequestUri -Method "POST" -Headers $headers -Body $content -UseBasicParsing

            Write-Warning "Status: $($psResponse.StatusCode)" 
            Write-Warning "Content: $($psResponse.Content)"

            $response = [System.Net.Http.HttpResponseMessage]::new($psResponse.StatusCode)
            $StringContent = [System.Net.Http.StringContent]::new($psResponse.Content, [System.Text.Encoding]::UTF8, "application/json")
            $response.Content = $StringContent

            foreach ($header in $psResponse.Headers.GetEnumerator()) {
                Write-Warning "Header: $($header.Key) - $($header.Value)"
                $response.Headers.TryAddWithoutValidation($header.Key, $header.Value)
            }

            $HandleResponse.Invoke($this, @($response, $processedEnvelope))
        }

        return [System.Threading.Tasks.Task]::CompletedTask
    }
}

It is enabled by setting the transport in the SentryOptions:

Start-Sentry {
    $_.Transport = [SynchronousTransport]::new($_)
}

Example of Deadlock

Stop-Sentry and SentrySdk::Flush() seem to cause a deadlock with the SynchronousWorker enabled. The script hangs indefinitely and must be manually terminated.

try
{
    throw 'Test error'
}
catch
{
    $_ | Out-Sentry 5>&1
}
finally {
    Stop-Sentry
}

Hope this helps and thank you for getting this project going. This will be a valuable tool for debugging in Azure Automation environments.

[vaind]

With debug logging enabled I can observe the HttpTransport actually transmits to Sentry after the PowerShell script has returned an exit code.

Yes and that's actually why I've added the synchronous worker. The problem is that when a script is launched outside terminal window (e.g. in its own shell), it won't get transfered before the script (and the shell) exits.

As you've actually found out yourself, reeding the rest of your comment:

seems to terminate execution immediately upon script exit, so the previous workaround didn't work in this environment.

[vaind]

I have prototyped a synchronous transport class that is working under the PowerShell 5.1 Azure Automation Hybrid Worker and is compatible with the SynchronousWorker

would you like to give a PR a try? If not, I'll try it once I find some time if you're fine with me using your code and adding it to the repo (a.k.a if your code has MIT compatible OSS license :) )