Integrations: Ariadne - `handled` tag is always `False`

[bmarkovicvega]

How do you use Sentry?

Sentry Saas (sentry.io)

Version

1.42.0

Steps to Reproduce

1. Define graphql query:

extend type Query {
    testQuery: testPayload!
}

type TestResult {
    field1: Boolean
}

type TestPayload {
    status: Boolean!
    result: TestResult
    errors: [Error]
}

2. Execute the query with requesting data not defined in TestResult:

query { 
    testQuery{
        resut{
            non_existing_field
        }
    }
}

note: AriadneIntegration() is added upon sentry_sdk.init()

Expected Result

Error is reported in sentry with flag -> handled: True

Actual Result

Error is reported in sentry with flag -> handled: False

Why the error is reported as not handled when it seems that it's handled by library and a proper message is received
Message received and reported:

Cannot query field 'non_existing field' on type 'TestResult'

[sentrivana]

Hey @bmarkovicvega, thanks for writing in. Looks like in the Ariadne integration we report all error events as unhandled, but this also seems to be a pattern in other similar integrations.

From your perspective as someone who uses the integration, what would you consider to be an unhandled Ariadne error? Would you prefer all Ariadne errors raised by Sentry to be marked as handled?

[bmarkovicvega]

Hey @bmarkovicvega, thanks for writing in. Looks like in the Ariadne integration we report all error events as unhandled, but this also seems to be a pattern in other similar integrations.

From your perspective as someone who uses the integration, what would you consider to be an unhandled Ariadne error? Would you prefer all Ariadne errors raised by Sentry to be marked as handled?

Hi @sentrivana thanks for answering. What I would expect, from my perspective, is that unhandled error would be the error raised by the code I've added, but for errors handled by ariadne library such as one I mentioned in the issue I would make them handled or even I wouldn't consider them as error that should be logged and reported in sentry.
I do request with non existing field, I got message that it's not existing withing the certain query, for me it seems handled.

Giving an example and making a parallel with the REST endpoint: if we do POST /users that requires username and password and if I don't pass password which is required I will get a response with http status 400 with a usual validation error message right, this doesn't seem like unhandled error that should be reported in sentry (at least it looks like handled)

[sentrivana]

Thanks @bmarkovicvega, that makes sense to me. For web frameworks we usually don't report client (4xx) errors at all since those should be reported by the client side, so it'd make sense to do something analogous for our GraphQL integrations as well. What's at the moment not clear to me is how we can categorize the errors we see e.g. in Ariadne into client and server errors, this needs some investigation into what the different errors look like and how we can tell them apart.

In the meantime though, if you find the errors not actionable, you can always define a custom before_send (docs) where you either drop the error altogether (by returning None) or you manually change the handled field.

[bmarkovicvega]

Thanks @bmarkovicvega, that makes sense to me. For web frameworks we usually don't report client (4xx) errors at all since those should be reported by the client side, so it'd make sense to do something analogous for our GraphQL integrations as well. What's at the moment not clear to me is how we can categorize the errors we see e.g. in Ariadne into client and server errors, this needs some investigation into what the different errors look like and how we can tell them apart.

In the meantime though, if you find the errors not actionable, you can always define a custom before_send (docs) where you either drop the error altogether (by returning None) or you manually change the handled field.

I appreciate you have looked at the issue and discussed it @sentrivana. Thanks for your time and thoughts about it. Also, thank you for giving a suggestion how to solve it for now.