Skip to content

Streaming Fails Due to {"include_usage": True} #442

Open
@sixwaaaay

Description

@sixwaaaay

Description
When using Mistral AI's OpenAI-compatible API with streaming output Runner.run_streamed, the hardcoded stream_options={"include_usage": True} in the code triggers a 422 Unprocessable Entity error. Mistral's API does not support the include_usage parameter in stream_options, causing compatibility issues.

Root Cause:

stream_options={"include_usage": True} if stream else NOT_GIVEN,

Proposed Fix
Add an override mechanism for stream_options

Error

openai.UnprocessableEntityError: Error code: 422 - {'object': 'error', 'message': {'detail': [{'type': 'extra_forbidden', 'loc': ['body', 'stream_options', 'include_usage'], 'msg': 'Extra inputs are not permitted', 'input': True}]}, 'type': 'invalid_request_error', 'param': None, 'code': None}

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions