Automatic Audio Feedback on Silence in VoicePipeline

[joowon-dm-snu]

Please read this first

• Have you read the docs?Agents SDK docs
• Have you searched for related issues? Others may have had similar requests

Describe the feature

While working on the VoicePipeline—as with my previous issue #488 —I’ve identified a scenario where the voice agent may become unresponsive due to inadequate speech recognition.
In practice, even if a user speaks, there are instances when the transcription session (transcription_session.transcribe_turns()) does not get triggered, resulting in an extended period of silence.

To mitigate this issue, I propose that when a turn concludes, a separate thread should be initiated. Within this thread, if no new turn is started within 3 seconds, the system should automatically stream a fallback message, such as:

"Excuse me, it seems the audio was interrupted. Could you please repeat that?"

This feature would provide a smoother conversational experience by proactively addressing potential audio dropouts or misrecognitions.

[joowon-dm-snu]

It seems possible to use the SingleAgentWorkflowCallbacks feature for this, but since the turn management is handled on the Pipeline side, I thought it would be more appropriate to modify the VoicePipeline instead.