pod_exec() yields inconsistent results

[plup]

What happened (please include outputs or screenshots):

I'm trying to implement this code sample: https://github.com/kubernetes-client/python/blob/master/examples/pod_exec.py#L69-L82

But running multiple times the same command returns different results.

What you expected to happen:

I'm expecting all command execution to return the same result

How to reproduce it (as minimally and precisely as possible):

Create a test.py

from kubernetes import client, config
from kubernetes.stream import stream

config.load_kube_config()
r = stream(client.CoreV1Api().connect_get_namespaced_pod_exec, name='mypod', namespace='default',
           container='mgmt', command=['false'], stderr=True, stdin=False, stdout=True, tty=False,
           _preload_content=False)

r.run_forever()
print(vars(r))
print(r.returncode)

Run it multiple times:

$ for i in {1..10}; do python test.py; sleep 2; done
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x103325840>, 'sock': <websocket._core.WebSocket object at 0x103364d70>, '_returncode': None}
0
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x105c89840>, 'sock': <websocket._core.WebSocket object at 0x105cc8d70>, '_returncode': None}
0
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x104ead840>, 'sock': <websocket._core.WebSocket object at 0x104eecd70>, '_returncode': None}
0
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x103115840>, 'sock': <websocket._core.WebSocket object at 0x103154d70>, '_returncode': None}
0
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x1079cd840>, 'sock': <websocket._core.WebSocket object at 0x107a0cd70>, '_returncode': None}
0
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"command terminated with non-zero exit code: command terminated with exit code 1","reason":"NonZeroExitCode","details":{"causes":[{"reason":"ExitCode","message":"1"}]}}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x105019840>, 'sock': <websocket._core.WebSocket object at 0x105058d70>, '_returncode': None}
1
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x10742d840>, 'sock': <websocket._core.WebSocket object at 0x107468d70>, '_returncode': None}
0
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x105dcd840>, 'sock': <websocket._core.WebSocket object at 0x105e08d70>, '_returncode': None}
0
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"command terminated with non-zero exit code: command terminated with exit code 1","reason":"NonZeroExitCode","details":{"causes":[{"reason":"ExitCode","message":"1"}]}}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x105ae5840>, 'sock': <websocket._core.WebSocket object at 0x105b24d70>, '_returncode': None}
1
{'_connected': False, '_channels': {3: '{"kind":"Status","apiVersion":"v1","metadata":{},"status":"Success"}\n'}, 'binary': False, 'newline': '\n', '_all': <_io.StringIO object at 0x1076a5840>, 'sock': <websocket._core.WebSocket object at 0x1076e4d70>, '_returncode': None}
0

All those runs should return a NonZeroExit status.

Anything else we need to know?:

Environment:

• Kubernetes version (kubectl version): Client Version: v1.28.13
  Kustomize Version: v5.0.4-0.20230601165947-6ce0bf390ce3
  Server Version: v1.29.11
• OS (e.g., MacOS 10.13.6): MacOS Sequoia 15.2
• Python version (python --version): Python 3.13.1
• Python client version (pip list | grep kubernetes): kubernetes 31.0.0

[plup]

Looking at the code, it's very possible that it's a platform dependent issue.

[yliaog]

/help

[k8s-ci-robot]

@yliaog:
This request has been marked as needing help from a contributor.

Guidelines

Please ensure that the issue body includes answers to the following questions:

• Why are we solving this issue?
• To address this issue, are there any code changes? If there are code changes, what needs to be done in the code and what places can the assignee treat as reference points?
• Does this issue have zero to low barrier of entry?
• How can the assignee reach out to you for help?

For more details on the requirements of such an issue, please see here and ensure that they are met.

If this request no longer meets these requirements, the label can be removed
by commenting with the /remove-help command.

In response to this:

/help

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[plup]

Adding to the issue, I tried the same code with client/server matching versions and got the some results.

$ pip list | grep kubernetes
kubernetes         29.0.0

$ kubectl version
Server Version: v1.29.11

[plup]

Running some more tests, I observed that the pod_exec command returns a status=0 most of the time, even when the pod doesn't exist.