Skip to content

gh-133485: Use _interpreters.call() in InterpreterPoolExecutor #133957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

ericsnowcurrently
Copy link
Member

@ericsnowcurrently ericsnowcurrently commented May 13, 2025

Most importantly, this resolves the issues with functions and types defined in __main__.
It also expands the number of supported objects.

(This is based on gh-133484, thus only the last commit.)

@ericsnowcurrently ericsnowcurrently added needs backport to 3.14 bugs and security fixes and removed awaiting core review labels May 13, 2025
@ericsnowcurrently ericsnowcurrently force-pushed the interp-pool-executor-use-interp-call branch from 5340a57 to 62d7c2c Compare May 13, 2025 01:13
@ericsnowcurrently ericsnowcurrently marked this pull request as draft May 13, 2025 01:14
@ericsnowcurrently ericsnowcurrently force-pushed the interp-pool-executor-use-interp-call branch 9 times, most recently from b3c2477 to 7697c11 Compare May 27, 2025 16:31
Comment on lines +23 to +26
# InterpreterPoolInitializerTest.test_initializer fails
# if we don't have a LOAD_GLOBAL. (It could be any global.)
# We will address this separately.
INITIALIZER_STATUS
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@markshannon, any ideas on why this is happening? It smells like a ceval bug, but it certainly could be something I've done wrong.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor

@neonene neonene Jun 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There seem to be related changes in inspect.getclosurevars() since 83ba8c2:

before:
ClosureVars(nonlocals={},
            globals={'INITIALIZER_STATUS': 'uninitialized'},
            builtins={}, unbound=set())
after:
ClosureVars(nonlocals={},
            globals={},
            builtins={}, unbound=set())
  • init() on main (without L26):
  3           RESUME                   0

  5           LOAD_FAST_BORROW         0 (x)
              STORE_GLOBAL             0 (INITIALIZER_STATUS)
              LOAD_CONST               0 (None)
              RETURN_VALUE
  • 3.3.5 (2014):
  5           0 LOAD_FAST                0 (x)
              3 STORE_GLOBAL             0 (INITIALIZER_STATUS)
              6 LOAD_CONST               0 (None)
              9 RETURN_VALUE

@ericsnowcurrently ericsnowcurrently force-pushed the interp-pool-executor-use-interp-call branch from 5ea2bb2 to 14a8eb9 Compare May 29, 2025 21:02
@ericsnowcurrently ericsnowcurrently force-pushed the interp-pool-executor-use-interp-call branch from 14a8eb9 to ccc135c Compare May 30, 2025 15:27
@ericsnowcurrently ericsnowcurrently marked this pull request as ready for review May 30, 2025 21:32
@ericsnowcurrently ericsnowcurrently added the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 30, 2025
@bedevere-bot
Copy link

🤖 New build scheduled with the buildbot fleet by @ericsnowcurrently for commit ccc135c 🤖

Results will be shown at:

https://buildbot.python.org/all/#/grid?branch=refs%2Fpull%2F133957%2Fmerge

If you want to schedule another build, you need to add the 🔨 test-with-buildbots label again.

@bedevere-bot bedevere-bot removed the 🔨 test-with-buildbots Test PR w/ buildbots; report in status section label May 30, 2025
@neonene
Copy link
Contributor

neonene commented May 31, 2025

The wasm32-wasi Non-Debug buildbot seems to be unused on merged commits and out of order on PRs. For example: https://buildbot.python.org/#/builders/1373/builds/507

@neonene
Copy link
Contributor

neonene commented Jun 1, 2025

Is the following usage invalid?

from concurrent.futures import InterpreterPoolExecutor
INITIALIZER_STATUS = 'uninitialized'

def init(x):
    global INITIALIZER_STATUS
    INITIALIZER_STATUS = x
    INITIALIZER_STATUS

def get_init_status():
    return INITIALIZER_STATUS

if __name__ == "__main__":
    exe = InterpreterPoolExecutor(initializer=init, initargs=('initialized',))
    f = exe.submit(get_init_status)
    assert f.result() == 'initialized'  # fails
    exe.shutdown(wait=True)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants