asyncio.TaskGroup corrupts uncancel() stack when the parent task replaces the cancelled error

[graingert]

Bug report

import asyncio

class MyException(Exception):
    pass

async def async_fn():
    await asyncio.sleep(0)
    raise MyException

async def main():
    task = asyncio.current_task()

    try:
        async with asyncio.TaskGroup() as tg:
            tg.create_task(async_fn())
            try:
                await asyncio.sleep(1)
            except asyncio.CancelledError:
                pass
    except* MyException:
        print("done!")

    print(f"{task.cancelling()=} should be 0")

asyncio.run(main())

[graingert]

this is also quite easy to trigger in typical task group code:

import asyncio
import contextlib

class ConnectionClosedError(Exception):
    pass


async def poll_database(database):
    raise ConnectionClosedError


async def close_connection():
    raise ConnectionClosedError


@contextlib.asynccontextmanager
async def database():
    try:
        yield
    finally:
        await close_connection()


async def main():
    task = asyncio.current_task()

    try:
        async with asyncio.TaskGroup() as tg:
            async with database() as db:
                tg.create_task(poll_database(db))
                await asyncio.sleep(1)
    except* ConnectionClosedError:
        print("done!")

    print(f"{task.cancelling()=} should be 0")


asyncio.run(main())

[graingert]

cc @ambv who helped me find this

[graingert]

Tagging @pablogsal as potential release-blocker

[YvesDup]

Hello, sorry for my question but I have never seen a * after an except statement. I couldn't find an explanation on the web. What does it mean, please?

In your first example, if you replace pass with raise, you have the expected result.
May be this will help you ?

[graingert]

Hello, sorry for my question but I have never seen a * after an except statement. I couldn't find an explanation on the web. What does it mean, please?

Hi there - it's the new 3.11 ExceptionGroup feature https://peps.python.org/pep-0654/ it's required to handle an issue where multiple errors can happen concurrently. Eg a connection error canceles a group of concurrent operations, and the cancellation error leads to some teardown that also fails. In 3.10 library authors had to either choose an error to raise or wrap them in a list and raise a generic "MultiError"

[YvesDup]

Hi @graingert, thank you for your response about except * feature.

About the issue, I read the documentation about 'Task groups' and IMHO there is a part about the two cases, in the second paragraph below the exemple:

... At this point, if the body of the async with statement is still active (__aexit__() hasn’t been called yet), the task directly containing the async with statement is also cancelled.

Do you think that your both examples match the situation described ?

[ambv]

@YvesDup, you stopped your quote just before the sentence that describes why what Thomas reported is a bug:

The resulting asyncio.CancelledError will interrupt an await, but it will not bubble out of the containing async with statement.

In other words, after leaving async with we want print(f"{task.cancelling()=} should be 0").

[kumaraditya303]

Have you tried this on the edgedb's task group ¹ implementation from which the current one is derived from?

Footnotes

1. https://github.com/edgedb/edgedb/blob/master/edb/common/taskgroup.py ↩