Skip to content

[Distributed] Improve getting return type metadata for distributed invocations #79381

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Feb 15, 2025

Conversation

ktoso
Copy link
Contributor

@ktoso ktoso commented Feb 14, 2025

Sometimes we may get incomplete type metadata when using the swift_func_getReturnTypeInfo function, which then could result in a crash when trying to return a result of such type from a distributed func. This happens very rarely, only if the type metadata has never been fully loaded in the process, but can happen and then cause a crash during returning from a remote call distributed method.

I was not able to reproduce the problem in the test since we always end up "touching" the type, even though I tried to avoid that by doing the remote call directly 🤔

Resolves: rdar://141313340

@ktoso ktoso requested a review from xedin February 14, 2025 02:12
@ktoso
Copy link
Contributor Author

ktoso commented Feb 14, 2025

@swift-ci please test

@ktoso
Copy link
Contributor Author

ktoso commented Feb 14, 2025

Unrelated failure: FAIL: Swift-validation(linux-x86_64) :: SILOptimizer/large_nested_array.swift.gyb (18915 of 18915)

@ktoso
Copy link
Contributor Author

ktoso commented Feb 14, 2025

@swift-ci please test linux platform

@al45tair
Copy link
Contributor

I was not able to reproduce the problem in the test since we always end up "touching" the type, even though I tried to avoid that by doing the remote call directly 🤔

Could we maybe split the test into two separate executables and actually do a remote call, say using a pipe between them? Would that help? It'd be better if we could reproduce the problem and prove the change fixes it, if that's possible.

@mikeash
Copy link
Contributor

mikeash commented Feb 14, 2025

The issue is actually that the original metadata request always comes back complete, but requests from other threads may return early, so it needs a multithreaded test to work. We can avoid the need for separate processes by calling swift_func_getReturnTypeInfo directly. I think I've got a test that hammers it enough to reliably catch this problem.

@ktoso ktoso force-pushed the wip-correct-metadata-lookup-result-type branch from 3a998d7 to d8f89bd Compare February 15, 2025 09:16
@ktoso
Copy link
Contributor Author

ktoso commented Feb 15, 2025

We managed to form a test which reproduces reliably, thank you @mikeash !

I also corrected the impl in one more way thanks to this test -- we cannot do non-blocking metadata requests as that's how we can get those zero sizes as well.

@ktoso
Copy link
Contributor Author

ktoso commented Feb 15, 2025

@swift-ci please test

@ktoso ktoso enabled auto-merge February 15, 2025 09:18
@ktoso ktoso merged commit 6ef0a93 into swiftlang:main Feb 15, 2025
4 of 5 checks passed
@ktoso ktoso deleted the wip-correct-metadata-lookup-result-type branch February 17, 2025 02:47
ktoso added a commit to ktoso/swift that referenced this pull request Apr 26, 2025
We had fixed this bug in swiftlang#79381
but missed to realize the same problem existed for parameters as well.

This corrects the swift_func_getParameterTypeInfo impl, and also removes
the entire "unsafe" method, we no longer use it anywhere.

Resolves rdar://146679254
ktoso added a commit to ktoso/swift that referenced this pull request Apr 26, 2025
We had fixed this bug in swiftlang#79381
but missed to realize the same problem existed for parameters as well.

This corrects the swift_func_getParameterTypeInfo impl, and also removes
the entire "unsafe" method, we no longer use it anywhere.

Resolves rdar://146679254
ktoso added a commit to ktoso/swift that referenced this pull request Apr 26, 2025
We had fixed this bug in swiftlang#79381
but missed to realize the same problem existed for parameters as well.

This corrects the swift_func_getParameterTypeInfo impl, and also removes
the entire "unsafe" method, we no longer use it anywhere.

Resolves rdar://146679254
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants