Docs: std::alloc::set_alloc_error_hook clarifications

[brson]

In using this function I am wondering a few things:

• What happens if I oom while handling an oom? Is the oom handler called again? Does the process immediately abort (ala panic)?

• Does triggering oom remove the custom oom handler? The presence of take_alloc_error_hook makes it seem like that may the case, but it is not. It looks like the custom handler stays in place through its invocation. Docs would be better to mention that it's possible to oom recursively and invoke your oom handler recursively.

• Is OOM the only time that memory allocation fails? The document doesn't mention "out of memory", but the situation this handles is commonly called "OOM". The docs would better mention "out of memory" and whether there is a distinction between allocation failure types.

• Does take_alloc_error_hook restore the default error hook?

Are the answers to these questions known and intended? Please add them to the docs.

Here are some harder questions:

Can I and should I try to recover from an oom? The oom handling mechanism calls abort after calling the oom handler, but what if I panic in the handler?

I could empty my caches or otherwise mess with the allocator, panic, catch the panic, and try to continue. Is that a valid and intended use case?

[steveklabnik]

re-tagging as @rust-lang/libs ; please let me know what semantics you want documented, and then we can document them. Thanks!

[sfackler]

• What happens if I oom while handling an oom? Is the oom handler called again? Does the process immediately abort (ala panic)?

The alloc_error_hook is a little bit less "magical" than panic_hook - it just sets what code runs when you call handle_alloc_error, which is what library code does by convention when it hits an allocation failure that it doesn't want to handle itself. There are no reentrancy issues - if you call the hook from the hook, it's just a normal recursive call.

• Does triggering oom remove the custom oom handler? The presence of take_alloc_error_hook makes it seem like that may the case, but it is not. It looks like the custom handler stays in place through its invocation. Docs would be better to mention that it's possible to oom recursively and invoke your oom handler recursively.

It doesn't remove the custom handler. take_alloc_error_hook is a mirror of take_panic_hook - it's designed to allow you to "wrap" the existing hook rather than replacing it entirely.

• Is OOM the only time that memory allocation fails? The document doesn't mention "out of memory", but the situation this handles is commonly called "OOM". The docs would better mention "out of memory" and whether there is a distinction between allocation failure types.

The other possible failure situation is if you ask for a layout that the allocator can't provide. I think the main example of this is an over-aligned allocation.

• Does take_alloc_error_hook restore the default error hook?

Yep.