panic runtime and C-unwind documentation

[BatmanAoD]

Tracking issue: rust-lang/rust#74990

[BatmanAoD]

Hm... not sure how to fix the links to the newly-introduced page. Is there an index page I need to edit?

Edit: I think I found it

[nikomatsakis]

Suggested change

	| panic runtime | ABI | `panic`-unwind | Unforced foreign unwind |
	| -------------- | ------------ | ------------------------------------- | ----------------------- |
	| `panic=unwind` | `"C-unwind"` | unwind | unwind |
	| `panic=unwind` | `"C"` | abort | UB |
	| `panic=abort` | `"C-unwind"` | `panic!` aborts | abort |
	| `panic=abort` | `"C"` | `panic!` aborts (no unwinding occurs) | UB |
	| panic runtime | ABI | `panic`-unwind | Unforced foreign unwind |
	| -------------- | ------------ | ------------------------------------- | ----------------------- |
	| `panic=unwind` | `"C-unwind"` | unwind | unwind |
	| `panic=unwind` | `"C"` | abort if unwinding reaches the function | UB if unwinding reaches the function |
	| `panic=abort` | `"C-unwind"` | aborts immediately (no unwinding occurs) | abort if unwinding reaches the function |
	| `panic=abort` | `"C"` | aborts immediately (no unwinding occurs) | UB if unwinding reaches the function |

I found this a bit confusing. I believe there are subtle differences in terms of where the aborts occur and so forth. I have tried to clarify above, but I think it may be worth further clarifying.

It may also be worth adding some (perhaps non-normative) discussion of implementation:

• When compiling a function F with panic=unwind and extern "C", the compiler inserts unwinding guards for Rust panics that trigger an abort when unwinding reaches F.

I am also be misunderstanding what's going on. I was a bit surprised to see "UB" for unforced-foreign-unwind with C=unwind. I guess that this table is combining two scenarios:

• what happens when you call a C++ function declared as extern "C", and it unwinds (UB, we haven't compiled any guards)
• what happens when an extern "C" Rust function invokes some C++ function that throws (probably, in practice, an abort, but perhaps we have simplified to call it UB?)

Is that right?

[nbdd0121]

It's only UB for a foreign function declared as extern "C" to unwind.

[nikomatsakis]

@nbdd0121 what happens when an extern "C" Rust function unwinds? I believe we insert an abort guard, but this table doesn't clarify that, right? Or maybe I don't understand what it's trying to convey. I'm imagining a scenario like

extern "C-unwind" fn throws();

extern "C" fn rust_fn() {
    throws(); // unwinds
}

In this case, I presume you get an abort -- and I think we guarantee that? But the way I read this table, it would be listed as UB.

[BatmanAoD]

Hm....I don't know if the panic abort guard would currently catch and abort in that case, or if it relies on the personality function to only abort on true Rust panics. I agree that the behavior in the table as-written is UB.

[nbdd0121]

@nbdd0121 what happens when an extern "C" Rust function unwinds? I believe we insert an abort guard, but this table doesn't clarify that, right? Or maybe I don't understand what it's trying to convey. I'm imagining a scenario like

extern "C-unwind" fn throws();

extern "C" fn rust_fn() {
    throws(); // unwinds
}

In this case, I presume you get an abort -- and I think we guarantee that? But the way I read this table, it would be listed as UB.

Unwinding out from extern "C" functions (defined in either Rust or foreign language) is UB.
In the case you listed, we insert guard to prevent unwinding from actually leaving a Rust extern "C" functions, therefore the function does not unwind, so UB is prevented; in this case we never unwinds out from a extern "C" Rust functions.

If you define a extern "C-unwind" Rust function and transmute it to extern "C" and then call it, it's not UB if unwinding does not happen, and it's UB if unwinding happens.

[BatmanAoD]

@nikomatsakis With the change to the verbiage above, explaining that the table entries are specifically describing behavior at function boundaries, do you still want to make a change here?

[daira]

Please check whether the notes I suggested to add under the table are correct.

[BatmanAoD]

@nikomatsakis Can I resolve this comment thread now?

[BatmanAoD]

Sorry for the delay; I think I've addressed all comments.

[BatmanAoD]

@tmandry @nikomatsakis I'm not sure you saw my comments & changes last week, but I think this is ready for re-review.

[nbdd0121]

Could you squash the commits?

[BatmanAoD]

@nbdd0121 Can that be done on merge? I've heard that GitHub sometimes has trouble with PR branches that receive force-pushes.

[BatmanAoD]

I think I've resolved all open questions and concerns. Is there anything else needed from me at the moment?

[nbdd0121]

This really needs rebasing now.

[BatmanAoD]

@nbdd0121 Done!

[BatmanAoD]

@tmandry two changes since your review:

• I added the new ABIs to items/external-blocks
• I added that catching an exception or unwind from the "wrong" language is UB

[BatmanAoD]

@tmandry @ehuss can this be merged, since the partial stabilization was a while ago now?

[RalfJung]

You changed the meaning of the text by making this about the final binary. I am not sure whether this is correct. I'd prefer if you could just restore the old wording, which was AFAIK reviewed by a bunch of people, and leave further improvements to a later PR so that the diff over the previous consensus can be more easily reviewed.

[RalfJung]

In fact I am fairly sure your new wording is wrong. If the unwinding comes from an external crate, enters through a -Cpanic=unwind crate, and then propagates into a -Cpanic=abort crate, that is UB, even if the binary is -Cpanic=unwind.

[BatmanAoD]

We certainly talked about the original text a lot, but my impression was that you still considered it confusing. I started the change because it seemed related to the entirely-new "foreign linking" section, but seemed even more confusing in its original form when juxtaposed with the new section.

I believe the rule has always been about whether or not the "abort" runtime is linked, and since only one panic runtime can be linked in a final Rust binary, I don't understand how the introduction of the word "final" changes this. I can remove the word "final", though.

A panic=abort crate will have shims for aborting the process in the presence of unwinding at extern boundaries. If there's no extern boundary, then the two crates are compiled as source code so they can't have different panic modes in the first place.

[RalfJung]

If there's no extern boundary, then the two crates are compiled as source code so they can't have different panic modes in the first place.

That's not right. You can link together different crates with different flags.

[RalfJung]

The old text was:

No crate may be linked with [the `panic=abort` runtime][panic-runtime] if it has
both of the following characteristics:
* It contains a call to an `-unwind` foreign function or function pointer
* It was compiled with `panic=unwind`

Actually looking at this I think there is a mistake here -- the problematic case is linking a crate with the "unwind" runtime, but compiling it with panic=abort, isn't it?

panic=abort sets a bunch of "nounwind" flags, and ultimately what this here is all about is preventing those from causing UB. So for a concrete example:

• crate A is panic=unwind, calls some C-unwind function from a "Rust" ABI function
• crate B is panic=abort, calls crate A and marks that call nounwind
• now we have UB if that C-unwind call ever unwinds

It doesn't even matter which panic runtime is linked in, I think? This is about panics from the outside world after all.

The panic runtime would come in in a situation like this:

• crate A is panic=abort, calls e.g. unwrap and marks that call with nounwind since it has the "Rust" ABI
• the panic_unwind runtime is linked in
• now we have UB if that unwrap ever fails

Not sure where we say that that is UB.

[chorman0773]

Suggested change

	* The final binary is linked with [the `panic=abort` runtime][panic-runtime]
	* The program unwinds into a rust crate compiled with `panic=abort`

Would this be correct?

[RalfJung]

Yes that sounds right. Cc @nbdd0121 to be sure.

[RalfJung]

But also, don't we have to say somewhere that if a crate is built with panic=abort, and later linked with the panic=unwind runtime, that's UB too?

[RalfJung]

In my view this is resolved by my later commits, but someone should take a look.

[chorman0773]

I added the identifiers to the panic chapter.

[chorman0773]

Should be ready other than the outstanding issue in linkage.

[RalfJung]

This comment also likely needs updating, since it's not about the panic=abort runtime, it's about mixing panic=abort and panic=unwind crates irrespective of which runtime is ultimately linked in.

[RalfJung]

I don't get the last sentence.

Also, this should explain the gap in this rustc check. Clearly there's a gap, otherwise we wouldn't have to document this UB.

[BatmanAoD]

The UB is only possible when using a linker other than rustc (a "foreign linker" as described in the section above), so AFAIK there is no gap in rustc itself.

This sentence explains why the Rust ABI is not included in the lint, which is what you were originally asking about before I changed the verbiage.

[RalfJung]

Ah, that's existing text so I missed it. It should probably mention dlopen-like situations? Even if not mixing Rust an non-Rust code, one may be using a non-rustc linker.

The entire section should then probably say:

If you are using a foreign linker, the following requirements must be upheld: [...]

And then the note can just say that when using rustc as a linker, those requirements are enforced automatically.

This sentence explains why the Rust ABI is not included in the lint, which is what you were originally asking about before I changed the verbiage.

This section isn't about the lint though? It's about a hard error on panic option mismatches. That one can't rely on what is "expected" to happen, it should be sound. Did the sentence end up in the wrong paragraph?

[chorman0773]

That will have to wait for @BatmanAoD then.

[RalfJung]

I have pushed proposed wording to https://github.com/RalfJung/reference/commits/c-unwind-documentation/. Unfortunately I can't push to this PR.
(EDIT: I deleted that branch and added the commits in this PR as @BatmanAoD gave me access, thanks a lot. :)

I sure hope what that text says is correct, but it would be good if @BatmanAoD or @nbdd0121 could take a look. :)

[BatmanAoD]

I don't have time right now to look, but I've added both you and Gary Guo as collaborators on my fork. I do hope to get confirmation from him because he understands this edge condition better than I do, I think.

[RalfJung]

Do we have a name for "the final artifact of Rust linkage, with a single Rust runtime included"? Always saying "Rust binary or staticlib" is awkward...

[nbdd0121]

I believe this also needs to account for cdylib?

[RalfJung]

Yeah we really need a proper term for this, not an enumeration. "final Rust artifact"? Or can we say "Rust binary" since staticlib and cdylib are binaries but in some sense rlib is not? Not sure if that makes any sense...

[RalfJung]

@bjorn3 maybe you have a good idea for what to call this?

[bjorn3]

I don't have a great name for it either unfortunately. bin, dylib, cdylib and proc-macro are all linked crates, but staticlib isn't.

[RalfJung]

I decided to just directly talk about artifacts and whether they share the same copy of the Rust runtime.

[RalfJung]

@traviscross @ehuss @chorman0773 I think that resolves all concerns, though it wouldn't hurt if someone could read over this to make sure it is still coherent.

[RalfJung]

@nbdd0121 could you fact-check this?

[nbdd0121]

Looks correct to me.

[traviscross]

@rustbot labels +I-lang-nominated

Given all the attention and iteration this has received, we should probably re-review this en banc on the lang side to confirm.

[ehuss]

Some questions:

• Does the clobber_abi list need to mention the -unwind variants? Sorry, I don't really know much about that. If in all cases they also support their -unwind variants, maybe that could just be stated instead of explicitly listing them all?

Before merging, I'd like to make a few changes (I can do this later to avoid churn now):

• Unwrap the lines. Per our style guide, the content should not be wrapped.
• Move most of the rustc-specific text into note blocks (CLI flags and such). We generally don't refer to those in the body of the reference. (The linkage chapter is a little odd in this regard, but has been intended to be reworked.)

[ehuss]

This links to the "ABI" chapter, which unfortunately doesn't really document the ABI yet. IIUC, this is referring to calling conventions, which I believe is documented at https://doc.rust-lang.org/nightly/reference/items/external-blocks.html#abi. Would it make sense to link there instead?

[chorman0773]

I should actually fix that up in the ABI PR anyways, since the rule would be calling a function with an incompatible signature (which includes incompatible ABI tags).

[RalfJung]

Any two types with size 0 and alignment 1 are ABI-compatible.

I would say so, yes.

[ehuss]

Is there something this can link to that explicitly says what frames don't allow unwinding?

[ehuss]

I'm wondering if it is strictly true that runtime assumptions can only be violated outside of Rust? For example, #[no_mangle] pub fn malloc() -> usize { 1 } is a classic example of something in safe-rust that breaks the runtime. Is that sentence necessary?

[RalfJung]

We have to tell people how they can avoid breaking assumptions of the Rust runtime, in lieu of an exact list of what those assumptions are. So If we remove this sentence this item becomes very non-helpful: we are telling people to not do something without giving them any chance of knowing whether they are doing it. So I think the sentence is necessary.

Note that your example uses an unsafe attribute, so there should be a warning -- or is that lint still allow-by-default?

[ehuss]

Maybe I wasn't being clear? I was specifically referring to "This is only possible using mechanisms outside Rust." This sentence doesn't say what any assumptions are, and doesn't tell the user what to do or not do. But it still seems to be making a strong assertion that doesn't seem true to me. I would say it is UB to violate the assumptions of the Rust runtime whether it is done in Rust code or outside it. What is the intent of trying to say you can't violate it in Rust?

unsafe-attr errors are only in the 2024 edition.

[RalfJung]

Ah, somehow I misread and I thought this talked about violations not being possible in safe Rust.

Or is that redundant because "obviously" you cannot cause UB in safe Rust?

[ehuss]

I'm a little uncertain about the wording here. An unwind can be caught inside main, correct? That is, after it reaches it?

Maybe this could be reworded to make that clearer? Maybe something like this:

Suggested change

	in Rust code compiled or linked with a different runtime) is not caught before
	reaching the `main` function, the process will be safely terminated. This may
	in Rust code compiled or linked with a different runtime) reaches `main` and is not caught,
	the process will be safely terminated. This may

[RalfJung]

I would just replace "reaches main" by "leaves main" or "propagates beyond main" or so.

[ehuss]

Maybe something that is a little more agnostic to the exact function name (since there are other things like ManuallyDrop)?

Suggested change

	### `forget`
	### Manually preventing destructors

[ehuss]

Although we don't have it written down, yet, we are likely going to stick with the policy of not documenting older behavior.

Suggested change

	> [!NOTE]
	> Prior to Rust 1.82, it is possible for only some of the destructors to execute, and not others.

[ehuss]

There are two concepts that I feel like are getting lost on me here:

• Which panic strategy the crate is compiled for (-Cpanic)
• Which panic runtime is linked into the resulting binary (panic_unwind or panic_abort)

I don't see the first concept being explicitly described. I'm wondering if it would make sense to move this section up above the Unwinding section, and also introduce the term panic strategy to explicitly refer to the first concept?

Could we also include a short blurb here just to make it clear how to control the panic runtime? Maybe something like:

Suggested change

	The actual behavior and implementation of `panic!` is controlled by the _panic
	runtime_.
	The actual behavior and implementation of `panic!` is controlled by the _panic
	runtime_, which is chosen at compile time when the final binary is linked.

or..I dunno, that's a little sloppy. But something along those lines to give a vague picture of how the runtime is chosen?

[ehuss]

I don't know what it means to unwind just like C++'s throw. Is there somewhere specific in the C++ spec that we can link or refer to that describes what this means?

Should this be defined in regard to a specific version of C++ (like C++23)?

How much of this is specific to particular C++ implementations? Does this need to be defined in terms of "like the platform's dominant C++ compiler implements it" like we do with the "C" repr?

[ehuss]

I think to clarify here, I believe this also includes an uncaught exceptions in threads?

Suggested change

	`catch_unwind`, `JoinHandle::join`, or by letting it propagate all the way to a
	Rust `main()` function will have one of two behaviors, and it is unspecified
	`catch_unwind`, `JoinHandle::join`, or by letting it propagate all the way to a
	thread root or past a Rust `main()` function will have one of two behaviors, and it is unspecified

[RalfJung]

FWIW I am not sure who is currently "in charge" of this PR. I won't have the time to do significant further edits here. I will just note that this has gone through an insane amount of revisions and at some point we should land something to document this long-stable feature rather then polishing it for another 6 months.

[ehuss]

I can help with edits in collaboration with others if @BatmanAoD or anyone else isn't available, but it is helpful to get some questions answered, especially in terms of correctness. There are some incorrect things here, and things that aren't clear to me, so it would be helpful to get some understanding. Some things can be deferred (like clobber_abi or anything that isn't complete), but I would prefer to not add things that seem wrong to me.

I very much understand the pain and struggle of trying to land something, especially after it has gone through several rounds of review, and has been open for several years. I apologize for that. I wanted to let the lang team go through it first, since to me it seems like a significant change to specify what we guarantee about panics, and I wanted to know what they wanted to commit for it. I should have been more active earlier on.