resolve: Report more visibility-related early resolution ambiguities for imports

[petrochenkov]

The new ambiguities are reported when the import's visibility is ambiguous and may depend on the resolution/expansion order.

Detailed description: #149596 (comment).

[rustbot]

r? @lcnr

rustbot has assigned @lcnr.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

[petrochenkov]

@bors try

[petrochenkov]

@craterbot run mode=check-only p=1 crates=https://crater-reports.s3.amazonaws.com/pr-149145/retry-regressed-list.txt

[rust-bors]

☀️ Try build successful (CI)
Build commit: ce3348c (ce3348c21531b911ca5aadbbbe7ad63a1bc4c382, parent: 8202d110516c6bd87cee375e9a9b774db499015c)

[petrochenkov]

@craterbot run mode=check-only p=1 crates=https://crater-reports.s3.amazonaws.com/pr-149145/retry-regressed-list.txt

[craterbot]

👌 Experiment pr-149596 created and queued.
🤖 Automatically detected try build ce3348c
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🚧 Experiment pr-149596 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

[craterbot]

🎉 Experiment pr-149596 is completed!
📊 359 regressed and 0 fixed (3583 total)
📊 218 spurious results on the retry-regessed-list.txt, consider a retry¹ if this is a significant amount.
📰 Open the summary report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

Footnotes

1. re-run the experiment with crates=https://crater-reports.s3.amazonaws.com/pr-149596/retry-regressed-list.txt ↩

[petrochenkov]

@bors try

[rust-bors]

☀️ Try build successful (CI)
Build commit: e617cc9 (e617cc9342f64b2efdd8eeb91834ef450a15825f, parent: 29e035e1728488dce5a03daa527ab1091d1cda47)

[tmandry]

@rfcbot fcp merge lang

[rust-rfcbot]

Team member @tmandry has proposed to merge this. The next step is review by the rest of the tagged team members:

• @joshtriplett
• @nikomatsakis
• @scottmcm
• @tmandry
• @traviscross

No concerns currently listed.

Once a majority of reviewers approve (and at most 2 approvals are outstanding), this will enter its final comment period. If you spot a major issue that hasn't been raised at any point in this process, please speak up!

cc @rust-lang/lang-advisors: FCP proposed for lang, please feel free to register concerns.
See this document for info about what commands tagged team members can give me.

[traviscross]

Possibly-relevant rules, by the way, in the pending Reference PR, include:

r[names.resolution.expansion.imports.shadowing.shared-scope]
Shadowing of names introduced via use declarations within a single scope is permitted in the following situations:

• [use glob shadowing]
• [Macro textual scope shadowing]

r[names.resolution.expansion.expansion-order-stability]
The resolution of names must be stable. After expansion, names in the fully expanded AST must resolve to the same definition regardless of the order in which macros are expanded and imports are resolved.

r[names.resolution.expansion.speculation]
All name resolution candidates selected during macro expansion are considered speculative. Once the crate has been fully expanded, all speculative import resolutions are validated to ensure that macro expansion did not introduce any new ambiguities.

r[names.resolution.expansion.imports.ambiguity.intro]
Some situations are an error when there is an ambiguity as to which macro definition, use declaration, or module an import or macro invocation's name refers to. This happens when there are two name candidates that do not resolve to the same entity where neither candidate is [permitted] to shadow the other.

r[names.resolution.expansion.imports.ambiguity.glob-vs-outer]
Names may not be resolved through glob imports when there is another candidate available in an [outer scope].

r[names.resolution.expansion.imports.ambiguity.path-vs-textual-macro]
Names may not be resolved through ambiguous macro re-exports. Macro re-exports are ambiguous when they would shadow a textual macro candidate for the same name in an [outer scope].

r[names.resolution.expansion.macros.ambiguity.more-expanded-vs-outer]
Name bindings from macro expansions may not shadow name bindings from outside of those expansions.

(See the PR for examples for each of these rules and explanatory admonitions.)

[traviscross]

Let's talk through what we're doing here. Today, this is an error:

mod m {
    pub struct S {}
}
macro_rules! mac { () => { struct S {} }}

pub use m::*;
mac!();

pub use S as _;
//~^ error[E0659]: `S` is ambiguous

Why? What do the rules say? First:

r[names.resolution.expansion.imports.ambiguity.intro]
Some situations are an error when there is an ambiguity as to which macro definition, use declaration, or module an import or macro invocation's name refers to. This happens when there are two name candidates that do not resolve to the same entity where neither candidate is [permitted] to shadow the other.

Are there two name candidates? Yes; the struct in m and the one defined via the macro invocation.

Do they resolve to the same entity? No; they have two distinct definitions.

Is one permitted to shadow the other? Let's see.

r[names.resolution.expansion.imports.shadowing.shared-scope]
Shadowing of names introduced via use declarations within a single scope is permitted in the following situations:

• [use glob shadowing]
• [Macro textual scope shadowing]

This is indeed use glob shadowing, which says:

r[items.use.glob.shadowing]
Items and named imports are allowed to shadow names from glob imports in the same [namespace]. That is, if there is a name already defined by another item in the same namespace, the glob import will be shadowed.

So this would be allowed normally. If S were used in a function body during primary name resolution (late resolution), it would be accepted). But S is used in an import, which is resolved during expansion-time resolution (early resolution).

r[names.resolution.expansion.expansion-order-stability]
The resolution of names must be stable. After expansion, names in the fully expanded AST must resolve to the same definition regardless of the order in which macros are expanded and imports are resolved.

Because the struct S is introduced by mac!(), it might not be visible when S in pub use S as _ is first resolved. If the compiler resolves S before expanding mac!(), it would see m::S (from the glob). If it expands mac!() first, it would see the struct from the macro invocation. Since the resolution depends on the order of expansion, it violates the stability rule and reports an error.

Conversely, this is accepted today:

mod m {
    pub struct S {}
}
macro_rules! mac { () => { use m::S; }}

pub use m::*;
mac!();

pub use S as _; //~ OK

Why? Are there two name candidates? Yes. But now they resolve to the same entity. The ambiguity still exists, but because both resolution paths lead to the same definition, the compiler currently considers it benign and accepts it.

Should we lint about this? @petrochenkov gives three rules:

1. The two declarations are ambiguous.
2. The two declarations have the same definition but different visibilities.
3. min(vis(decl1), vis(import)) != min(vis(decl2), vis(import))

Taking them in turn:

One. What's ambiguous mean? As I read this (and, @petrochenkov or @yaahc, correct me if I'm wrong), this is suggesting that two declarations are ambiguous when they are candidates for the same name in the same scope and neither is permitted to shadow the other or the resolution order would make the shadowing unpredictable. I.e., they are ambiguous when we would give an error if the definitions of those declarations are not the same.

So these declarations are ambiguous.

Two. They have the same definition. Do they have different visibilities? Yes. pub (from pub use m::*) vs private (from use m::S).

Three. The visibility of the import is pub (from pub use S as _), so the minimum visibility will be the visibility of each declaration. As above, the declarations have different visibilities.

QED. We lint this case.

---

Net-net, with this FCW, we're reducing the scope of what we consider to be a benign ambiguity.

Based on that understanding...

@rfcbot reviewed

[traviscross]

What's ambiguous mean? As I read this (and, @petrochenkov or @yaahc, correct me if I'm wrong), this is suggesting that two declarations are ambiguous when they are candidates for the same name in the same scope and neither is permitted to shadow the other or the resolution order would make the shadowing unpredictable. I.e., they are ambiguous when we would give an error if the definitions of those declarations are not the same.

I do note a possible counterexample to the way I stated it here (in the i.e.). We give an error for this:

mod m1 {
    pub struct S {}
}
mod m2 {
    pub struct S {}
}

pub mod m3 {
    pub use crate::m1::*;
    use crate::m2::*;
}
pub use m3::S as _;
//~^ error[E0659]: `S` is ambiguous

But the PR branch does not warn if we make the definition for each declaration the same:

mod m1 {
    pub struct S {}
}

pub mod m3 {
    pub use crate::m1::*;
    use crate::m1::*;
}
pub use m3::S as _;

Do we not consider these declarations ambiguous, or what's the rule we're using to decide to not lint here?

[petrochenkov]

@traviscross

Do we not consider these declarations ambiguous, or what's the rule we're using to decide to not lint here?

They are also ambiguous, this part of the lint is just not implemented yet.
(Scope-relative part use ambig_name - implemented, module relative part - use module::ambig_name - not implemented, that's primarily glob-vs-glob ambiguities in the same module.)

I plan to implement it too, but to do that you'd need something like https://github.com/petrochenkov/rust/tree/neverwrite, which it is mostly blocked on #149195.

[traviscross]

Makes sense. So we're expecting a later expansion of this lint.

[nikomatsakis]

@petrochenkov, I wanted to ask you a question. Thinking through this issue made me realize that my "mental model" for how pub use and privacy interact doesn't seem to be aligned with how the code works. I wanted to get your take on whether my understanding of the code is accurate and on how difficult it would be to alter how things work.

How I think the code works

I believe that when you have a use in the code, that creates a kind of "pseudo-item" whose privacy is the intersection of the original item and the use. Therefore we get this desired behavior:

mod mod_a {
    use crate::mod_b::B;
    pub struct A;
}

mod mod_b {
    pub struct B;
}

use mod_a::A; // errors, the `use` is not public here

However it also means that this example errors:

mod mod_a {
    use crate::mod_b::B as B2; // "private" item
    pub use B2 as B3; // re-exporting a private item as public, error
    pub struct A;
}

mod mod_b {
    pub struct B;
}

use mod_a::B2; // errors, the `use` is not public here
use mod_a::B3; // errors, the `use` is not public here

Indeed, I think this is the case that the issue describes.

An alternative model

Things could also work a bit different. Instead of having use items have the intersection of privacies, we could instead say that items have a visibility that is determined at the point of their declaration. Use statements also have a privacy that is distinct (but must be less than the item which the use statement names).

To resolve a path X::Y::Z from a module M, that module must be able to see each item in the path. If that item is a declaration (not a use statement), then this is determined by the declarations privacy. If that item is a use statement, then it is determined by the visibility of the use statement.

In a pub(v) use, the path must be "evaluatable" per above, but also the item that is ultimately referred to must be "at least as public" as v.

This implies that the first example errors, but the second example behaves a touch differently:

mod mod_a {
    use crate::mod_b::B as B2; // "private" item
    pub use B2 as B3; // re-exporting a private item as public, error
    pub struct A;
}

mod mod_b {
    pub struct B;
}

use mod_a::B2; // errors, the `use` is not public here
use mod_a::B3; // OK!

• use crate::mod_b::B as B2 creates a private-to-mod_a use statement to a public item
• the path B2 is evaluated from insight mod_a , and therefore it works
• the path mod_a::B2 evaluated from crate root is an error (B2 is not a visible name)
• but the path mod_a::B3 involves only a visible name (B3) and resolves to an item that is visible (B, world-public).

My question then

• Do you understand the two models I described above, or can I clarify something?
• Is my mental model for the current code (intersection of privacy scopes) accurate?
• Is there a reason we did not take the alternative design? Do you see any issues with it?
• Any idea how easy/hard it would be to implement?

[petrochenkov]

Do you understand the two models I described above, or can I clarify something?

• Looks like the first example has a typo: use mod_a::A -> use mod_a::B
• We now call imports "name declarations" too (and non-use declarations are "definitions"), so it would be good to disambiguate the places like "If that item is a declaration (not a use statement)" by using "definitions".
• Is the second example still supposed to report the "re-exporting a private item as public" error?
  • If yes, then the only difference I see is in error recovery (which visibility we assign to the erroneous import), for recovery we can make any reasonable choice, it's not a language concern
  • (So I don't understand the new model yet.)

Is my mental model for the current code (intersection of privacy scopes) accurate?

Yes, but there are more nuances because use a::b::c can import up to 3 names in different namespaces, and all of them may have different visibilities.
So, e.g.

mod m {
    pub        type name = ();
    pub(crate) fn name() {}
               macro name() {}
}

pub use m::name;

// Works like 
pub use m::name as type;
pub(crate) use m::name as fn;
// macro `name` is filtered away as too private, not imported

, exactly like with glob imports

pub use m::*;

// Works like 
pub use m::name as type;
pub(crate) use m::name as fn;
// macro `name` is filtered away as too private, not imported

[rustbot]

This PR was rebased onto a different main commit. Here's a range-diff highlighting what actually changed.

Rebasing is a normal part of keeping PRs up to date, so no action is needed—this note is just to help reviewers.

[rust-bors]

☔ The latest upstream changes (presumably #139493) made this pull request unmergeable. Please resolve the merge conflicts.

[lcnr]

r? @yaahc

[petrochenkov]

ping @joshtriplett @nikomatsakis @scottmcm
This is now the only thing blocking parallel import resolution (#145108).

[joshtriplett]

Checking a box on this.

Separately, I'm looking forward to the future edition changes proposed in #148610 to simplify pub use of macros, and hopefully eventually allow pub macro_rules! ....

[rust-rfcbot]

🔔 This is now entering its final comment period, as per the review above. 🔔

[nikomatsakis]

@rfcbot reviewed

I was reluctant to check my box here but I've become convinced that the current behavior is nice in its own way and that changing it in the way I proposed might cause private APIs to become "accidentally" exposed. The example that @tmandry showed me, which I believe came from @Nadrieril, is:

mod internal {
  mod inner {
    pub struct Foo;
  }
  pub(crate) use inner::Foo;
  pub struct Bar;
}
pub use internal::*; // `Foo` would now be part of the public API of the crate

Here, here the intent of the inner module is not to think about visibility at all, and to have the visibility defined in the pub-use re-exports from the internal module.

Put another way: what I consider most important is "locality of reasoning" about visibility. I want to be able to look at a particular declaration and understand who might be relying on it. And I can see that the model I proposed in fact violates that locality of reasoning in an example like this. Not for the inner module but for the internal module, where the pub(crate) use inner::Foo in fact exposes something publicly but only due to "action at a distance" from a pub-use in the root crate. This is exactly the kind of thing we wanted to avoid by requiring you to put pub on items that may be exported publicly from your crate by other pub use statements.

So, I'm convinced. Thanks all for the discussion. I think it'd be useful to try and capture these notes somewhere for the next time I am confused.