Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Forbid usage of hir Infer const/ty variants in ambiguous contexts #135272

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

BoxyUwU
Copy link
Member

@BoxyUwU BoxyUwU commented Jan 8, 2025

The feature generic_arg_infer allows providing _ as an argument to const generics in order to infer them. This introduces a syntactic ambiguity as to whether generic arguments are type or const arguments. In order to get around this we introduced a fourth GenericArg variant, Infer used to represent _ as an argument to generic parameters when we don't know if its a type or a const argument.

This made hir visitors that care about TyKind::Infer or ConstArgKind::Infer very error prone as checking for TyKind::Infers in visit_ty would find some type infer arguments but not all of them as they would sometimes be lowered to GenericArg::Infer instead.

Additionally the visit_infer method would previously only visit GenericArg::Infer not all infers (e.g. TyKind::Infer), this made it very easy to override visit_infer and expect it to visit all infers when in reality it would only visit some infers.


This PR aims to fix those issues by making the TyKind and ConstArgKind types generic over whether the infer types/consts are represented by Ty/ConstArgKind::Infer or out of line (e.g. by a GenericArg::Infer or accessible by overiding visit_infer). We then make HIR Visitors convert all const args and types to the versions where infer vars are stored out of line and call visit_infer in cases where a Ty/Const would previously have had a Ty/ConstArgKind::Infer variant:

API Summary

enum AmbigArg {}

enum Ty/ConstArgKind<Unambig = ()> {
   ...
   Infer(Unambig),
}

impl Ty/ConstArg {
  fn try_as_ambig_ty/ct(self) -> Option<Ty/ConstArg<AmbigArg>>;
}
impl Ty/ConstArg<AmbigArg> {
  fn as_unambig_ty/ct(self) -> Ty/ConstArg;
}

enum InferKind {
  Ty(Ty),
  Const(ConstArg),
  Ambig(InferArg),
}

trait Visitor {
  ...
  fn visit_ty/const_arg(&mut self, Ty/ConstArg<AmbigArg>) -> Self::Result;
  fn visit_infer(&mut self, id: HirId, sp: Span, kind: InferKind) -> Self::Result; 
}

// blanket impl'd, not meant to be overriden
trait VisitorExt {
  fn visit_ty/const_arg_unambig(&mut self, Ty/ConstArg) -> Self::Result;
}

fn walk_unambig_ty/const_arg(&mut V, Ty/ConstArg) -> Self::Result;
fn walk_ty/const_arg(&mut V, Ty/ConstArg<AmbigArg>) -> Self::Result;

The end result is that visit_infer visits all infer args and is also the only way to visit an infer arg, visit_ty and visit_const_arg can now no longer encounter a Ty/ConstArgKind::Infer. Representing this in the type system means that it is now very difficult to mess things up, either accessing TyKind::Infer "just works" and you won't miss some type infers- or it doesn't work and you have to look at visit_infer or some GenericArg::Infer which forces you to think about the full complexity involved.

Unfortunately there is no lint right now about explicitly matching on uninhabited variants, I can't find the context for why this is the case 🤷‍♀️

I'm not convinced the framing of un/ambig ty/consts is necessarily the right one but I'm not sure what would be better. I somewhat like calling them full/partial types based on the fact that Ty<Partial>/Ty<Full> directly specifies how many of the type kinds are actually represented compared to Ty<Ambig> which which leaves that to the reader to figure out based on the logical consequences of it the type being in an ambiguous position.


tool changes have been modified in their own commits for easier reviewing by anyone getting cc'd from subtree changes. I also attempted to split out "bug fixes arising from the refactoring" into their own commit so they arent lumped in with a big general refactor commit

Fixes #112110

@rustbot
Copy link
Collaborator

rustbot commented Jan 8, 2025

r? @fmease

rustbot has assigned @fmease.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue. labels Jan 8, 2025
@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 8, 2025

sry fmease
r? @ghost

@rustbot
Copy link
Collaborator

rustbot commented Jan 8, 2025

Failed to set assignee to ghost: invalid assignee

Note: Only org members with at least the repository "read" role, users with write permissions, or people who have commented on the PR may be assigned.

@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 8, 2025

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

@rustbot rustbot added the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 8, 2025
bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 8, 2025
…y_2, r=<try>

[PERF] dont represent infer vars two different ways in hir visitors
@bors
Copy link
Contributor

bors commented Jan 8, 2025

⌛ Trying commit c19e8c3 with merge 397cf05...

@rust-log-analyzer

This comment has been minimized.

@rust-log-analyzer

This comment has been minimized.

@bors
Copy link
Contributor

bors commented Jan 9, 2025

💔 Test failed - checks-actions

@bors bors added S-waiting-on-author Status: This is awaiting some action (such as code changes or more information) from the author. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 9, 2025
@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 9, 2025

@bors try @rust-timer queue

@rust-timer

This comment has been minimized.

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 9, 2025
…y_2, r=<try>

[PERF] dont represent infer vars two different ways in hir visitors

to lampshade a bit for anyone who decides to review a draft PR: yes this is not very a good PR right now
@bors
Copy link
Contributor

bors commented Jan 9, 2025

⌛ Trying commit d53cf04 with merge 9ce13d8...

@bors
Copy link
Contributor

bors commented Jan 9, 2025

☀️ Try build successful - checks-actions
Build commit: 9ce13d8 (9ce13d8853ff501c75672ba2b97a12ad6b00251e)

@rust-timer

This comment has been minimized.

@rust-timer
Copy link
Collaborator

Finished benchmarking commit (9ce13d8): comparison URL.

Overall result: ❌ regressions - no action needed

Benchmarking this pull request likely means that it is perf-sensitive, so we're automatically marking it as not fit for rolling up. While you can manually mark this PR as fit for rollup, we strongly recommend not doing so since this PR may lead to changes in compiler perf.

@bors rollup=never
@rustbot label: -S-waiting-on-perf -perf-regression

Instruction count

This is the most reliable metric that we have; it was used to determine the overall result at the top of this comment. However, even this metric can sometimes exhibit noise.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
0.3% [0.3%, 0.3%] 2
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) - - 0

Max RSS (memory usage)

Results (secondary -3.6%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
- - 0
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
-3.6% [-3.6%, -3.6%] 1
All ❌✅ (primary) - - 0

Cycles

Results (primary 4.3%)

This is a less reliable metric that may be of interest but was not used to determine the overall result at the top of this comment.

mean range count
Regressions ❌
(primary)
4.3% [3.4%, 7.4%] 15
Regressions ❌
(secondary)
- - 0
Improvements ✅
(primary)
- - 0
Improvements ✅
(secondary)
- - 0
All ❌✅ (primary) 4.3% [3.4%, 7.4%] 15

Binary size

This benchmark run did not return any relevant results for this metric.

Bootstrap: 764.798s -> 764.043s (-0.10%)
Artifact size: 325.77 MiB -> 325.78 MiB (0.00%)

@rustbot rustbot removed the S-waiting-on-perf Status: Waiting on a perf run to be completed. label Jan 9, 2025
@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 9, 2025

yippee

@BoxyUwU BoxyUwU force-pushed the generic_arg_infer_reliability_2 branch from d53cf04 to 909f1f6 Compare January 10, 2025 20:32
@rust-log-analyzer

This comment has been minimized.

@BoxyUwU BoxyUwU changed the title [PERF] dont represent infer vars two different ways in hir visitors Forbid usage of hir Infer const/ty variants in ambiguous contexts Jan 10, 2025
@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 19, 2025

Rebased and pushed additional commits for the reviews so if you want to look at just the new stuff u can just look at the commits after Bless and add tests. Will squash them into the other commits and whatnot before merging

@BoxyUwU BoxyUwU force-pushed the generic_arg_infer_reliability_2 branch from 6fec71e to d7b916b Compare January 19, 2025 05:49
@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 19, 2025

@bors try

@bors
Copy link
Contributor

bors commented Jan 19, 2025

⌛ Trying commit d7b916b with merge ae258e1fcbee4a062f5f3d869f06004d7cea088e...

bors added a commit to rust-lang-ci/rust that referenced this pull request Jan 19, 2025
…y_2, r=<try>

Forbid usage of `hir` `Infer` const/ty variants in ambiguous contexts

The feature `generic_arg_infer` allows providing `_` as an argument to const generics in order to infer them. This introduces a syntactic ambiguity as to whether generic arguments are type or const arguments. In order to get around this we introduced a fourth `GenericArg` variant, `Infer` used to represent `_` as an argument to generic parameters when we don't know if its a type or a const argument.

This made hir visitors that care about `TyKind::Infer` or `ConstArgKind::Infer` very error prone as checking for `TyKind::Infer`s in  `visit_ty` would find *some* type infer arguments but not *all* of them as they would sometimes be lowered to `GenericArg::Infer` instead.

Additionally the `visit_infer` method would previously only visit `GenericArg::Infer` not *all* infers (e.g. `TyKind::Infer`), this made it very easy to override `visit_infer` and expect it to visit all infers when in reality it would only visit *some* infers.

---

This PR aims to fix those issues by making the `TyKind` and `ConstArgKind` types generic over whether the infer types/consts are represented by `Ty/ConstArgKind::Infer` or out of line (e.g. by a `GenericArg::Infer` or accessible by overiding `visit_infer`). We then make HIR Visitors convert all const args and types to the versions where infer vars are stored out of line and call `visit_infer` in cases where a `Ty`/`Const` would previously have had a `Ty/ConstArgKind::Infer` variant:

API Summary
```rust
enum AmbigArg {}

enum Ty/ConstArgKind<Unambig = ()> {
   ...
   Infer(Unambig),
}

impl Ty/ConstArg {
  fn try_as_ambig_ty/ct(self) -> Option<Ty/ConstArg<AmbigArg>>;
}
impl Ty/ConstArg<AmbigArg> {
  fn as_unambig_ty/ct(self) -> Ty/ConstArg;
}

enum InferKind {
  Ty(Ty),
  Const(ConstArg),
  Ambig(InferArg),
}

trait Visitor {
  ...
  fn visit_ty/const_arg(&mut self, Ty/ConstArg<AmbigArg>) -> Self::Result;
  fn visit_infer(&mut self, id: HirId, sp: Span, kind: InferKind) -> Self::Result;
}

// blanket impl'd, not meant to be overriden
trait VisitorExt {
  fn visit_ty/const_arg_unambig(&mut self, Ty/ConstArg) -> Self::Result;
}

fn walk_unambig_ty/const_arg(&mut V, Ty/ConstArg) -> Self::Result;
fn walk_ty/const_arg(&mut V, Ty/ConstArg<AmbigArg>) -> Self::Result;
```

The end result is that `visit_infer` visits *all* infer args and is also the *only* way to visit an infer arg, `visit_ty` and `visit_const_arg` can now no longer encounter a `Ty/ConstArgKind::Infer`. Representing this in the type system means that it is now very difficult to mess things up, either accessing `TyKind::Infer` "just works" and you won't miss *some* type infers- or it doesn't work and you have to look at `visit_infer` or some `GenericArg::Infer` which forces you to think about the full complexity involved.

Unfortunately there is no lint right now about explicitly matching on uninhabited variants, I can't find the context for why this is the case 🤷‍♀️

I'm not convinced the framing of un/ambig ty/consts is necessarily the right one but I'm not sure what would be better. I somewhat like calling them full/partial types based on the fact that `Ty<Partial>`/`Ty<Full>` directly specifies how many of the type kinds are actually represented compared to `Ty<Ambig>` which which leaves that to the reader to figure out based on the logical consequences of it the type being in an ambiguous position.

---

tool changes have been modified in their own commits for easier reviewing by anyone getting cc'd from subtree changes. I also attempted to split out "bug fixes arising from the refactoring" into their own commit so they arent lumped in with a big general refactor commit

Fixes rust-lang#112110
@bors
Copy link
Contributor

bors commented Jan 19, 2025

☀️ Try build successful - checks-actions
Build commit: ae258e1 (ae258e1fcbee4a062f5f3d869f06004d7cea088e)

@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 19, 2025

@craterbot check

@craterbot
Copy link
Collaborator

👌 Experiment pr-135272 created and queued.
🤖 Automatically detected try build ae258e1
🔍 You can check out the queue and this experiment's details.

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-crater Status: Waiting on a crater run to be completed. and removed S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. labels Jan 19, 2025
@craterbot
Copy link
Collaborator

🚧 Experiment pr-135272 is now running

ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot
Copy link
Collaborator

🎉 Experiment pr-135272 is completed!
📊 21 regressed and 3 fixed (570539 total)
📰 Open the full report.

⚠️ If you notice any spurious failure please add them to the denylist!
ℹ️ Crater is a tool to run experiments across parts of the Rust ecosystem. Learn more

@craterbot craterbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. and removed S-waiting-on-crater Status: Waiting on a crater run to be completed. labels Jan 20, 2025
@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 20, 2025

aww lol

@BoxyUwU
Copy link
Member Author

BoxyUwU commented Jan 20, 2025

this is not a new bug from this PR this is a pre-existing bug in generic_arg_infer. ast::TyKind::Infer is not the only ast ty that can be lowered to a hir::TyKind::Infer- a ast::TyKind::Paren wrapping a ast::TyKind::Infer also can be. e.g. let a: Vec<(_)> will ice under this PR as we attempt to create a hir::TyKind::<AmbigArg>::Infer for the GenericArg to Vec

all the regressions are due to an old version of the geo crate containing a let a: Vec<(_)>

@BoxyUwU BoxyUwU force-pushed the generic_arg_infer_reliability_2 branch from b827a7b to c03a819 Compare January 20, 2025 07:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. T-rustdoc Relevant to the rustdoc team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

generic_arg_infer impl is prone to bugs in hir visitors
8 participants