Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Upgrade completions, definition, hover #166

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

milesziemer
Copy link
Contributor

This commit is a rewrite of how language features (i.e. completions, definition, hover) are implemented. It improves the accuracy and expands the functionality of each feature significantly. Improvements include:

  • Completions - Trait values - Builtin control keys and metadata - Namespaces, based on other namespaces in the project - Keywords - Member names (like inside resources, maps) - Member values (like inside the list of operation errors, resource property targets, etc.) - Elided members - Some trait values have special completions, like examples has completions for the target operation's input/output parameters
  • Definition - Trait values - Elided members - Shape ids referenced within trait values
  • Hover - Trait values - Elided members - Builtin metadata

There's a lot going on here, but there's a few key pieces of this commit that all work together to make this work:

At the core of these improvements is the addition of a custom parser for the IDL that provides the needed syntactic information to implement these features. See the javadoc on the Syntax class for more details on how the parser works, and why it was written that way. At a high level though, the parser produces a flat list of Syntax.Statement, and that list is searched through to find things, such as the statement the cursor is currently in. It is also used to search 'around' a statement, like to find the shape a trait is being applied to.

Another key piece of these changes is NodeCursor and NodeSearch. There are a few places in the syntax of a smithy file where you may have a node value whose structure is (or can be) described by a Smithy model. For example, trait values. NodeCursor is basically two things: 1. A path from the start of a Node to a position within that Node, 2. An index into that path. NodeSearch is used to search a model along the path of a NodeCursor, from a starting shape. For example, when the cursor is within a trait value, the NodeCursor is that path from the root of the trait value, to the cursor position, and NodeSearch is used to search in the model, starting at the trait's definition, along the path of the NodeCursor, to find what shape corresponds to the cursor's location. That shape can then be used e.g. to provide completions.

Finally, there's the Builtins class, and the corresponding Smithy model it uses. I originally had a completely different abstraction for describing the structure of metadata, different shape types' members, and even smithy-build.json. But it was basically just a 'structured graph', like a Smithy model. So I decided to just use a Smithy model itself, since I already had the abstractions for traversing it (like I had to for trait values). The Builtins model contains shapes that define the structure of certain Smithy constructs. For example, I use it to model the shape of builtin metadata, like suppressions. I also use it to model the shape of shapes, that is, what members shapes have, and what their targets are. Some shapes in this model are considered 'builtins' (in the builtins.smithy files). Builtins are shapes that require some custom processing, or have some special meaning, like AnyNamespace, which is used for describing a namespace that can be used in
https://smithy.io/2.0/spec/model-validation.html#suppression-metadata. The builtin model pretty 'meta', and I don't love it, but it reduces a significant amount of duplicated logic. For example, if we want to give documentation for some metadata, it is as easy as adding it to the builtins model. We can also use it to add support for smithy-build.json completions, hover, and even validation, later. It would be nice if these definitions lived elsewhere, so other tooling could consume them, like the Smithy docs for example, and I have some other ideas on how we can use it, but they're out of scope here.

Testing for this commit comes mostly from the completions, definitions, and hover tests, which indirectly test lower-level components like the parser (there are still some parser tests, though).

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

@milesziemer milesziemer requested a review from a team as a code owner September 24, 2024 18:11
@milesziemer milesziemer requested review from gosar and hpmellema and removed request for gosar September 24, 2024 18:11
import software.amazon.smithy.lsp.syntax.Syntax;
import software.amazon.smithy.lsp.syntax.SyntaxSearch;

sealed interface IdlPosition {
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I hate this class. I spent some time looking for a way to abstract away the direct usage of documentIndex and statementIndex, both here and in SyntaxSearch, but didn't come up with anything great. Open to suggestions.

This commit is a rewrite of how language features (i.e. completions,
definition, hover) are implemented. It improves the accuracy and expands
the functionality of each feature significantly. Improvements include:
- Completions
    - Trait values
    - Builtin control keys and metadata
    - Namespaces, based on other namespaces in the project
    - Keywords
    - Member names (like inside resources, maps)
    - Member values (like inside the list of operation errors, resource
      property targets, etc.)
    - Elided members
    - Some trait values have special completions, like `examples` has
      completions for the target operation's input/output parameters
- Definition
    - Trait values
    - Elided members
    - Shape ids referenced within trait values
- Hover
    - Trait values
    - Elided members
    - Builtin metadata

There's a lot going on here, but there's a few key pieces of this commit
that all work together to make this work:

At the core of these improvements is the addition of a custom parser for
the IDL that provides the needed syntactic information to implement
these features. See the javadoc on the Syntax class for more details on
how the parser works, and why it was written that way. At a high level
though, the parser produces a flat list of `Syntax.Statement`, and that
list is searched through to find things, such as the statement the
cursor is currently in. It is also used to search 'around' a statement,
like to find the shape a trait is being applied to.

Another key piece of these changes is `NodeCursor` and `NodeSearch`.
There are a few places in the syntax of a smithy file where you may have
a node value whose structure is (or can be) described by a Smithy model.
For example, trait values. `NodeCursor` is basically two things: 1. A
path from the start of a `Node` to a position within that `Node`, 2. An
index into that path. `NodeSearch` is used to search a model along the
path of a `NodeCursor`, from a starting shape. For example, when the
cursor is within a trait value, the `NodeCursor` is that path from the
root of the trait value, to the cursor position, and `NodeSearch` is
used to search in the model, starting at the trait's definition, along
the path of the `NodeCursor`, to find what shape corresponds to the
cursor's location. That shape can then be used e.g. to provide completions.

Finally, there's the `Builtins` class, and the corresponding Smithy
model it uses. I originally had a completely different abstraction for
describing the structure of metadata, different shape types' members,
and even `smithy-build.json`. But it was basically just a 'structured
graph', like a Smithy model. So I decided to just _use_ a Smithy model
itself, since I already had the abstractions for traversing it (like I
had to for trait values). The `Builtins` model contains shapes that
define the structure of certain Smithy constructs. For example, I use it
to model the shape of builtin metadata, like suppressions. I also use it
to model the shape of shapes, that is, what members shapes have, and
what their targets are. Some shapes in this model are considered
'builtins' (in the builtins.smithy files). Builtins are shapes that
require some custom processing, or have some special meaning, like
`AnyNamespace`, which is used for describing a namespace that can be
used in
https://smithy.io/2.0/spec/model-validation.html#suppression-metadata.
The builtin model pretty 'meta', and I don't _love_ it, but it reduces a
significant amount of duplicated logic. For example, if we want to give
documentation for some metadata, it is as easy as adding it to the
builtins model. We can also use it to add support for smithy-build.json
completions, hover, and even validation, later. It would be nice if
these definitions lived elsewhere, so other tooling could consume them,
like the Smithy docs for example, and I have some other ideas on how we
can use it, but they're out of scope here.

Testing for this commit comes mostly from the completions, definitions,
and hover tests, which indirectly test lower-level components like the
parser (there are still some parser tests, though).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants