How to hook up Semantic Tokens from a server ?

[AntoineBalaine]

Hello,

I’m trying to hook-up the semantic tokens sample into the lsp-sample’s server.

Following the semantic highlight guide doesn’t quite give me all the info I need. The guide points to vscode.languages.registerDocumentSemanticTokensProvider(selector, provider, legend);, but:

• After passing {semanticTokensProvider: legend} in the server capabilities; in the server’s initialize, how is the DocumentSemanticTokensProvider supposed to be hooked-up to the connection?
• is there also some setup to do on the client side ?
• do the SemanticTokens get automatically handled at every update or must they be continuously updated in the documentsManager.onDidChangeContent ?

Referencing lsp-sample/server/src/server.ts:

import {
	DocumentSemanticTokensProvider,
	legend,
} from './semanticTokensProvider/tokensProvider';

const connection = createConnection(ProposedFeatures.all);
const SemanticProvider = new DocumentSemanticTokensProvider(); // where does this guy go?

const documentsManager: TextDocuments<TextDocument> = new TextDocuments(TextDocument);
//[...]
connection.onInitialize((params: InitializeParams) => {
	const capabilities = params.capabilities;

	const serverCapabilities: InitializeResult = {
		capabilities: {
                       //[...]
			semanticTokensProvider: {
				legend,
			},
		},
	};
	return serverCapabilities;
});

[AlecGhost]

Hello there,
unfortunatly I cannot really answer your first question.

However, I can confirm, that you do not need any additional setup on the client side. If the client starts the server and the server announces, that it has the capability to provide semantic tokens, you are fine.

Regarding your final question: VSCode as a client automatically requests semantic tokens, when it needs them. So you do not need to send them on your own. In fact, there is not even a method defined in the LSP, that lets you do this. The only thing, that you can do (if I read the protocol correctly) is to send a workspace/semanticTokens/refresh request, which recommends to the client to ask for semantic tokens again.

I hope this information can help you.

[AntoineBalaine]

Got it, thank you very much!

[caryyu]

@AntoineBalaine Did you get it resolved? Any solutions for this, I got this from the Extension Development Host output:

[Error - 8:12:35 PM] Request textDocument/semanticTokens/full failed.
  Message: Unhandled method textDocument/semanticTokens/full
  Code: -32601 

Here's my config very similar to yours:

  const result: InitializeResult = {
    capabilities: {
      semanticTokensProvider: {
        legend: {
          tokenTypes: [
            'comment', 'string', 'keyword', 'number', 'regexp', 'operator', 'namespace',
            'type', 'struct', 'class', 'interface', 'enum', 'typeParameter', 'function',
            'method', 'decorator', 'macro', 'variable', 'parameter', 'property', 'label'
          ],
          tokenModifiers: [
            'declaration', 'documentation', 'readonly', 'static', 'abstract', 'deprecated',
            'modification', 'async'
          ],
        },
        full: true,
      },
....

[mtiller-jh]

I found this useful:

microsoft/vscode-discussions#819

[wakaztahir]

@AlecGhost Hi, you said client doesn't need additional setup, however VSCode is just not sending the request to server, so client requires additional setup through

vscode.languages.registerDocumentSemanticTokensProvider(selector, provider, legend);

As stated in :
microsoft/vscode#204906 (comment)

Even though, this should not be the case, The client should send the request automatically or at least when instructed by the programmer without having to reimplement it because the LSP Client API is not understandable for someone who didn't work on it.

[AlecGhost]

@wakaztahir I have no knowledge of the VSCode internals, I can only report the behaviour I experience. My extension works without additional client-side setup for the semantic tokens, see https://github.com/AlecGhost/LSP4SPL/blob/master/editors/code/src/extension.ts.

Besides, registerDocumentSemanticTokensProvider sounds very much like a server method, not like a client method.
Maybe your server does not announce a semanticTokenProvider in its capabilities during the initialization phase?

[wakaztahir]

Hi @AlecGhost , I have fixed everything, I've mentioned everything here

microsoft/vscode#204906