Semantic highlighting

[ejolly]

I'm not sure how feasible this is but it would be really nice to have a semantic highlighting option, e.g. something like what pycharm has. I apologize in advance if this was discussed elsewhere but I didn't see any issues in this repo about it.

This could be particularly useful not when writing code for building libraries, but also when developing data analysis scripts/notebooks using the already great data-science features.

[jakebailey]

We're limited by the LSP specification, which currently does not include syntactic highlighting. For VS Code as a client, they appear to be working on that for the client side, as their JS/TS support in the latest preview release includes it (https://code.visualstudio.com/updates/v1_42#_semantic-highlighting-for-typescript-javascript).

Overall, we can't work on this until both the LSP and the client support it, as both sides lack the required APIs.

[thomasjm]

@jakebailey I'm confused, isn't this exactly what the textDocument/documentHighlight request does? I thought that had been part of the LSP spec from the beginning.

(I was considering switching to the Microsoft language server from the Palantir one, which already implements this, and noticed that this one unfortunately doesn't yet.)

[jakebailey]

Nope, textDocument/documentHighlight will look at the position your cursor is at for a variable, then highlight all uses of that specific variable in the document. This allows language servers to only highlight the variable within a scope, rather than every instance of the word of the document. For example, if I have a variable named i in one function, and a bunch of other i variables in other functions, this call would let the client know which is are actually the same. It's essentially a reference call, except you could do something like also highlighting uses in structured documentation or something (as it has no meaning other than a visual highlight). See #1767 and the description you linked.

It's also not textDocument/documentColor, which is used for something like an HTML or CSS LS where they want to tell the client that #285577 is a hex color and to display a little box previewing the color.

Real semantic highlighting has only recently been added to VS Code (only available in TS/JS, which are not LSP but bundled extensions). As far as I know, true semantic highlighting support is still a work in progress for the LSP spec.

[thomasjm]

Oh, I see -- real semantic highlighting = adding colors to all the different parts of the code, regardless of where the cursor is. Makes sense. Glad to see #1767 is in progress.

[jakebailey]

Note that the cursor thing is specifically a highlighted box in VS Code, not the color of the text itself. It's just a visual indicator. You can get it now, it will just be less exact.

[xsoheilalizadeh]

This is an awesome feature that could happen, is there any timeline for this?

[munael]

I think PyLance is doing something like this now. It's still pretty early (e.g. currently it colors self with the same color as any other "available" variable). But it does differentiate functions, classes, etc. But all names of a certain kind are still colored the same (no per-name coloring).

[christianbundy]

I think textDocument/semanticTokens is what we want here: https://microsoft.github.io/language-server-protocol/specifications/specification-current/#textDocument_semanticTokens