Since July update cause java highlight break

[ericsia]

I love to use VSC Java plugin but I had so much pain working with java now.
Now the highlight is glitchy, it don't highlight properly when Semantic Highlight is turn on.
Comment don't get highlighted :( I feel sad
Also I would appreciate it if it is easier to use junit assertEqual, when I try to open a single .java file, it cannot import org.junit

Environment

• Operating System: Windows 10
• JDK version: 14
• Visual Studio Code version: 1.48.2 before update also the same
• Java extension version: -

Steps To Reproduce

1. Turn on semantic highlight
2. start debugging

Current Result

Expected Result

Additional Informations

[fbricon]

Are you saying semantic highlighting is fine when not debugging?

[ericsia]

Are you saying semantic highlighting is fine when not debugging?

after I close and open it sometime it appears fine

[0dinD]

Does the highlighting also "fix" itself after you type a character?

Not sure if it's related, but I've seen something very similar happen when applying large changes to the document, such as undo/redo or using git reset --hard. I unfortunately could not find a way to consistently reproduce the issue, but the semantic highlighting broke in the same way as in your screenshot.

@ericsia if you could find a way to consistently reproduce the issue, that would help a lot in finding a fix. Does it always happen when debugging? Also, what version of the extension are you using? The latest?

[clankill3r]

I also notice that this happens a lot since July, I'm on a mac OSX 10.12.6.
Often typing and deleting something fixes it, but it stays annoying.

[0dinD]

@clankill3r can you provide some more info about what you're doing when the issue occurs? It would make the debugging process a lot easier if we had a way to reproduce the issue. And by "since July", do you mean the 0.65.0 update or do you not know which version you were using?

Personally I've seen this issue occur earlier than 0.65.0, but I couldn't find a way to consistently reproduce it. It seemed to occur somewhat randomly when applying large changes to the document. Spamming undo/redo sometimes reproduced it, but there was not pattern to it.

[ericsia]

@clankill3r can you provide some more info about what you're doing when the issue occurs? It would make the debugging process a lot easier if we had a way to reproduce the issue. And by "since July", do you mean the 0.65.0 update or do you not know which version you were using?

Personally I've seen this issue occur earlier than 0.65.0, but I couldn't find a way to consistently reproduce it. It seemed to occur somewhat randomly when applying large changes to the document. Spamming undo/redo sometimes reproduced it, but there was not pattern to it.

most accurately I didn't know since when do I turn it on. During last 2 update if I didn't rmb wrongly, I saw VS Code prompt me a msg to enable semantic highlight. So I go and turn it on. There is where I notice the highlights broken.
It is the best to try trace back from older build

[ericsia]

Does the highlighting also "fix" itself after you type a character?

Not sure if it's related, but I've seen something very similar happen when applying large changes to the document, such as undo/redo or using git reset --hard. I unfortunately could not find a way to consistently reproduce the issue, but the semantic highlighting broke in the same way as in your screenshot.

@ericsia if you could find a way to consistently reproduce the issue, that would help a lot in finding a fix. Does it always happen when debugging? Also, what version of the extension are you using? The latest?

During I start typing code from scratch I think. When I added comment, add more code then it might occurs

[0dinD]

During last 2 update if I didn't rmb wrongly

Ok, that strengthens my belief that this issue was not introduced via any of my changes in 0.65.0, and has been present since the initial implementation.

Again, if someone could provide a way to consistently reproduce this issue (via steps), that would make the debugging process a lot less time-consuming. (Meaning a fix can be released sooner.)

@ericsia how often do you estimate that this issue appears for you? And what's the connection between this issue and debug mode? Are you saying that the issue only appears when debugging, or that it appears more often when debugging?

[ericsia]

During last 2 update if I didn't rmb wrongly

Ok, that strengthens my belief that this issue was not introduced via any of my changes in 0.65.0, and has been present since the initial implementation.

Again, if someone could provide a way to consistently reproduce this issue (via steps), that would make the debugging process a lot less time-consuming. (Meaning a fix can be released sooner.)

@ericsia how often do you estimate that this issue appears for you? And what's the connection between this issue and debug mode? Are you saying that the issue only appears when debugging, or that it appears more often when debugging?

The color highlight when get messed like that doesn't go back to normal (when stop debug) until I reopen vscode again. I am also trying hard to find how to reproduce it but right now I'm not writing any code atm.
Now when I freshly open the java code everything look normal. But maybe tomorrow or 2 days later I will need to code again I will pay extra attention

[0dinD]

While I have not come up with a reliable way to reproduce this issue, it did occur again during development, so I thought I'd share another screenshot. For me it occurred quite randomly when typing code, possibly as an effect of pasting ~10 lines of code, but I'm not sure yet whether the two were related. The bug doesn't happen very often, but it's still annoying the few times where it does.

What's interesting to note is that just like the screenshot shared in the first comment of this issue, the tokens are not completely wrong, just offset by a few characters. Weirdly enough, the Developer: Inspect Editor Tokens and Scopes command reports that the semantic tokens are in the correct positions. So the offset either only appears visually, or the Developer: Inspect Editor Tokens and Scopes command is somehow also broken and doesn't show the correct information.

The issue goes away as soon as new semantic tokens are requested, so all you have to do is type a character and the highlighting is no longer broken.

[0dinD]

This issue has proven hard to debug, but I believe I've found a possible explanation, even if I don't know the exact cause or how to fix it. It seems to occur in some rare cases, when changes are made to the document during a request for semantic tokens.

The most reliable I've been able to reproduce it is to make multiple small changes to the document in rapid succession, but the timing required to reproduce the issue is very specific. I would also guess that the issue can occur when there is a change to the document during a particularly long running request for semantic tokens. Perhaps then, the issue will occur more frequently on machines with lower performance or when working with very large Java files.

All of the information that I've gathered seems to suggest that this is a bug in VS Code, rather than this extension. But again, I'm not entirely sure, as this issue is hard to reproduce. The code below this line looks like it could be part of the cause. It seems to shift the semantic tokens around and wait for new ones, if changes are made to the document during a request for semantic tokens.

I think I've noticed this behavior when trying to reproduce the issue, as the tokens do get shifted around (like in the previous screenshot!), only to get corrected a split second later. My best guess is that the request for new semantic tokens occasionally gets ignored on this line for some reason, in which case this issue would appear since the tokens have been shifted around but not fixed by requesting new ones. Again, as soon as you type a character, new semantic tokens are requested and the issue is gone.

If someone with more time on their hands or experience with the VS Code source-code could look into this, that would be greatly appreciated. I might submit an issue over at the VS Code repo, but right now I'm not sure if it's entirely the fault of VS Code, and I don't have a very reliable way to reproduce the issue.

[JoeyRxy]

yeah, I've encountered this problem since the beginning of semantic color rendering tech.
it's a severe problem I think.