Query Console has no result when splitting supplementary code points

[Marcono1234]

It appears predicates for the built-in type string do not have any result in the Query Console when splitting supplementary Unicode code points into the corresponding surrogate chars.
I am not sure whether the VSCode extension behaves differently, and am not sure either what the expected behavior is.

select "😊".prefix(1)

Query Console link

Edit: Same also applies when not including the split supplementary code point in the result:

where exists("😊".prefix(1))
select "hello world"

As a side note: Trying to use unpaired surrogates in the Query Console causes "Failed to execute" (cannot provide example code here because GitHub does not preservere unpaired surrogate; and Query Console does not run so cannot provide link either):

[github-actions]

This issue is stale because it has been open 14 days with no activity. Comment or remove the stale label in order to avoid having this issue closed in 7 days.

[edoardopirovano]

The spec defines strings in CodeQL as sequences of 16-bit characters: https://codeql.github.com/docs/ql-language-reference/types/. What is omitted from this definition is that strings that contain unpaired surrogates are not considered valid.

When code points from the supplementary plane are used, they get split into their surrogates, if these end up being separated by an operation like "😊".prefix(1) to produce strings with unpaired surrogates, this is not considered a valid string and is omitted from the results. In my opinion, this isn't really a bug but more an edge case of the spec that isn't very well-documented but is working as intended.

Separate from this, there are issues with how some of the tooling handles characters from the supplementary plane. I think LGTM's failure to execute a query with an unpaired surrogate is reasonable, as this isn't a valid string. However, the VS code extension fails to Quick Eval even strings with correctly paired surrogates, which I think is a bug. I've created an issue for this: github/vscode-codeql#837

[Marcono1234]

Thanks for the explanation!

In my opinion, this isn't really a bug but more an edge case of the spec that isn't very well-documented but is working as intended.

It could be an issue when source code contains unpaired surrogates, e.g. as part of String literals, and a query tries to include them in the results causing false negatives. Or when a query tries to include a 'preview' of a string by only including e.g. the first 5 16-bit chars and accidentially splits a surrogate pair.
Though these are likely rare corner cases.