Skip to content

feat(singlestore): Added Tokenizer#5492

Merged
georgesittas merged 6 commits intotobymao:mainfrom
singlestore-labs:singlestore-3
Jul 29, 2025
Merged

feat(singlestore): Added Tokenizer#5492
georgesittas merged 6 commits intotobymao:mainfrom
singlestore-labs:singlestore-3

Conversation

@AdalbertMemSQL
Copy link
Contributor

Added several new tokens.
:> and :!> are used in SingleStore as cast operators.
::$ and ::% are used as JSON extract operators.

Copy link
Collaborator

@VaggelisD VaggelisD left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @AdalbertMemSQL, thank you for the PR once again!

  • Can you please point at the relevant docs?
  • Can we add some identity tests that validate those new tokens?

@AdalbertMemSQL
Copy link
Contributor Author

AdalbertMemSQL commented Jul 29, 2025

Hey @VaggelisD
Regarding docs:

  • For BYTE_STRINGS I didn't find information in SingleStore documentation, but I tested e'' syntax and it works
singlestore> select e'text'; 
+------+
| text |
+------+
| text |
+------+
1 row in set (0.05 sec)

Copy link
Collaborator

@georgesittas georgesittas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good @AdalbertMemSQL, just one comment on my end. Thanks!

Let's also resolve the conflicts, similarly to the other PR.

@georgesittas georgesittas merged commit a7a6f16 into tobymao:main Jul 29, 2025
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants