Skip to content

feat(starrocks): add full support for partitions#6804

Merged
geooo109 merged 4 commits intotobymao:mainfrom
petrikoro:feat/add-full-support-for-starrocks-partitions
Jan 26, 2026
Merged

feat(starrocks): add full support for partitions#6804
geooo109 merged 4 commits intotobymao:mainfrom
petrikoro:feat/add-full-support-for-starrocks-partitions

Conversation

@petrikoro
Copy link
Contributor

@petrikoro petrikoro commented Jan 19, 2026

Fixes #6803

Description

This PR adds comprehensive support for StarRocks partitioning syntax including:

Previously, multi-expression partitioning and LIST partitioning were not supported.

Examples

Expression-based partitioning:

-- Single expression
CREATE TABLE t (col DATE) PARTITION BY DATE_TRUNC('DAY', col)

-- Multiple expressions  
CREATE TABLE t (col1 STRING, col2 BIGINT) PARTITION BY FROM_UNIXTIME(col2, '%Y%m%d'), col1

LIST partitioning:

-- Single column
CREATE TABLE t (city STRING) PARTITION BY LIST (city) (
    PARTITION pLA VALUES IN ('Los Angeles'),
    PARTITION pSF VALUES IN ('San Francisco')
)

-- Multi-column
CREATE TABLE t (dt DATE, city STRING) PARTITION BY LIST (dt, city) (
    PARTITION p1 VALUES IN (('2022-04-01', 'LA'), ('2022-04-01', 'SF'))
)

RANGE partitioning with explicit values:

CREATE TABLE t (col DATE) PARTITION BY RANGE (col) (
    PARTITION p1 VALUES LESS THAN ('2020-01-31'),
    PARTITION p2 VALUES LESS THAN ('2020-02-29'),
    PARTITION p_max VALUES LESS THAN (MAXVALUE)
)

-- With expression
CREATE TABLE t (col STRING) PARTITION BY RANGE (STR2DATE(col, '%Y-%m-%d')) (
    PARTITION p1 VALUES LESS THAN ('2021-01-01'),
    PARTITION p2 VALUES LESS THAN ('2021-01-02')
)

RANGE partitioning with START/END/EVERY:

CREATE TABLE t (col DATE) PARTITION BY RANGE (col) (
    START ('2019-01-01') END ('2021-01-01') EVERY (INTERVAL 1 YEAR),
    START ('2021-01-01') END ('2021-05-01') EVERY (INTERVAL 1 MONTH)
)

See more in tests/dialects/test_starrocks.py

Testing

All syntax variations have been validated against a local StarRocks instance (tested on StarRocks 4.0.2 and 3.5.0).

@georgesittas georgesittas requested a review from geooo109 January 20, 2026 13:31
@petrikoro petrikoro force-pushed the feat/add-full-support-for-starrocks-partitions branch from f93c63c to 09aef20 Compare January 20, 2026 13:38
@geooo109
Copy link
Collaborator

geooo109 commented Jan 20, 2026

@petrikoro thank you for the PR, great work.

I have some suggestions.

  1. There is a similar implementation in doris.py that we should check in order to factor out some code. ( a relevant commit here: 73c2894 )
  2. As I checked MySQL has some similar PARTITION BY syntax that we currently don't cover (same cases with the ones you posted without the dynamic one because it isn't supported in MySQL e.g. https://dev.mysql.com/doc/refman/8.4/en/partitioning-range.html ). We can push some implemenation in this dialect and inherit + implement some extra logic in the derived dialects, thus adding functionallity in the MySQL dialect + removing extra code from the deried dialects (Doris, Starrocks).
  3. For common patterns between Doris and Starrocks that don't exist in MySQL we can factor out in the Dialect class.

I will add some extra inline comments for help.

jaogoy added a commit to jaogoy/sqlglot that referenced this pull request Jan 21, 2026
Depends on tobymao#6804

Please review/merge tobymao#6804 first.
This PR only contains changes on top of that PR.

- Import expression partitioning for MV.
- Enabled ALTER TABLE … RENAME for StarRocks.
- Emitted ORDER BY via CLUSTER BY for StarRocks outputs.
- Added MV (REFRESH) properties handling for StarRocks materialized
views.
- And, tests updated/added for the new StarRocks behaviors.

Signed-off-by: jaogoy <jaogoy@gmail.com>
@georgesittas
Copy link
Collaborator

Hey @petrikoro 👋

Are you planning to take this to the finish line?

@petrikoro
Copy link
Contributor Author

Hey @petrikoro 👋

Are you planning to take this to the finish line?

Hi 👋

Sure, I plan to get back to PR tomorrow. Thanks for the suggestions @geooo109!

@petrikoro
Copy link
Contributor Author

@petrikoro thank you for the PR, great work.

I have some suggestions.

  1. There is a similar implementation in doris.py that we should check in order to factor out some code. ( a relevant commit here: 73c2894 )
  2. As I checked MySQL has some similar PARTITION BY syntax that we currently don't cover (same cases with the ones you posted without the dynamic one because it isn't supported in MySQL e.g. https://dev.mysql.com/doc/refman/8.4/en/partitioning-range.html ). We can push some implemenation in this dialect and inherit + implement some extra logic in the derived dialects, thus adding functionallity in the MySQL dialect + removing extra code from the deried dialects (Doris, Starrocks).
  3. For common patterns between Doris and Starrocks that don't exist in MySQL we can factor out in the Dialect class.

I will add some extra inline comments for help.

Hi! Take a look at a477f75, did I get that right?

@geooo109
Copy link
Collaborator

@petrikoro thank you very much, will check it soon.

Copy link
Collaborator

@geooo109 geooo109 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work!! left some comments.

@geooo109 geooo109 merged commit 394bba4 into tobymao:main Jan 26, 2026
9 checks passed
jaogoy added a commit to jaogoy/sqlglot that referenced this pull request Jan 28, 2026
Depends on tobymao#6804

Please review/merge tobymao#6804 first.
This PR only contains changes on top of that PR.

- Import expression partitioning for MV.
- Enabled ALTER TABLE … RENAME for StarRocks.
- Emitted ORDER BY via CLUSTER BY for StarRocks outputs.
- Added MV (REFRESH) properties handling for StarRocks materialized
views.
- And, tests updated/added for the new StarRocks behaviors.

Signed-off-by: jaogoy <jaogoy@gmail.com>
geooo109 pushed a commit that referenced this pull request Jan 28, 2026
* Feat(starrocks)!: improve some starrocks properties generation

Depends on #6804

Please review/merge #6804 first.
This PR only contains changes on top of that PR.

- Import expression partitioning for MV.
- Enabled ALTER TABLE … RENAME for StarRocks.
- Emitted ORDER BY via CLUSTER BY for StarRocks outputs.
- Added MV (REFRESH) properties handling for StarRocks materialized
views.
- And, tests updated/added for the new StarRocks behaviors.

Signed-off-by: jaogoy <jaogoy@gmail.com>

* feat(starrocks)!: optimize the code according to review comments

1. move alterrename_sql to mysql
2. implement parse_refresh_property for starrocks
3. refine refresh property generatation

Signed-off-by: jaogoy <jaogoy@gmail.com>

* optimize code style

Signed-off-by: jaogoy <jaogoy@gmail.com>

* move RefreshProperty location into base generator

Signed-off-by: jaogoy <jaogoy@gmail.com>

* rebase code

Signed-off-by: jaogoy <jaogoy@gmail.com>

* modify the parentheses for partition

Signed-off-by: jaogoy <jaogoy@gmail.com>

---------

Signed-off-by: jaogoy <jaogoy@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

StarRocks PARTITION BY expression not generated correctly

3 participants