Add query id to `system.part_log`, `system_exports` and `system.replicated_partition_exports` #1330

arthurpassos · 2026-01-22T15:49:14Z

Changelog category (leave one):

Improvement

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Add query id to system.part_log, system_exports and system.replicated_partition_exports

Documentation entry for user-facing changes

...

CI/CD Options

Exclude tests:

Regression jobs to run:

arthurpassos · 2026-01-22T15:49:30Z

I don't think automated tests are required for such a change

github-actions · 2026-01-22T15:50:09Z

Workflow [PR], commit [fe41dbd]

arthurpassos · 2026-01-22T15:50:52Z

src/Storages/MergeTree/ExportPartitionTaskScheduler.cpp

    {
        auto context_copy = Context::createCopy(context);
        context_copy->makeQueryContextForExportPart();
-        context_copy->setCurrentQueryId(manifest.transaction_id);


You might be wondering why I was setting to transaction_id. Guess what: I don't remember :).

Honestly, I think it is simply because the query id was not being saved in the manifest and it wasn't available, so transaction_id sounded like a good replacement.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1a06f236ec

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-01-22T15:54:30Z

src/Storages/ExportReplicatedMergeTreePartitionManifest.h

        ExportReplicatedMergeTreePartitionManifest manifest;
        manifest.transaction_id = json->getValue<String>("transaction_id");
+        manifest.query_id = json->getValue<String>("query_id");
        manifest.partition_id = json->getValue<String>("partition_id");


Handle missing query_id in persisted manifests

This now requires query_id to be present in every persisted manifest JSON. Manifests created by older versions (or any already-queued exports before this change) won’t have that key, so json->getValue<String>("query_id") throws and the export cannot be resumed/inspected after upgrade. That can strand in-flight replicated exports until the manifest is manually fixed. Consider defaulting query_id to transaction_id or an empty string when the key is missing to keep backward compatibility.

Useful? React with 👍 / 👎.

As always, I have to explain: this feature is still very experimental and not used at all. Completely fine to make breaking changes

mkmkme

LGTM

ianton-ru

LGTM

DimensionWieldr · 2026-01-26T09:04:43Z

Export part test suite covers system.part_log and system.exports expected table columns. All tests pass.

Link to relevant test: https://github.com/Altinity/clickhouse-regression/blob/main/s3/tests/export_part/system_monitoring.py#L381

Selfeer · 2026-01-26T19:49:33Z

Wrote an automated tests that checks every field is present and populated after export partition in system.replicated_partition_exports - all tests pass.

link to tests: https://github.com/Altinity/clickhouse-regression/blob/0a049a9dcdf96b5396277be15e8c8f120f4950fb/s3/tests/export_partition/system_monitoring.py#L338

Passing

✔ [ OK ] '/s3/minio/export tests/export partition/system monitoring/all required fields present' (10s 109ms)
✔ [ OK ] '/s3/minio/export tests/export partition/system monitoring' (10s 110ms)
✔ [ OK ] '/s3/minio/export tests/export partition' (54s 582ms)
✔ [ OK ] '/s3/minio/export tests' (54s 596ms)
✔ [ OK ] '/s3/minio' (1m 43s)
✔ [ OK ] '/s3' (1m 43s)

zvonand · 2026-01-27T16:43:44Z

03321_clickhouse_local_initialization_not_too_slow_even_under_sanitizers is failing twice in the latest run

Selfeer · 2026-01-28T15:31:33Z

Summary of Failed Tests and Root Causes

Test name	Status	Reason for failure
03321_clickhouse_local_initialization_not_too_slow_even_under_sanitizers	Failed	Startup-performance canary for `clickhouse-local`. Within the ~10-second window, `clickhouse-local -q 1` started, executed, and exited 9 times, while the test expects at least 10. This indicates slower-than-expected process initialization on this CI run, not a functional or correctness issue. The test logic itself has not changed recently (last edited ~11 months ago in upstream master), suggesting an environment/runtime performance issue rather than a test change.
test_s3_cache_locality/test.py::test_cache_locality[0]	Failed	Marked as BROKEN after the last CI/CD run so it will be included in the broken list on the next run.
export_part related tests	Failed	The commit hash was outdated, so an old version of `export_part` was used. Some of the failing tests no longer exist, as they have been renamed or updated.

Selfeer · 2026-01-28T16:07:53Z

The CI/CD run is expected to be red, we will monitor the final run in the branch. @arthurpassos please merge this PR.

arthurpassos · 2026-01-28T16:08:49Z

The CI/CD run is expected to be red, we will monitor the final run in the branch. @arthurpassos please merge this PR.

I don't have the powers to do it. Perhaps @zvonand

zvonand · 2026-01-28T16:15:52Z

94 jobs are in progress yet

add query id to part log and system exports

1a06f23

arthurpassos added antalya port-antalya PRs to be ported to all new Antalya releases antalya-25.8 labels Jan 22, 2026

arthurpassos commented Jan 22, 2026

View reviewed changes

chatgpt-codex-connector bot reviewed Jan 22, 2026

View reviewed changes

mkmkme previously approved these changes Jan 22, 2026

View reviewed changes

ianton-ru previously approved these changes Jan 22, 2026

View reviewed changes

svb-alt added antalya-25.8.14.20001 antalya-25.8.14 labels Jan 22, 2026

Selfeer added verified Verified by QA and removed verified Verified by QA labels Jan 26, 2026

Merge branch 'antalya-25.8' into query_id_on_part_log

f916d8d

Update regression hash

fe41dbd

CarlosFelipeOR dismissed stale reviews from mkmkme and ianton-ru via fe41dbd January 28, 2026 13:27

Selfeer added the verified Verified by QA label Jan 28, 2026

MyroTk merged commit f3a7dce into antalya-25.8 Jan 29, 2026
524 of 728 checks passed

CarlosFelipeOR mentioned this pull request Jan 29, 2026

Antalya 25.8: Update regression hash - Fix Export Partition #1352

Merged

25 tasks

DimensionWieldr mentioned this pull request Jan 29, 2026

Accept table function as destination for part export #1320

Merged

25 tasks

arthurpassos mentioned this pull request Jan 29, 2026

query_id in system.part_log from part export #1322

Closed

Add query id to system.part_log, system_exports and system.replicated_partition_exports #1330

Add query id to system.part_log, system_exports and system.replicated_partition_exports #1330

Uh oh!

Conversation

arthurpassos commented Jan 22, 2026 • edited by DimensionWieldr Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changelog category (leave one):

Changelog entry (a user-readable short description of the changes that goes to CHANGELOG.md):

Documentation entry for user-facing changes

CI/CD Options

Exclude tests:

Regression jobs to run:

Uh oh!

arthurpassos commented Jan 22, 2026

Uh oh!

github-actions bot commented Jan 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

arthurpassos Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector bot Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

arthurpassos Jan 22, 2026

Choose a reason for hiding this comment

Uh oh!

mkmkme left a comment

Choose a reason for hiding this comment

Uh oh!

ianton-ru left a comment

Choose a reason for hiding this comment

Uh oh!

DimensionWieldr commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Selfeer commented Jan 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zvonand commented Jan 27, 2026

Uh oh!

Selfeer commented Jan 28, 2026

Uh oh!

Selfeer commented Jan 28, 2026

Uh oh!

arthurpassos commented Jan 28, 2026

Uh oh!

zvonand commented Jan 28, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

10 participants

Add query id to `system.part_log`, `system_exports` and `system.replicated_partition_exports` #1330

Add query id to `system.part_log`, `system_exports` and `system.replicated_partition_exports` #1330

arthurpassos commented Jan 22, 2026 •

edited by DimensionWieldr

Loading

github-actions bot commented Jan 22, 2026 •

edited

Loading

DimensionWieldr commented Jan 26, 2026 •

edited

Loading

Selfeer commented Jan 26, 2026 •

edited

Loading