From 5fddc50bf5d089deccf02efc7aee25fa26d7d033 Mon Sep 17 00:00:00 2001 From: Anton Ivashkin Date: Mon, 24 Nov 2025 19:36:34 +0100 Subject: [PATCH 1/4] Swarm documentation --- .../table-engines/integrations/iceberg.md | 58 ++++++++++++++ .../sql-reference/distribution-on-cluster.md | 12 +++ .../azureBlobStorageCluster.md | 14 ++++ .../table-functions/deltalakeCluster.md | 11 +++ .../table-functions/hdfsCluster.md | 12 +++ .../table-functions/hudiCluster.md | 12 +++ .../sql-reference/table-functions/iceberg.md | 43 ++++++++++ .../table-functions/icebergCluster.md | 78 +++++++++++++++++++ .../table-functions/s3Cluster.md | 17 ++++ 9 files changed, 257 insertions(+) create mode 100644 docs/en/sql-reference/distribution-on-cluster.md diff --git a/docs/en/engines/table-engines/integrations/iceberg.md b/docs/en/engines/table-engines/integrations/iceberg.md index e667dca4ebd4..582b10ba1c1d 100644 --- a/docs/en/engines/table-engines/integrations/iceberg.md +++ b/docs/en/engines/table-engines/integrations/iceberg.md @@ -296,6 +296,64 @@ CREATE TABLE example_table ENGINE = Iceberg( `Iceberg` table engine and table function support metadata cache storing the information of manifest files, manifest list and metadata json. The cache is stored in memory. This feature is controlled by setting `use_iceberg_metadata_files_cache`, which is enabled by default. +## Altinity Antalya branch + +### Specify storage type in arguments + +Only in Altinity Antalya branch table engine `Iceberg` can work with all storages. +In this case storage may be specified with named argument `storage_type`. +Possible values are `s3`, `azure`, `hdfs`, `local`: + +```sql +CREATE TABLE iceberg_table_s3 + ENGINE = Iceberg(storage_type='s3', url, [, NOSIGN | access_key_id, secret_access_key, [session_token]], format, [,compression]) + +CREATE TABLE iceberg_table_azure + ENGINE = Iceberg(storage_type='azure', connection_string|storage_account_url, container_name, blobpath, [account_name, account_key, format, compression]) + +CREATE TABLE iceberg_table_hdfs + ENGINE = Iceberg(storage_type='hdfs', path_to_table, [,format] [,compression_method]) + +CREATE TABLE iceberg_table_local + ENGINE = Iceberg(storage_type='local', path_to_table, [,format] [,compression_method]) +``` + +### Specify storage type in named collection + +Only in Altinity Antalya branch `storage_type` may be part of named collection. + +```xml + + + + http://test.s3.amazonaws.com/clickhouse-bucket/ + test + test + auto + auto + s3 + + + +``` + +```sql +CREATE TABLE iceberg_table ENGINE=Iceberg(iceberg_conf, filename = 'test_table') +``` + +Default value for `storage_type` is `s3`. + +### `object_storage_cluster` setting. + +Only in Altinity Antalya branch alternative syntax for `Iceberg` table engine can be executed on cluster with non-empty setting `object_storage_cluster` with cluster name. + +```sql +CREATE TABLE iceberg_table_s3 + ENGINE = Iceberg(storage_type='s3', url, [, NOSIGN | access_key_id, secret_access_key, [session_token]], format, [,compression]); + +SELECT * FROM iceberg_table_s3 SETTINGS object_storage_cluster='cluster_simple'; +``` + ## See also {#see-also} - [iceberg table function](/sql-reference/table-functions/iceberg.md) diff --git a/docs/en/sql-reference/distribution-on-cluster.md b/docs/en/sql-reference/distribution-on-cluster.md new file mode 100644 index 000000000000..62b5a9f60492 --- /dev/null +++ b/docs/en/sql-reference/distribution-on-cluster.md @@ -0,0 +1,12 @@ +# Task distribution in *Cluster family functions + +Table functions `s3Cluster`, `azureBlobStorageCluster`, `hdsfCluster`, `icebergCluster`, etc., as well as table engines `S3`, `Azure`, `HDFS`, `Iceberg` with setting `object_storage_cluster` distribute tasks between all cluster nodes, or only between `object_storage_max_nodes` cluster nodes. Setting `object_storage_max_nodes` limits number of nodes to process distributed query. In this case random nodes are choosed for each query. + +Single task is a single source file. + +For each file one of cluster nodes selected as primary node. Primary node selected with consistence Rendezvous Hashing algorythm, this algorythm guarantees that for each file sthe same node selected as primary when cluster not changed, and when cluster is changed only files from deleted nodes or with new nodes as primary are affected. This incerasy cache efficiency. +Each node starts to process files for which this node is primary. When node processes all those files it can take some files from other nodes. Node gets file immediately or only when primary node does not ask for new files in `lock_object_storage_task_distribution_ms` milliseconds. Settiing `lock_object_storage_task_distribution_ms` has default value in 500 milliseconds, and can be used to achieve balance between caching and processing files when some nodes are overloaded. + +If node must be shutdowned in some time, command `SYSTEM STOP SWARM MODE` can be used to stop getting new tasks for `*Cluster`-family queries. In this case node stop getting new files, but processes alrready started files. When it processes all files, node can be shutdowned withou any error on initiator. + +Getting new tasks can be reenabled with `SYSTEM START SWARM MODE` command. diff --git a/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md b/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md index 95011c7d6186..e7695f8eab83 100644 --- a/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md +++ b/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md @@ -53,6 +53,20 @@ SELECT count(*) FROM azureBlobStorageCluster( See [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage#using-shared-access-signatures-sas-sas-tokens) for examples. +## Altinity Antalya branch + +### `object_storage_cluster` setting. + +Only in Altinity Antalya branch alternative syntax for `azureBlobStorageCluster` table function is `azureBlobStorage` function with non-empty setting `object_storage_cluster` with cluster name. + +```sql +SELECT count(*) FROM azureBlobStorage( + 'http://azurite1:10000/devstoreaccount1', 'testcontainer', 'test_cluster_count.csv', 'devstoreaccount1', + 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==', 'CSV', + 'auto', 'key UInt64') +SETTINGS object_storage_cluster='cluster_simple' +``` + ## Related {#related} - [AzureBlobStorage engine](../../engines/table-engines/integrations/azureBlobStorage.md) diff --git a/docs/en/sql-reference/table-functions/deltalakeCluster.md b/docs/en/sql-reference/table-functions/deltalakeCluster.md index 5a36565c67f0..3cedf4cc2998 100644 --- a/docs/en/sql-reference/table-functions/deltalakeCluster.md +++ b/docs/en/sql-reference/table-functions/deltalakeCluster.md @@ -36,6 +36,17 @@ A table with the specified structure for reading data from cluster in the specif - `_time` — Last modified time of the file. Type: `Nullable(DateTime)`. If the time is unknown, the value is `NULL`. - `_etag` — The etag of the file. Type: `LowCardinality(String)`. If the etag is unknown, the value is `NULL`. +## Altinity Antalya branch + +### `object_storage_cluster` setting. + +Only in Altinity Antalya branch alternative syntax for `deltaLakeCluster` table function is `deltaLake` function with non-empty setting `object_storage_cluster` with cluster name. + +```sql +SELECT count(*) FROM deltaLake(url [,aws_access_key_id, aws_secret_access_key] [,format] [,structure] [,compression]) +SETTINGS object_storage_cluster='cluster_simple' +``` + ## Related {#related} - [deltaLake engine](engines/table-engines/integrations/deltalake.md) diff --git a/docs/en/sql-reference/table-functions/hdfsCluster.md b/docs/en/sql-reference/table-functions/hdfsCluster.md index a2556e86ae97..6d8868329dbb 100644 --- a/docs/en/sql-reference/table-functions/hdfsCluster.md +++ b/docs/en/sql-reference/table-functions/hdfsCluster.md @@ -59,6 +59,18 @@ FROM hdfsCluster('cluster_simple', 'hdfs://hdfs1:9000/{some,another}_dir/*', 'TS If your listing of files contains number ranges with leading zeros, use the construction with braces for each digit separately or use `?`. ::: +## Altinity Antalya branch + +### `object_storage_cluster` setting. + +Only in Altinity Antalya branch alternative syntax for `hdfsCluster` table function is `hdfs` function with non-empty setting `object_storage_cluster` with cluster name. + +```sql +SELECT count(*) +FROM hdfs('hdfs://hdfs1:9000/{some,another}_dir/*', 'TSV', 'name String, value UInt32') +SETTINGS object_storage_cluster='cluster_simple' +``` + ## Related {#related} - [HDFS engine](../../engines/table-engines/integrations/hdfs.md) diff --git a/docs/en/sql-reference/table-functions/hudiCluster.md b/docs/en/sql-reference/table-functions/hudiCluster.md index 974ac8bb7f7b..7f14bcc08841 100644 --- a/docs/en/sql-reference/table-functions/hudiCluster.md +++ b/docs/en/sql-reference/table-functions/hudiCluster.md @@ -42,6 +42,18 @@ A table with the specified structure for reading data from cluster in the specif - `_time` — Last modified time of the file. Type: `Nullable(DateTime)`. If the time is unknown, the value is `NULL`. - `_etag` — The etag of the file. Type: `LowCardinality(String)`. If the etag is unknown, the value is `NULL`. +## Altinity Antalya branch + +### `object_storage_cluster` setting. + +Only in Altinity Antalya branch alternative syntax for `hudiCluster` table function is `hudi` function with non-empty setting `object_storage_cluster` with cluster name. + +```sql +SELECT * +FROM hudi(url [,aws_access_key_id, aws_secret_access_key] [,format] [,structure] [,compression]) +SETTINGS object_storage_cluster='cluster_simple' +``` + ## Related {#related} - [Hudi engine](engines/table-engines/integrations/hudi.md) diff --git a/docs/en/sql-reference/table-functions/iceberg.md b/docs/en/sql-reference/table-functions/iceberg.md index 92282e871f44..d13056af65b9 100644 --- a/docs/en/sql-reference/table-functions/iceberg.md +++ b/docs/en/sql-reference/table-functions/iceberg.md @@ -302,6 +302,49 @@ Table function `iceberg` is an alias to `icebergS3` now. - `_time` — Last modified time of the file. Type: `Nullable(DateTime)`. If the time is unknown, the value is `NULL`. - `_etag` — The etag of the file. Type: `LowCardinality(String)`. If the etag is unknown, the value is `NULL`. +## Altinity Antalya branch + +### Specify storage type in arguments + +Only in Altinity Antalya branch table function `iceberg` can work with all storages. +In this case storage may be specified with named argument `storage_type`. +Possible values are `s3`, `azure`, `hdfs`, `local`: + +```sql +iceberg(storage_type='s3', url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) + +iceberg(storage_type='azure', connection_string|storage_account_url, container_name, blobpath, [,account_name], [,account_key] [,format] [,compression_method]) + +iceberg(storage_type='hdfs', path_to_table, [,format] [,compression_method]) + +iceberg(storage_type='local', path_to_table, [,format] [,compression_method]) +``` + +### Specify storage type in named collection + +Only in Altinity Antalya branch `storage_type` may be part of named collection. + +```xml + + + + http://test.s3.amazonaws.com/clickhouse-bucket/ + test + test + auto + auto + s3 + + + +``` + +```sql +iceberg(named_collection[, option=value [,..]]) +``` + +Default value for `storage_type` is `s3`. + ## See Also {#see-also} * [Iceberg engine](/engines/table-engines/integrations/iceberg.md) diff --git a/docs/en/sql-reference/table-functions/icebergCluster.md b/docs/en/sql-reference/table-functions/icebergCluster.md index f66151d002cb..96857ccd8f71 100644 --- a/docs/en/sql-reference/table-functions/icebergCluster.md +++ b/docs/en/sql-reference/table-functions/icebergCluster.md @@ -49,6 +49,84 @@ SELECT * FROM icebergS3Cluster('cluster_simple', 'http://test.s3.amazonaws.com/c - `_time` — Last modified time of the file. Type: `Nullable(DateTime)`. If the time is unknown, the value is `NULL`. - `_etag` — The etag of the file. Type: `LowCardinality(String)`. If the etag is unknown, the value is `NULL`. +## Altinity Antalya branch + +### `icebergLocalCluster` table function + +Only in Altinity Antalya branch `icebergLocalCluster` is a function to make distributed cluster request when iceberg data is storend on shared network storage, mounted with local path. +Path must be the same on all replicas. + +```sql +icebergLocalCluster(cluster_name, path_to_table, [,format] [,compression_method]) +``` + +### Specify storage type in arguments + +Only in Altinity Antalya branch table function `iceberg` can work with all storages. +In this case storage may be specified with named argument `storage_type`. +Possible values are `s3`, `azure`, `hdfs`, `local`: + +```sql +icebergCluster(storage_type='s3', cluster_name, url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) + +icebergCluster(storage_type='azure', cluster_name, connection_string|storage_account_url, container_name, blobpath, [,account_name], [,account_key] [,format] [,compression_method]) + +icebergCluster(storage_type='hdfs', cluster_name, path_to_table, [,format] [,compression_method]) + +icebergCluster(storage_type='local', cluster_name, path_to_table, [,format] [,compression_method]) +``` + +### Specify storage type in named collection + +Only in Altinity Antalya branch `storage_type` may be part of named collection. + +```xml + + + + http://test.s3.amazonaws.com/clickhouse-bucket/ + test + test + auto + auto + s3 + + + +``` + +```sql +icebergCluster(iceberg_conf[, option=value [,..]]) +``` + +Default value for `storage_type` is `s3`. + +### `object_storage_cluster` setting. + +Only in Altinity Antalya branch alternative syntax for `icebergCluster` table function is `iceberg` function with non-empty setting `object_storage_cluster` with cluster name. + +```sql +icebergS3(url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +icebergAzure(connection_string|storage_account_url, container_name, blobpath, [,account_name], [,account_key] [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +icebergHDSF(path_to_table, [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +icebergLocal(path_to_table, [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +icebergS3(option=value [,..]]) SETTINGS object_storage_cluster='cluster_name' + +iceberg(storage_type='s3', url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +iceberg(storage_type='azure', connection_string|storage_account_url, container_name, blobpath, [,account_name], [,account_key] [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +iceberg(storage_type='hdfs', path_to_table, [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +iceberg(storage_type='local', path_to_table, [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' + +iceberg(iceberg_conf[, option=value [,..]]) SETTINGS object_storage_cluster='cluster_name' +``` + **See Also** - [Iceberg engine](/engines/table-engines/integrations/iceberg.md) diff --git a/docs/en/sql-reference/table-functions/s3Cluster.md b/docs/en/sql-reference/table-functions/s3Cluster.md index 2023a3777380..91626cac2fe8 100644 --- a/docs/en/sql-reference/table-functions/s3Cluster.md +++ b/docs/en/sql-reference/table-functions/s3Cluster.md @@ -90,6 +90,23 @@ Users can use the same approaches as document for the s3 function [here](/sql-re For details on optimizing the performance of the s3 function see [our detailed guide](/integrations/s3/performance). +## Altinity Antalya branch + +### `object_storage_cluster` setting. + +Only in Altinity Antalya branch alternative syntax for `s3Cluster` table function is `s3` function with non-empty setting `object_storage_cluster` with cluster name. + +```sql +SELECT * FROM s3( + 'http://minio1:9001/root/data/{clickhouse,database}/*', + 'minio', + 'ClickHouse_Minio_P@ssw0rd', + 'CSV', + 'name String, value UInt32, polygon Array(Array(Tuple(Float64, Float64)))' +) ORDER BY (name, value, polygon) +SETTINGS object_storage_cluster='cluster_simple' +``` + ## Related {#related} - [S3 engine](../../engines/table-engines/integrations/s3.md) From af3304d6d30fbcd1f9a7882f278e1b9c3f10ca0c Mon Sep 17 00:00:00 2001 From: Anton Ivashkin Date: Mon, 24 Nov 2025 20:30:40 +0100 Subject: [PATCH 2/4] Better English --- docs/en/antalya/swarm.md | 73 +++++++++++++++++++ .../table-engines/integrations/iceberg.md | 12 ++- .../sql-reference/distribution-on-cluster.md | 23 ++++-- .../azureBlobStorageCluster.md | 2 +- .../table-functions/deltalakeCluster.md | 2 +- .../table-functions/hdfsCluster.md | 2 +- .../table-functions/hudiCluster.md | 2 +- .../sql-reference/table-functions/iceberg.md | 8 +- .../table-functions/icebergCluster.md | 17 ++--- .../table-functions/s3Cluster.md | 2 +- 10 files changed, 110 insertions(+), 33 deletions(-) create mode 100644 docs/en/antalya/swarm.md diff --git a/docs/en/antalya/swarm.md b/docs/en/antalya/swarm.md new file mode 100644 index 000000000000..a26f9de26e0a --- /dev/null +++ b/docs/en/antalya/swarm.md @@ -0,0 +1,73 @@ +# Antalya branch + +## Swarm + +### Difference with upstream version + +#### `storage_type` argument in object storage functions + +In upstream ClickHouse, there are several table functions to read Iceberg tables from different storage backends such as `icebergLocal`, `icebergS3`, `icebergAzure`, `icebergHDFS`, cluster variants, the `iceberg` function as a synonym for `icebergS3`, and table engines like `IcebergLocal`, `IcebergS3`, `IcebergAzure`, `IcebergHDFS`. + +In the Antalya branch, the `iceberg` table function and the `Iceberg` table engine unify all variants into one by using a new named argument, `storage_type`, which can be one of `local`, `s3`, `azure`, or `hdfs`. + +Old syntax examples: + +```sql +SELECT * FROM icebergS3('http://minio1:9000/root/table_data', 'minio', 'minio123', 'Parquet'); +SELECT * FROM icebergAzureCluster('mycluster', 'http://azurite1:30000/devstoreaccount1', 'cont', '/table_data', 'devstoreaccount1', 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==', 'Parquet'); +CREATE TABLE mytable ENGINE=IcebergHDFS('/table_data', 'Parquet'); +``` + +New syntax examples: + +```sql +SELECT * FROM iceberg(storage_type='s3', 'http://minio1:9000/root/table_data', 'minio', 'minio123', 'Parquet'); +SELECT * FROM icebergCluster('mycluster', storage_type='azure', 'http://azurite1:30000/devstoreaccount1', 'cont', '/table_data', 'devstoreaccount1', 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==', 'Parquet'); +CREATE TABLE mytable ENGINE=Iceberg('/table_data', 'Parquet', storage_type='hdfs'); +``` + +Also, if a named collection is used to store access parameters, the field `storage_type` can be included in the same named collection: + +```xml + + + http://minio1:9001/root/ + minio + minio123 + s3 + + +``` + +```sql +SELECT * FROM iceberg(s3, filename='table_data'); +``` + +By default `storage_type` is `'s3'` to maintain backward compatibility. + + +#### `object_storage_cluster` setting + +The new setting `object_storage_cluster` controls whether a single-node or cluster variant of table functions reading from object storage (e.g., `s3`, `azure`, `iceberg`, and their cluster variants like `s3Cluster`, `azureCluster`, `icebergCluster`) is used. + +Old syntax examples: + +```sql +SELECT * from s3Cluster('myCluster', 'http://minio1:9001/root/data/{clickhouse,database}/*', 'minio', 'minio123', 'CSV', + 'name String, value UInt32, polygon Array(Array(Tuple(Float64, Float64)))'); +SELECT * FROM icebergAzureCluster('mycluster', 'http://azurite1:30000/devstoreaccount1', 'cont', '/table_data', 'devstoreaccount1', 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==', 'Parquet'); +``` + +New syntax examples: + +```sql +SELECT * from s3('http://minio1:9001/root/data/{clickhouse,database}/*', 'minio', 'minio123', 'CSV', + 'name String, value UInt32, polygon Array(Array(Tuple(Float64, Float64)))') + SETTINGS object_storage_cluster='myCluster'; +SELECT * FROM icebergAzure('http://azurite1:30000/devstoreaccount1', 'cont', '/table_data', 'devstoreaccount1', 'Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==', 'Parquet') + SETTINGS object_storage_cluster='myCluster'; +``` + +This setting also applies to table engines and can be used with tables managed by Iceberg Catalog. + +Note: The upstream ClickHouse has introduced analogous settings, such as `parallel_replicas_for_cluster_engines` and `cluster_for_parallel_replicas`. Since version 25.10, these settings work with table engines. It is possible that in the future, the `object_storage_cluster` setting will be deprecated. diff --git a/docs/en/engines/table-engines/integrations/iceberg.md b/docs/en/engines/table-engines/integrations/iceberg.md index 582b10ba1c1d..60c959ce7761 100644 --- a/docs/en/engines/table-engines/integrations/iceberg.md +++ b/docs/en/engines/table-engines/integrations/iceberg.md @@ -300,9 +300,7 @@ CREATE TABLE example_table ENGINE = Iceberg( ### Specify storage type in arguments -Only in Altinity Antalya branch table engine `Iceberg` can work with all storages. -In this case storage may be specified with named argument `storage_type`. -Possible values are `s3`, `azure`, `hdfs`, `local`: +Only in the Altinity Antalya branch does `Iceberg` table engine support all storage types. The storage type can be specified using the named argument `storage_type`. Supported values are `s3`, `azure`, `hdfs`, and `local`. ```sql CREATE TABLE iceberg_table_s3 @@ -320,7 +318,7 @@ CREATE TABLE iceberg_table_local ### Specify storage type in named collection -Only in Altinity Antalya branch `storage_type` may be part of named collection. +Only in Altinity Antalya branch `storage_type` can be included as part of a named collection. This allows for centralized configuration of storage settings. ```xml @@ -341,11 +339,11 @@ Only in Altinity Antalya branch `storage_type` may be part of named collection. CREATE TABLE iceberg_table ENGINE=Iceberg(iceberg_conf, filename = 'test_table') ``` -Default value for `storage_type` is `s3`. +The default value for `storage_type` is `s3`. -### `object_storage_cluster` setting. +### The `object_storage_cluster` setting. -Only in Altinity Antalya branch alternative syntax for `Iceberg` table engine can be executed on cluster with non-empty setting `object_storage_cluster` with cluster name. +Only in the Altinity Antalya branch is an alternative syntax for the `Iceberg` table engine available. This syntax allows execution on a cluster when the `object_storage_cluster` setting is non-empty and contains the cluster name. ```sql CREATE TABLE iceberg_table_s3 diff --git a/docs/en/sql-reference/distribution-on-cluster.md b/docs/en/sql-reference/distribution-on-cluster.md index 62b5a9f60492..e563961b4286 100644 --- a/docs/en/sql-reference/distribution-on-cluster.md +++ b/docs/en/sql-reference/distribution-on-cluster.md @@ -1,12 +1,23 @@ # Task distribution in *Cluster family functions -Table functions `s3Cluster`, `azureBlobStorageCluster`, `hdsfCluster`, `icebergCluster`, etc., as well as table engines `S3`, `Azure`, `HDFS`, `Iceberg` with setting `object_storage_cluster` distribute tasks between all cluster nodes, or only between `object_storage_max_nodes` cluster nodes. Setting `object_storage_max_nodes` limits number of nodes to process distributed query. In this case random nodes are choosed for each query. +## Task distribution algorythm -Single task is a single source file. +Table functions such as `s3Cluster`, `azureBlobStorageCluster`, `hdsfCluster`, `icebergCluster`, and table engines like `S3`, `Azure`, `HDFS`, `Iceberg` with the setting `object_storage_cluster` distribute tasks across all cluster nodes or a subset limited by the `object_storage_max_nodes` setting. This setting limits the number of nodes involved in processing a distributed query, randomly selecting nodes for each query. -For each file one of cluster nodes selected as primary node. Primary node selected with consistence Rendezvous Hashing algorythm, this algorythm guarantees that for each file sthe same node selected as primary when cluster not changed, and when cluster is changed only files from deleted nodes or with new nodes as primary are affected. This incerasy cache efficiency. -Each node starts to process files for which this node is primary. When node processes all those files it can take some files from other nodes. Node gets file immediately or only when primary node does not ask for new files in `lock_object_storage_task_distribution_ms` milliseconds. Settiing `lock_object_storage_task_distribution_ms` has default value in 500 milliseconds, and can be used to achieve balance between caching and processing files when some nodes are overloaded. +A single task corresponds to processing one source file. -If node must be shutdowned in some time, command `SYSTEM STOP SWARM MODE` can be used to stop getting new tasks for `*Cluster`-family queries. In this case node stop getting new files, but processes alrready started files. When it processes all files, node can be shutdowned withou any error on initiator. +For each file, one cluster node is selected as the primary node using a consistent Rendezvous Hashing algorithm. This algorithm guarantees that: + * The same node is consistently selected as primary for each file, as long as the cluster remains unchanged. + * When the cluster changes (nodes added or removed), only files assigned to those affected nodes change their primary node assignment. -Getting new tasks can be reenabled with `SYSTEM START SWARM MODE` command. +This improves cache efficiency by minimizing data movement among nodes. + +## `lock_object_storage_task_distribution_ms` setting + +Each node begins processing files for which it is the primary node. After completing its assigned files, a node may take tasks from other nodes, either immediately or after waiting for `lock_object_storage_task_distribution_ms` milliseconds if the primary node does not request new files during that interval. The default value of `lock_object_storage_task_distribution_ms` is 500 milliseconds. This setting balances between caching efficiency and workload redistribution when nodes are imbalanced. + +## `SYSTEM STOP SWARM MODE` command + +If a node needs to shut down gracefully, the command `SYSTEM STOP SWARM MODE` prevents the node from receiving new tasks for *Cluster-family queries. The node finishes processing already assigned files before it can safely shut down without errors. + +Receiving new tasks can be resumed with the command `SYSTEM START SWARM MODE`. diff --git a/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md b/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md index e7695f8eab83..c5ad04b739e4 100644 --- a/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md +++ b/docs/en/sql-reference/table-functions/azureBlobStorageCluster.md @@ -57,7 +57,7 @@ See [azureBlobStorage](/sql-reference/table-functions/azureBlobStorage#using-sha ### `object_storage_cluster` setting. -Only in Altinity Antalya branch alternative syntax for `azureBlobStorageCluster` table function is `azureBlobStorage` function with non-empty setting `object_storage_cluster` with cluster name. +Only in the Altinity Antalya branch, the alternative syntax for the `azureBlobStorageCluster` table function is avilable. This allows the `azureBlobStorage` function to be used with the non-empty `object_storage_cluster` setting, specifying a cluster name. This enables distributed queries over Azure Blob Storage across a ClickHouse cluster. ```sql SELECT count(*) FROM azureBlobStorage( diff --git a/docs/en/sql-reference/table-functions/deltalakeCluster.md b/docs/en/sql-reference/table-functions/deltalakeCluster.md index 3cedf4cc2998..073bd5a41440 100644 --- a/docs/en/sql-reference/table-functions/deltalakeCluster.md +++ b/docs/en/sql-reference/table-functions/deltalakeCluster.md @@ -40,7 +40,7 @@ A table with the specified structure for reading data from cluster in the specif ### `object_storage_cluster` setting. -Only in Altinity Antalya branch alternative syntax for `deltaLakeCluster` table function is `deltaLake` function with non-empty setting `object_storage_cluster` with cluster name. +Only in the Altinity Antalya branch alternative syntax for `deltaLakeCluster` table function is available. This allows the `deltaLake` function to be used with the non-empty `object_storage_cluster` setting, specifying a cluster name. This enables distributed queries over Delta Lake Storage across a ClickHouse cluster. ```sql SELECT count(*) FROM deltaLake(url [,aws_access_key_id, aws_secret_access_key] [,format] [,structure] [,compression]) diff --git a/docs/en/sql-reference/table-functions/hdfsCluster.md b/docs/en/sql-reference/table-functions/hdfsCluster.md index 6d8868329dbb..72a87276a5bd 100644 --- a/docs/en/sql-reference/table-functions/hdfsCluster.md +++ b/docs/en/sql-reference/table-functions/hdfsCluster.md @@ -63,7 +63,7 @@ If your listing of files contains number ranges with leading zeros, use the cons ### `object_storage_cluster` setting. -Only in Altinity Antalya branch alternative syntax for `hdfsCluster` table function is `hdfs` function with non-empty setting `object_storage_cluster` with cluster name. +Only in the Altinity Antalya branch alternative syntax for `hdfsCluster` table function is available. This allows the `hdfs` function to be used with the non-empty `object_storage_cluster` setting, specifying a cluster name. This enables distributed queries over HDFS Storage across a ClickHouse cluster. ```sql SELECT count(*) diff --git a/docs/en/sql-reference/table-functions/hudiCluster.md b/docs/en/sql-reference/table-functions/hudiCluster.md index 7f14bcc08841..e409c0a9f906 100644 --- a/docs/en/sql-reference/table-functions/hudiCluster.md +++ b/docs/en/sql-reference/table-functions/hudiCluster.md @@ -46,7 +46,7 @@ A table with the specified structure for reading data from cluster in the specif ### `object_storage_cluster` setting. -Only in Altinity Antalya branch alternative syntax for `hudiCluster` table function is `hudi` function with non-empty setting `object_storage_cluster` with cluster name. +Only in the Altinity Antalya branch alternative syntax for `hudiCluster` table function is available. This allows the `hudi` function to be used with the non-empty `object_storage_cluster` setting, specifying a cluster name. This enables distributed queries over Hudi Storage across a ClickHouse cluster. ```sql SELECT * diff --git a/docs/en/sql-reference/table-functions/iceberg.md b/docs/en/sql-reference/table-functions/iceberg.md index d13056af65b9..c3bf7fc41921 100644 --- a/docs/en/sql-reference/table-functions/iceberg.md +++ b/docs/en/sql-reference/table-functions/iceberg.md @@ -306,9 +306,7 @@ Table function `iceberg` is an alias to `icebergS3` now. ### Specify storage type in arguments -Only in Altinity Antalya branch table function `iceberg` can work with all storages. -In this case storage may be specified with named argument `storage_type`. -Possible values are `s3`, `azure`, `hdfs`, `local`: +Only in the Altinity Antalya branch does the `iceberg` table function support all storage types. The storage type can be specified using the named argument `storage_type`. Supported values are `s3`, `azure`, `hdfs`, and `local`. ```sql iceberg(storage_type='s3', url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) @@ -322,7 +320,7 @@ iceberg(storage_type='local', path_to_table, [,format] [,compression_method]) ### Specify storage type in named collection -Only in Altinity Antalya branch `storage_type` may be part of named collection. +Only in the Altinity Antalya branch can storage_type be included as part of a named collection. This allows for centralized configuration of storage settings. ```xml @@ -343,7 +341,7 @@ Only in Altinity Antalya branch `storage_type` may be part of named collection. iceberg(named_collection[, option=value [,..]]) ``` -Default value for `storage_type` is `s3`. +The default value for `storage_type` is `s3`. ## See Also {#see-also} diff --git a/docs/en/sql-reference/table-functions/icebergCluster.md b/docs/en/sql-reference/table-functions/icebergCluster.md index 96857ccd8f71..15d54609e5ac 100644 --- a/docs/en/sql-reference/table-functions/icebergCluster.md +++ b/docs/en/sql-reference/table-functions/icebergCluster.md @@ -53,18 +53,15 @@ SELECT * FROM icebergS3Cluster('cluster_simple', 'http://test.s3.amazonaws.com/c ### `icebergLocalCluster` table function -Only in Altinity Antalya branch `icebergLocalCluster` is a function to make distributed cluster request when iceberg data is storend on shared network storage, mounted with local path. -Path must be the same on all replicas. +Only in the Altinity Antalya branch, `icebergLocalCluster` designed to make distributed cluster queries when Iceberg data is stored on shared network storage mounted with a local path. The path must be identical on all replicas. ```sql icebergLocalCluster(cluster_name, path_to_table, [,format] [,compression_method]) ``` -### Specify storage type in arguments +### Specify storage type in function arguments -Only in Altinity Antalya branch table function `iceberg` can work with all storages. -In this case storage may be specified with named argument `storage_type`. -Possible values are `s3`, `azure`, `hdfs`, `local`: +Only in the Altinity Antalya branch, the `icebergCluster` table function supports all storage backends. The storage backend can be specified using the named argument `storage_type`. Valid values include `s3`, `azure`, `hdfs`, and `local`. ```sql icebergCluster(storage_type='s3', cluster_name, url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) @@ -76,9 +73,9 @@ icebergCluster(storage_type='hdfs', cluster_name, path_to_table, [,format] [,com icebergCluster(storage_type='local', cluster_name, path_to_table, [,format] [,compression_method]) ``` -### Specify storage type in named collection +### Specify storage type in a named collection -Only in Altinity Antalya branch `storage_type` may be part of named collection. +Only in the Altinity Antalya branch, `storage_type` can be part of a named collection. ```xml @@ -99,11 +96,11 @@ Only in Altinity Antalya branch `storage_type` may be part of named collection. icebergCluster(iceberg_conf[, option=value [,..]]) ``` -Default value for `storage_type` is `s3`. +The default value for `storage_type` is `s3`. ### `object_storage_cluster` setting. -Only in Altinity Antalya branch alternative syntax for `icebergCluster` table function is `iceberg` function with non-empty setting `object_storage_cluster` with cluster name. +Only in the Altinity Antalya branch, an alternative syntax for `icebergCluster` table function is available. This allows the `iceberg` function to be used with the non-empty `object_storage_cluster` setting, specifying a cluster name. This enables distributed queries over Iceberg table across a ClickHouse cluster. ```sql icebergS3(url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' diff --git a/docs/en/sql-reference/table-functions/s3Cluster.md b/docs/en/sql-reference/table-functions/s3Cluster.md index 91626cac2fe8..e1b2c80f3efb 100644 --- a/docs/en/sql-reference/table-functions/s3Cluster.md +++ b/docs/en/sql-reference/table-functions/s3Cluster.md @@ -94,7 +94,7 @@ For details on optimizing the performance of the s3 function see [our detailed g ### `object_storage_cluster` setting. -Only in Altinity Antalya branch alternative syntax for `s3Cluster` table function is `s3` function with non-empty setting `object_storage_cluster` with cluster name. +Only in the Altinity Antalya branch alternative syntax for `s3Cluster` table function is available. This allows the `s3` function to be used with the non-empty `object_storage_cluster` setting, specifying a cluster name. This enables distributed queries over S3 Storage across a ClickHouse cluster. ```sql SELECT * FROM s3( From d904075ca6ea21310ecb9653df0bae23d9333cce Mon Sep 17 00:00:00 2001 From: Anton Ivashkin <56930273+ianton-ru@users.noreply.github.com> Date: Mon, 8 Dec 2025 17:41:59 +0100 Subject: [PATCH 3/4] Fix XML tags and function names in icebergCluster.md --- docs/en/sql-reference/table-functions/icebergCluster.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/en/sql-reference/table-functions/icebergCluster.md b/docs/en/sql-reference/table-functions/icebergCluster.md index 15d54609e5ac..178e8b8b4351 100644 --- a/docs/en/sql-reference/table-functions/icebergCluster.md +++ b/docs/en/sql-reference/table-functions/icebergCluster.md @@ -82,7 +82,7 @@ Only in the Altinity Antalya branch, `storage_type` can be part of a named colle http://test.s3.amazonaws.com/clickhouse-bucket/ - test + test test auto auto @@ -107,11 +107,11 @@ icebergS3(url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,f icebergAzure(connection_string|storage_account_url, container_name, blobpath, [,account_name], [,account_key] [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' -icebergHDSF(path_to_table, [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' +icebergHDFS(path_to_table, [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' icebergLocal(path_to_table, [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' -icebergS3(option=value [,..]]) SETTINGS object_storage_cluster='cluster_name' +icebergS3(option=value [,..]) SETTINGS object_storage_cluster='cluster_name' iceberg(storage_type='s3', url [, NOSIGN | access_key_id, secret_access_key, [session_token]] [,format] [,compression_method]) SETTINGS object_storage_cluster='cluster_name' From 439b7e6d2c6c6aeb14c1e781cdb35bda50fe91e8 Mon Sep 17 00:00:00 2001 From: Andrey Zvonov <32552679+zvonand@users.noreply.github.com> Date: Tue, 9 Dec 2025 13:15:22 +0100 Subject: [PATCH 4/4] Fix spelling of 'algorithm' in documentation --- docs/en/sql-reference/distribution-on-cluster.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/en/sql-reference/distribution-on-cluster.md b/docs/en/sql-reference/distribution-on-cluster.md index e563961b4286..3a9835e23856 100644 --- a/docs/en/sql-reference/distribution-on-cluster.md +++ b/docs/en/sql-reference/distribution-on-cluster.md @@ -1,6 +1,6 @@ # Task distribution in *Cluster family functions -## Task distribution algorythm +## Task distribution algorithm Table functions such as `s3Cluster`, `azureBlobStorageCluster`, `hdsfCluster`, `icebergCluster`, and table engines like `S3`, `Azure`, `HDFS`, `Iceberg` with the setting `object_storage_cluster` distribute tasks across all cluster nodes or a subset limited by the `object_storage_max_nodes` setting. This setting limits the number of nodes involved in processing a distributed query, randomly selecting nodes for each query.