From 688091f71a8e508c42f707e18198cfd926e270ba Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Fri, 20 May 2022 01:32:52 -0700 Subject: [PATCH 01/10] document arrays in sql --- docs/querying/sql-array-functions.md | 57 +++++++++++++++++++ docs/querying/sql-data-types.md | 42 +++++++++++++- .../sql-multivalue-string-functions.md | 5 +- 3 files changed, 99 insertions(+), 5 deletions(-) create mode 100644 docs/querying/sql-array-functions.md diff --git a/docs/querying/sql-array-functions.md b/docs/querying/sql-array-functions.md new file mode 100644 index 000000000000..2e2a2112d1e1 --- /dev/null +++ b/docs/querying/sql-array-functions.md @@ -0,0 +1,57 @@ +--- +id: sql-array-functions +title: "SQL ARRAY functions" +sidebar_label: "Array functions" +--- + + + + + + +> Apache Druid supports two query languages: Druid SQL and [native queries](querying.md). +> This document describes the SQL language. + +This page describes the operations you can perform on arrays using [Druid SQL](./sql.md). See [`ARRAY` data type documentation](./sql-data-types.md#arrays) for additional details. + +All 'array' references in the array function documentation can refer to multi-value string columns or `ARRAY` literals. These functions are largely +identical to the [array functions](#array-functions), but use `ARRAY` types and behavior. + +|Function|Notes| +|--------|-----| +|`ARRAY[expr1, expr2, ...]`|Constructs a SQL ARRAY literal from the expression arguments, using the type of the first argument as the output array type.| +|`ARRAY_LENGTH(arr)`|Returns length of array expression.| +|`ARRAY_OFFSET(arr, long)`|Returns the array element at the 0 based index supplied, or null for an out of range index.| +|`ARRAY_ORDINAL(arr, long)`|Returns the array element at the 1 based index supplied, or null for an out of range index.| +|`ARRAY_CONTAINS(arr, expr)`|Returns 1 if the array contains the element specified by `expr`, or contains all elements specified by `expr` if `expr` is an array, else 0.| +|`ARRAY_OVERLAP(arr1, arr2)`|Returns 1 if arr1 and arr2 have any elements in common, else 0.| +|`ARRAY_OFFSET_OF(arr, expr)`|Returns the 0 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| +|`ARRAY_ORDINAL_OF(arr, expr)`|Returns the 1 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| +|`ARRAY_PREPEND(expr, arr)`|Adds `expr` to `arr` at the beginning, the resulting array type determined by the type of the array.| +|`ARRAY_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of the first array.| +|`ARRAY_CONCAT(arr1, arr2)`|Concatenates 2 arrays, the resulting array type determined by the type of the first array.| +|`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index start(inclusive) to end(exclusive), or `null`, if start is less than 0, greater than length of arr or less than end.| +|`ARRAY_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| +|`STRING_TO_ARRAY(str1, str2)`|Splits `str1` into an array on the delimiter specified by `str2`.| diff --git a/docs/querying/sql-data-types.md b/docs/querying/sql-data-types.md index 38014631fa89..06f372d571b9 100644 --- a/docs/querying/sql-data-types.md +++ b/docs/querying/sql-data-types.md @@ -62,6 +62,7 @@ converted to zeroes). |BIGINT|LONG|`0`|Druid LONG columns (except `__time`) are reported as BIGINT| |TIMESTAMP|LONG|`0`, meaning 1970-01-01 00:00:00 UTC|Druid's `__time` column is reported as TIMESTAMP. Casts between string and timestamp types assume standard SQL formatting, e.g. `2000-01-02 03:04:05`, _not_ ISO8601 formatting. For handling other formats, use one of the [time functions](sql-scalar.md#date-and-time-functions).| |DATE|LONG|`0`, meaning 1970-01-01|Casting TIMESTAMP to DATE rounds down the timestamp to the nearest day. Casts between string and date types assume standard SQL formatting, e.g. `2000-01-02`. For handling other formats, use one of the [time functions](sql-scalar.md#date-and-time-functions).| +|ARRAY|ARRAY|Druid native array types work as SQL arrays, and multi-value strings can be converted to arrays. See the [`ARRAY` details](#arrays) for more details.| |OTHER|COMPLEX|none|May represent various Druid column types such as hyperUnique, approxHistogram, etc.| ## Multi-value strings @@ -70,7 +71,8 @@ Druid's native type system allows strings to potentially have multiple values. T [multi-value string dimensions](multi-value-dimensions.md) will be reported in SQL as `VARCHAR` typed, and can be syntactically used like any other VARCHAR. Regular string functions that refer to multi-value string dimensions will be applied to all values for each row individually. Multi-value string dimensions can also be treated as arrays via special -[multi-value string functions](sql-multivalue-string-functions.md), which can perform powerful array-aware operations. +[multi-value string functions](sql-multivalue-string-functions.md), which can perform powerful array-aware operations, but retain +their `VARCHAR` typing and behavior. Grouping by a multi-value expression will observe the native Druid multi-value aggregation behavior, which is similar to the `UNNEST` functionality available in some other SQL dialects. Refer to the documentation on @@ -80,8 +82,42 @@ the `UNNEST` functionality available in some other SQL dialects. Refer to the do > they are handled in Druid SQL and in native queries. For example, expressions involving multi-value dimensions may be > incorrectly optimized by the Druid SQL planner: `multi_val_dim = 'a' AND multi_val_dim = 'b'` will be optimized to > `false`, even though it is possible for a single row to have both "a" and "b" as values for `multi_val_dim`. The -> SQL behavior of multi-value dimensions will change in a future release to more closely align with their behavior -> in native queries. +> SQL behavior of multi-value dimensions may change in a future release to more closely align with their behavior +> in native queries, but the [multi-value string functions](./sql-multivalue-string-functions.md) should be able to provide +> nearly all possible native functionality. + +## Arrays +Multi-value dimensions may also be converted to standard SQL arrays, either by explicitly converting them with `MV_TO_ARRAY`, +or implicitly when used within the [array functions](./sql-array-functions.md). `ARRAY` types behave as standard SQL arrays, where +grouping on them will group on the entire array of values instead of the implicit `UNNEST` that occurs when grouping on +multi-value dimensions directly or when used with the multi-value functions. Arrays may also be constructed from multiple +columns using the array functions. + +## Multi-value strings behavior +The behavior of Druid [multi-value string dimensions](multi-value-dimensions.md) varies depending on the context of their usage. + +When used as `VARCHAR` functions, which are not "aware" that their inputs which claim to be `VARCHAR` might actually have multiple +values such as `CONCAT`, Druid will map the function across all values in the row. If the row is null or empty, the function will +recieve `NULL` as its input, otherwise it will be applied to every row value and continue its life as a multi-value VARCHAR. + +When used with the explicit [multi-value string functions](./sql-multivalue-string-functions.md), the column is acknowledged to be multi-valued, +and during processing the values are operated on as if they were `ARRAY` typed, so any operations which produce null and empty rows are +distinguished as separate values (unlike implicit mapping behavior), but retain their `VARCHAR` type after the computation is complete. +Note that Druid multi-value columns do _not_ distinguish between empty and null rows, so an empty row will never appear natively as input +to a multi-valued function, but any multi-value function which manipulates the array form of the value may produce an empty array, which +will be handled separately while processing. + +> Do not mix the usage of multi-value functions and normal scalar functions within the same expression, as the planner will be unable +> to determine how to properly process the value given its ambiguous usage. A multi-value string must be treated consistently within +> an expression. + +Finally, when converted to `ARRAY` or used with [array functions](./sql-array-functions.md), they behave as standard SQL arrays and can no longer +be manipulated with non-array functions. + +When `VARCHAR` typed, multi-value results produced by non-aggregating selection of values will be serialized into a JSON string of the array. +`ARRAY` typed results will be serialized into stringified JSON arrays if the context parameter `sqlStringifyArrays` is set, otherwise they +will remain in their array format. + ## NULL values diff --git a/docs/querying/sql-multivalue-string-functions.md b/docs/querying/sql-multivalue-string-functions.md index 85bab5e6793c..2372700c4a89 100644 --- a/docs/querying/sql-multivalue-string-functions.md +++ b/docs/querying/sql-multivalue-string-functions.md @@ -36,10 +36,10 @@ sidebar_label: "Multi-value string functions" Druid supports string dimensions containing multiple values. This page describes the operations you can perform on multi-value string dimensions using [Druid SQL](./sql.md). -See [Multi-value dimensions](multi-value-dimensions.md) for more information. +See [SQL multi-value string](./sql-data-types.md#multi-value-strings) and native [Multi-value dimensions](multi-value-dimensions.md) for more information. All "array" references in the multi-value string function documentation can refer to multi-value string columns or -`ARRAY` literals. +`ARRAY` types. Multi-value strings can also be converted to `ARRAY` types using `MV_TO_ARRAY` (See [Multi-value string functions](sql-multivalue-string-functions.md) for more details.) |Function|Notes| |--------|-----| @@ -59,3 +59,4 @@ All "array" references in the multi-value string function documentation can refe |`MV_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index start(inclusive) to end(exclusive), or `null`, if start is less than 0, greater than length of arr or less than end.| |`MV_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| |`STRING_TO_MV(str1, str2)`|Splits `str1` into an array on the delimiter specified by `str2`.| +|`MV_TO_ARRAY(str)`|Converts a multi-value string from being processed as a `VARCHAR` to an `ARRAY` instead.| From fd3894116bfa6182710b897b2723ecbca0bb0f4b Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Wed, 8 Feb 2023 18:23:50 -0800 Subject: [PATCH 02/10] adjustments --- docs/querying/sql-array-functions.md | 12 +++--- docs/querying/sql-data-types.md | 43 ++++++++++--------- .../sql-multivalue-string-functions.md | 9 ++-- 3 files changed, 34 insertions(+), 30 deletions(-) diff --git a/docs/querying/sql-array-functions.md b/docs/querying/sql-array-functions.md index 2e2a2112d1e1..7cfed488e32a 100644 --- a/docs/querying/sql-array-functions.md +++ b/docs/querying/sql-array-functions.md @@ -41,17 +41,17 @@ identical to the [array functions](#array-functions), but use `ARRAY` types and |Function|Notes| |--------|-----| -|`ARRAY[expr1, expr2, ...]`|Constructs a SQL ARRAY literal from the expression arguments, using the type of the first argument as the output array type.| +|`ARRAY[expr1, expr2, ...]`|Constructs a SQL `ARRAY` literal from the expression arguments, using the type of the first argument as the output array type.| |`ARRAY_LENGTH(arr)`|Returns length of array expression.| |`ARRAY_OFFSET(arr, long)`|Returns the array element at the 0 based index supplied, or null for an out of range index.| |`ARRAY_ORDINAL(arr, long)`|Returns the array element at the 1 based index supplied, or null for an out of range index.| |`ARRAY_CONTAINS(arr, expr)`|Returns 1 if the array contains the element specified by `expr`, or contains all elements specified by `expr` if `expr` is an array, else 0.| -|`ARRAY_OVERLAP(arr1, arr2)`|Returns 1 if arr1 and arr2 have any elements in common, else 0.| +|`ARRAY_OVERLAP(arr1, arr2)`|Returns 1 if `arr1` and `arr2` have any elements in common, else 0.| |`ARRAY_OFFSET_OF(arr, expr)`|Returns the 0 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| |`ARRAY_ORDINAL_OF(arr, expr)`|Returns the 1 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| -|`ARRAY_PREPEND(expr, arr)`|Adds `expr` to `arr` at the beginning, the resulting array type determined by the type of the array.| -|`ARRAY_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of the first array.| -|`ARRAY_CONCAT(arr1, arr2)`|Concatenates 2 arrays, the resulting array type determined by the type of the first array.| -|`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index start(inclusive) to end(exclusive), or `null`, if start is less than 0, greater than length of arr or less than end.| +|`ARRAY_PREPEND(expr, arr)`|Prepends `expr` to `arr` at the beginning, the resulting array type determined by the type of `arr`.| +|`ARRAY_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of `arr1`.| +|`ARRAY_CONCAT(arr1, arr2)`|Concatenates two arrays, the resulting array type determined by the type of `arr1`.| +|`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index `start` (inclusive) to `end` (exclusive). Returns `null`, if `start` is less than 0, greater than length of `arr`, or less than `end`.| |`ARRAY_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| |`STRING_TO_ARRAY(str1, str2)`|Splits `str1` into an array on the delimiter specified by `str2`.| diff --git a/docs/querying/sql-data-types.md b/docs/querying/sql-data-types.md index c8d312fe4ada..1a04d6b11184 100644 --- a/docs/querying/sql-data-types.md +++ b/docs/querying/sql-data-types.md @@ -33,7 +33,7 @@ Columns in Druid are associated with a specific data type. This topic describes Druid natively supports five basic column types: "long" (64 bit signed int), "float" (32 bit float), "double" (64 bit float) "string" (UTF-8 encoded strings and string arrays), and "complex" (catch-all for more exotic data types like -json, hyperUnique, and approxHistogram columns). +json, hyperUnique, and approxHistogram columns), and "array" composed of any of these types. Timestamps (including the `__time` column) are treated by Druid as longs, with the value being the number of milliseconds since 1970-01-01 00:00:00 UTC, not counting leap seconds. Therefore, timestamps in Druid do not carry any @@ -65,7 +65,7 @@ The following table describes how Druid maps SQL types onto native types when ru |BIGINT|LONG|`0`|Druid LONG columns (except `__time`) are reported as BIGINT| |TIMESTAMP|LONG|`0`, meaning 1970-01-01 00:00:00 UTC|Druid's `__time` column is reported as TIMESTAMP. Casts between string and timestamp types assume standard SQL formatting, e.g. `2000-01-02 03:04:05`, _not_ ISO8601 formatting. For handling other formats, use one of the [time functions](sql-scalar.md#date-and-time-functions).| |DATE|LONG|`0`, meaning 1970-01-01|Casting TIMESTAMP to DATE rounds down the timestamp to the nearest day. Casts between string and date types assume standard SQL formatting, e.g. `2000-01-02`. For handling other formats, use one of the [time functions](sql-scalar.md#date-and-time-functions).| -|ARRAY|ARRAY|Druid native array types work as SQL arrays, and multi-value strings can be converted to arrays. See the [`ARRAY` details](#arrays) for more details.| +|ARRAY|ARRAY|Druid native array types work as SQL arrays, and multi-value strings can be converted to arrays. See the [`ARRAY` details](#arrays).| |OTHER|COMPLEX|none|May represent various Druid column types such as hyperUnique, approxHistogram, etc.| * Default value applies if `druid.generic.useDefaultValueForNull = true` (the default mode). Otherwise, the default value is `NULL` for all types. @@ -74,7 +74,7 @@ The following table describes how Druid maps SQL types onto native types when ru Druid's native type system allows strings to potentially have multiple values. These [multi-value string dimensions](multi-value-dimensions.md) are reported in SQL as `VARCHAR` typed, and can be -syntactically used like any other VARCHAR. Regular string functions that refer to multi-value string dimensions are +syntactically used like any other `VARCHAR`. Regular string functions that refer to multi-value string dimensions are applied to all values for each row individually. Multi-value string dimensions can also be treated as arrays via special [multi-value string functions](sql-multivalue-string-functions.md), which can perform powerful array-aware operations, but retain their `VARCHAR` typing and behavior. @@ -92,36 +92,39 @@ the `UNNEST` functionality available in some other SQL dialects. Refer to the do > nearly all possible native functionality. ## Arrays -Multi-value dimensions may also be converted to standard SQL arrays, either by explicitly converting them with `MV_TO_ARRAY`, -or implicitly when used within the [array functions](./sql-array-functions.md). `ARRAY` types behave as standard SQL arrays, where -grouping on them will group on the entire array of values instead of the implicit `UNNEST` that occurs when grouping on -multi-value dimensions directly or when used with the multi-value functions. Arrays may also be constructed from multiple -columns using the array functions. +Druid also has support for `ARRAY` types constructed at query time, though currently lacks the ability to store them in +segments. `ARRAY` types behave as standard SQL arrays, where results are grouped by matching entire arrays. This is in +contrast to the implicit `UNNEST` that occurs when grouping on multi-value dimensions directly or when used with the +multi-value functions. You can convert multi-value dimensions to standard SQL arrays either by explicitly converting +them with `MV_TO_ARRAY`, or implicitly when used within the [array functions](./sql-array-functions.md). Arrays may +also be constructed from multiple columns using the array functions. ## Multi-value strings behavior The behavior of Druid [multi-value string dimensions](multi-value-dimensions.md) varies depending on the context of their usage. -When used as `VARCHAR` functions, which are not "aware" that their inputs which claim to be `VARCHAR` might actually have multiple -values such as `CONCAT`, Druid will map the function across all values in the row. If the row is null or empty, the function will -recieve `NULL` as its input, otherwise it will be applied to every row value and continue its life as a multi-value VARCHAR. +When used with standard `VARCHAR` functions which expect a single input value per row, such as `CONCAT`, Druid will map +the function across all values in the row. If the row is null or empty, the function will recieve `NULL` as its input, +otherwise it will be applied to every row value and continue its life as a multi-value `VARCHAR`. -When used with the explicit [multi-value string functions](./sql-multivalue-string-functions.md), the column is acknowledged to be multi-valued, -and during processing the values are operated on as if they were `ARRAY` typed, so any operations which produce null and empty rows are -distinguished as separate values (unlike implicit mapping behavior), but retain their `VARCHAR` type after the computation is complete. -Note that Druid multi-value columns do _not_ distinguish between empty and null rows, so an empty row will never appear natively as input -to a multi-valued function, but any multi-value function which manipulates the array form of the value may produce an empty array, which -will be handled separately while processing. +When used with the explicit [multi-value string functions](./sql-multivalue-string-functions.md), Druid processes the +row values as if they were `ARRAY` typed, so any operations which produce null and empty rows are distinguished as +separate values (unlike implicit mapping behavior). These multi-value string functions, typically denoted with an `MV_` +prefix, retain their `VARCHAR` type after the computation is complete. Note that Druid multi-value columns do _not_ +distinguish between empty and null rows, so an empty row will never appear natively as input to a multi-valued function, +but any multi-value function which manipulates the array form of the value may produce an empty array, which will be +handled separately while processing. > Do not mix the usage of multi-value functions and normal scalar functions within the same expression, as the planner will be unable > to determine how to properly process the value given its ambiguous usage. A multi-value string must be treated consistently within > an expression. -Finally, when converted to `ARRAY` or used with [array functions](./sql-array-functions.md), they behave as standard SQL arrays and can no longer +When converted to `ARRAY` or used with [array functions](./sql-array-functions.md), multi-value strings behave as standard SQL arrays and can no longer be manipulated with non-array functions. -When `VARCHAR` typed, multi-value results produced by non-aggregating selection of values will be serialized into a JSON string of the array. +Druid serializes multi-value `VARCHAR` results as a JSON string of the array, if the value was not grouped on. If the +value was grouped, due to the implicit `UNNEST` behavior, all results will always be standard single value `VARCHAR`. `ARRAY` typed results will be serialized into stringified JSON arrays if the context parameter `sqlStringifyArrays` is set, otherwise they -will remain in their array format. +remain in their array format. ## NULL values diff --git a/docs/querying/sql-multivalue-string-functions.md b/docs/querying/sql-multivalue-string-functions.md index 2372700c4a89..a3544992520b 100644 --- a/docs/querying/sql-multivalue-string-functions.md +++ b/docs/querying/sql-multivalue-string-functions.md @@ -36,14 +36,15 @@ sidebar_label: "Multi-value string functions" Druid supports string dimensions containing multiple values. This page describes the operations you can perform on multi-value string dimensions using [Druid SQL](./sql.md). -See [SQL multi-value string](./sql-data-types.md#multi-value-strings) and native [Multi-value dimensions](multi-value-dimensions.md) for more information. +See [SQL multi-value strings](./sql-data-types.md#multi-value-strings) and native [Multi-value dimensions](multi-value-dimensions.md) for more information. All "array" references in the multi-value string function documentation can refer to multi-value string columns or -`ARRAY` types. Multi-value strings can also be converted to `ARRAY` types using `MV_TO_ARRAY` (See [Multi-value string functions](sql-multivalue-string-functions.md) for more details.) +`ARRAY` types. Multi-value strings can also be converted to `ARRAY` types using `MV_TO_ARRAY`, see +[Multi-value string functions](sql-multivalue-string-functions.md), [`ARRAY` data type documentation](./sql-data-types.md#arrays), +and [array functions](./sql-array-functions.md) for additional details. |Function|Notes| |--------|-----| -|`ARRAY[expr1, expr2, ...]`|Constructs a SQL ARRAY literal from the expression arguments, using the type of the first argument as the output array type.| |`MV_FILTER_ONLY(expr, arr)`|Filters multi-value `expr` to include only values contained in array `arr`.| |`MV_FILTER_NONE(expr, arr)`|Filters multi-value `expr` to include no values contained in array `arr`.| |`MV_LENGTH(arr)`|Returns length of array expression.| @@ -59,4 +60,4 @@ All "array" references in the multi-value string function documentation can refe |`MV_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index start(inclusive) to end(exclusive), or `null`, if start is less than 0, greater than length of arr or less than end.| |`MV_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| |`STRING_TO_MV(str1, str2)`|Splits `str1` into an array on the delimiter specified by `str2`.| -|`MV_TO_ARRAY(str)`|Converts a multi-value string from being processed as a `VARCHAR` to an `ARRAY` instead.| +|`MV_TO_ARRAY(str)`|Converts a multi-value string from a `VARCHAR` to an `ARRAY` instead.| From f56d3f3a86386bd8d78d5c21f37747c31e5435ec Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Thu, 9 Feb 2023 14:12:40 -0800 Subject: [PATCH 03/10] Update docs/querying/sql-array-functions.md Co-authored-by: Katya Macedo <38017980+ektravel@users.noreply.github.com> --- docs/querying/sql-array-functions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/querying/sql-array-functions.md b/docs/querying/sql-array-functions.md index 7cfed488e32a..259f69787705 100644 --- a/docs/querying/sql-array-functions.md +++ b/docs/querying/sql-array-functions.md @@ -49,7 +49,7 @@ identical to the [array functions](#array-functions), but use `ARRAY` types and |`ARRAY_OVERLAP(arr1, arr2)`|Returns 1 if `arr1` and `arr2` have any elements in common, else 0.| |`ARRAY_OFFSET_OF(arr, expr)`|Returns the 0 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| |`ARRAY_ORDINAL_OF(arr, expr)`|Returns the 1 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| -|`ARRAY_PREPEND(expr, arr)`|Prepends `expr` to `arr` at the beginning, the resulting array type determined by the type of `arr`.| +|`ARRAY_PREPEND(expr, arr)`|Prepends `expr` to `arr`. The type of `arr` determines the resulting array type.| |`ARRAY_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of `arr1`.| |`ARRAY_CONCAT(arr1, arr2)`|Concatenates two arrays, the resulting array type determined by the type of `arr1`.| |`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index `start` (inclusive) to `end` (exclusive). Returns `null`, if `start` is less than 0, greater than length of `arr`, or less than `end`.| From 0eaee08f17a919e69e57d3d8b25f29df92ad6a07 Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Thu, 9 Feb 2023 14:13:10 -0800 Subject: [PATCH 04/10] Update docs/querying/sql-data-types.md Co-authored-by: Katya Macedo <38017980+ektravel@users.noreply.github.com> --- docs/querying/sql-data-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/querying/sql-data-types.md b/docs/querying/sql-data-types.md index 1a04d6b11184..c452daf71057 100644 --- a/docs/querying/sql-data-types.md +++ b/docs/querying/sql-data-types.md @@ -102,7 +102,7 @@ also be constructed from multiple columns using the array functions. ## Multi-value strings behavior The behavior of Druid [multi-value string dimensions](multi-value-dimensions.md) varies depending on the context of their usage. -When used with standard `VARCHAR` functions which expect a single input value per row, such as `CONCAT`, Druid will map +When used with standard `VARCHAR` functions, which expect a single input value per row, Druid maps the function across all values in the row. the function across all values in the row. If the row is null or empty, the function will recieve `NULL` as its input, otherwise it will be applied to every row value and continue its life as a multi-value `VARCHAR`. From a09318fde5066314a0113c046699783bbca1f97d Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Thu, 9 Feb 2023 14:13:23 -0800 Subject: [PATCH 05/10] Update docs/querying/sql-data-types.md Co-authored-by: Katya Macedo <38017980+ektravel@users.noreply.github.com> --- docs/querying/sql-data-types.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/querying/sql-data-types.md b/docs/querying/sql-data-types.md index c452daf71057..75330cc531ce 100644 --- a/docs/querying/sql-data-types.md +++ b/docs/querying/sql-data-types.md @@ -92,7 +92,7 @@ the `UNNEST` functionality available in some other SQL dialects. Refer to the do > nearly all possible native functionality. ## Arrays -Druid also has support for `ARRAY` types constructed at query time, though currently lacks the ability to store them in +Druid supports `ARRAY` types constructed at query time, though it currently lacks the ability to store them in segments. `ARRAY` types behave as standard SQL arrays, where results are grouped by matching entire arrays. This is in contrast to the implicit `UNNEST` that occurs when grouping on multi-value dimensions directly or when used with the multi-value functions. You can convert multi-value dimensions to standard SQL arrays either by explicitly converting From ea41ea3b218d83d3b46e77444cbbb724e335fe3e Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Thu, 9 Feb 2023 14:13:32 -0800 Subject: [PATCH 06/10] Update docs/querying/sql-array-functions.md Co-authored-by: Katya Macedo <38017980+ektravel@users.noreply.github.com> --- docs/querying/sql-array-functions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/querying/sql-array-functions.md b/docs/querying/sql-array-functions.md index 259f69787705..7da95c8f3a64 100644 --- a/docs/querying/sql-array-functions.md +++ b/docs/querying/sql-array-functions.md @@ -51,7 +51,7 @@ identical to the [array functions](#array-functions), but use `ARRAY` types and |`ARRAY_ORDINAL_OF(arr, expr)`|Returns the 1 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| |`ARRAY_PREPEND(expr, arr)`|Prepends `expr` to `arr`. The type of `arr` determines the resulting array type.| |`ARRAY_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of `arr1`.| -|`ARRAY_CONCAT(arr1, arr2)`|Concatenates two arrays, the resulting array type determined by the type of `arr1`.| +|`ARRAY_CONCAT(arr1, arr2)`|Concatenates two arrays. The type of `arr1` determines the resulting array type.| |`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index `start` (inclusive) to `end` (exclusive). Returns `null`, if `start` is less than 0, greater than length of `arr`, or less than `end`.| |`ARRAY_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| |`STRING_TO_ARRAY(str1, str2)`|Splits `str1` into an array on the delimiter specified by `str2`.| From 953395fbfa9e5b853ed9fe821de41c15fe38466e Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Thu, 9 Feb 2023 14:13:43 -0800 Subject: [PATCH 07/10] Update docs/querying/sql-array-functions.md Co-authored-by: Katya Macedo <38017980+ektravel@users.noreply.github.com> --- docs/querying/sql-array-functions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/querying/sql-array-functions.md b/docs/querying/sql-array-functions.md index 7da95c8f3a64..ce01c61980e2 100644 --- a/docs/querying/sql-array-functions.md +++ b/docs/querying/sql-array-functions.md @@ -50,7 +50,7 @@ identical to the [array functions](#array-functions), but use `ARRAY` types and |`ARRAY_OFFSET_OF(arr, expr)`|Returns the 0 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| |`ARRAY_ORDINAL_OF(arr, expr)`|Returns the 1 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| |`ARRAY_PREPEND(expr, arr)`|Prepends `expr` to `arr`. The type of `arr` determines the resulting array type.| -|`ARRAY_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of `arr1`.| +|`ARRAY_APPEND(arr, expr)`|Appends `expr` to `arr`. The type of `arr` determines the resulting array type.| |`ARRAY_CONCAT(arr1, arr2)`|Concatenates two arrays. The type of `arr1` determines the resulting array type.| |`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index `start` (inclusive) to `end` (exclusive). Returns `null`, if `start` is less than 0, greater than length of `arr`, or less than `end`.| |`ARRAY_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| From d7faae4c020d3f2e623878461301f43e3765a8b9 Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Fri, 10 Feb 2023 00:50:48 -0800 Subject: [PATCH 08/10] Update sql-array-functions.md --- docs/querying/sql-array-functions.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/querying/sql-array-functions.md b/docs/querying/sql-array-functions.md index ce01c61980e2..f5d66be4557f 100644 --- a/docs/querying/sql-array-functions.md +++ b/docs/querying/sql-array-functions.md @@ -37,7 +37,7 @@ sidebar_label: "Array functions" This page describes the operations you can perform on arrays using [Druid SQL](./sql.md). See [`ARRAY` data type documentation](./sql-data-types.md#arrays) for additional details. All 'array' references in the array function documentation can refer to multi-value string columns or `ARRAY` literals. These functions are largely -identical to the [array functions](#array-functions), but use `ARRAY` types and behavior. +identical to the [multi-value string functions](./sql-multivalue-string-functions.md), but use `ARRAY` types and behavior. |Function|Notes| |--------|-----| From d8b67f87635d1d1ac5e1a1a3f26bc88e51c2d0b0 Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Mon, 17 Apr 2023 15:16:27 -0700 Subject: [PATCH 09/10] fix stuff --- docs/querying/sql-array-functions.md | 22 +++++------ docs/querying/sql-data-types.md | 38 ++++++++++--------- .../sql-multivalue-string-functions.md | 30 +++++++-------- web-console/script/create-sql-docs.js | 3 +- 4 files changed, 49 insertions(+), 44 deletions(-) diff --git a/docs/querying/sql-array-functions.md b/docs/querying/sql-array-functions.md index 7cfed488e32a..98ba7d61d93b 100644 --- a/docs/querying/sql-array-functions.md +++ b/docs/querying/sql-array-functions.md @@ -36,22 +36,22 @@ sidebar_label: "Array functions" This page describes the operations you can perform on arrays using [Druid SQL](./sql.md). See [`ARRAY` data type documentation](./sql-data-types.md#arrays) for additional details. -All 'array' references in the array function documentation can refer to multi-value string columns or `ARRAY` literals. These functions are largely -identical to the [array functions](#array-functions), but use `ARRAY` types and behavior. +All array references in the array function documentation can refer to multi-value string columns or `ARRAY` literals. These functions are largely +identical to the [Multi-value string functions](sql-multivalue-string-functions.md), but use `ARRAY` types and behavior. -|Function|Notes| +|Function|Description| |--------|-----| |`ARRAY[expr1, expr2, ...]`|Constructs a SQL `ARRAY` literal from the expression arguments, using the type of the first argument as the output array type.| -|`ARRAY_LENGTH(arr)`|Returns length of array expression.| -|`ARRAY_OFFSET(arr, long)`|Returns the array element at the 0 based index supplied, or null for an out of range index.| -|`ARRAY_ORDINAL(arr, long)`|Returns the array element at the 1 based index supplied, or null for an out of range index.| -|`ARRAY_CONTAINS(arr, expr)`|Returns 1 if the array contains the element specified by `expr`, or contains all elements specified by `expr` if `expr` is an array, else 0.| +|`ARRAY_LENGTH(arr)`|Returns length of the array expression.| +|`ARRAY_OFFSET(arr, long)`|Returns the array element at the 0-based index supplied, or null for an out of range index.| +|`ARRAY_ORDINAL(arr, long)`|Returns the array element at the 1-based index supplied, or null for an out of range index.| +|`ARRAY_CONTAINS(arr, expr)`|If `expr` is a scalar type, returns 1 if `arr` contains `expr`. If `expr` is an array, returns 1 if `arr` contains all elements of `expr`. Otherwise returns 0.| |`ARRAY_OVERLAP(arr1, arr2)`|Returns 1 if `arr1` and `arr2` have any elements in common, else 0.| -|`ARRAY_OFFSET_OF(arr, expr)`|Returns the 0 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| -|`ARRAY_ORDINAL_OF(arr, expr)`|Returns the 1 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| +|`ARRAY_OFFSET_OF(arr, expr)`|Returns the 0-based index of the first occurrence of `expr` in the array. If no matching elements exist in the array, returns `-1` or `null` if `druid.generic.useDefaultValueForNull=false`.| +|`ARRAY_ORDINAL_OF(arr, expr)`|Returns the 1-based index of the first occurrence of `expr` in the array. If no matching elements exist in the array, returns `-1` or `null` if `druid.generic.useDefaultValueForNull=false`.| |`ARRAY_PREPEND(expr, arr)`|Prepends `expr` to `arr` at the beginning, the resulting array type determined by the type of `arr`.| |`ARRAY_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of `arr1`.| -|`ARRAY_CONCAT(arr1, arr2)`|Concatenates two arrays, the resulting array type determined by the type of `arr1`.| -|`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index `start` (inclusive) to `end` (exclusive). Returns `null`, if `start` is less than 0, greater than length of `arr`, or less than `end`.| +|`ARRAY_CONCAT(arr1, arr2)`|Concatenates `arr2` to `arr1`. The resulting array type is determined by the type of `arr1`.| +|`ARRAY_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0-based index `start` (inclusive) to `end` (exclusive). Returns `null`, if `start` is less than 0, greater than length of `arr`, or greater than `end`.| |`ARRAY_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| |`STRING_TO_ARRAY(str1, str2)`|Splits `str1` into an array on the delimiter specified by `str2`.| diff --git a/docs/querying/sql-data-types.md b/docs/querying/sql-data-types.md index dfa34a35bbe9..3450c7063cd1 100644 --- a/docs/querying/sql-data-types.md +++ b/docs/querying/sql-data-types.md @@ -31,9 +31,13 @@ Columns in Druid are associated with a specific data type. This topic describes ## Standard types -Druid natively supports five basic column types: "long" (64 bit signed int), "float" (32 bit float), "double" (64 bit -float) "string" (UTF-8 encoded strings and string arrays), and "complex" (catch-all for more exotic data types like -json, hyperUnique, and approxHistogram columns), and "array" composed of any of these types. +Druid natively supports the following basic column types: +* LONG: (64 bit signed int) +* FLOAT (32 bit float) +* DOUBLE: (64 bit float) +* STRING: (UTF-8 encoded strings and string arrays) +* COMPLEX: non-standard data types, such as nested JSON, hyperUnique and approxHistogram, and DataSketches +* ARRAY: arrays composed of any of these types Timestamps (including the `__time` column) are treated by Druid as longs, with the value being the number of milliseconds since 1970-01-01 00:00:00 UTC, not counting leap seconds. Therefore, timestamps in Druid do not carry any @@ -65,7 +69,7 @@ The following table describes how Druid maps SQL types onto native types when ru |BIGINT|LONG|`0`|Druid LONG columns (except `__time`) are reported as BIGINT| |TIMESTAMP|LONG|`0`, meaning 1970-01-01 00:00:00 UTC|Druid's `__time` column is reported as TIMESTAMP. Casts between string and timestamp types assume standard SQL formatting, e.g. `2000-01-02 03:04:05`, _not_ ISO8601 formatting. For handling other formats, use one of the [time functions](sql-scalar.md#date-and-time-functions).| |DATE|LONG|`0`, meaning 1970-01-01|Casting TIMESTAMP to DATE rounds down the timestamp to the nearest day. Casts between string and date types assume standard SQL formatting, e.g. `2000-01-02`. For handling other formats, use one of the [time functions](sql-scalar.md#date-and-time-functions).| -|ARRAY|ARRAY|Druid native array types work as SQL arrays, and multi-value strings can be converted to arrays. See the [`ARRAY` details](#arrays).| +|ARRAY|ARRAY|`NULL`|Druid native array types work as SQL arrays, and multi-value strings can be converted to arrays. See the [`ARRAY` details](#arrays).| |OTHER|COMPLEX|none|May represent various Druid column types such as hyperUnique, approxHistogram, etc.| * Default value applies if `druid.generic.useDefaultValueForNull = true` (the default mode). Otherwise, the default value is `NULL` for all types. @@ -95,24 +99,24 @@ for additional details. Druid also has support for `ARRAY` types constructed at query time, though currently lacks the ability to store them in segments. `ARRAY` types behave as standard SQL arrays, where results are grouped by matching entire arrays. This is in contrast to the implicit `UNNEST` that occurs when grouping on multi-value dimensions directly or when used with the -multi-value functions. You can convert multi-value dimensions to standard SQL arrays either by explicitly converting -them with `MV_TO_ARRAY`, or implicitly when used within the [array functions](./sql-array-functions.md). Arrays may +multi-value functions. You can convert multi-value dimensions to standard SQL arrays either by explicitly by converting +them with `MV_TO_ARRAY` or implicitly when used within the [array functions](./sql-array-functions.md). Arrays may also be constructed from multiple columns using the array functions. ## Multi-value strings behavior -The behavior of Druid [multi-value string dimensions](multi-value-dimensions.md) varies depending on the context of their usage. +The behavior of Druid [multi-value string dimensions](multi-value-dimensions.md) varies depending on the context of +their usage. When used with standard `VARCHAR` functions which expect a single input value per row, such as `CONCAT`, Druid will map -the function across all values in the row. If the row is null or empty, the function will recieve `NULL` as its input, -otherwise it will be applied to every row value and continue its life as a multi-value `VARCHAR`. +the function across all values in the row. If the row is null or empty, the function recieves `NULL` as its input. When used with the explicit [multi-value string functions](./sql-multivalue-string-functions.md), Druid processes the -row values as if they were `ARRAY` typed, so any operations which produce null and empty rows are distinguished as +row values as if they were `ARRAY` typed. Any operations which produce null and empty rows are distinguished as separate values (unlike implicit mapping behavior). These multi-value string functions, typically denoted with an `MV_` prefix, retain their `VARCHAR` type after the computation is complete. Note that Druid multi-value columns do _not_ -distinguish between empty and null rows, so an empty row will never appear natively as input to a multi-valued function, -but any multi-value function which manipulates the array form of the value may produce an empty array, which will be -handled separately while processing. +distinguish between empty and null rows. An empty row will never appear natively as input to a multi-valued function, +but any multi-value function which manipulates the array form of the value may produce an empty array, which is handled +separately while processing. > Do not mix the usage of multi-value functions and normal scalar functions within the same expression, as the planner will be unable > to determine how to properly process the value given its ambiguous usage. A multi-value string must be treated consistently within @@ -121,10 +125,10 @@ handled separately while processing. When converted to `ARRAY` or used with [array functions](./sql-array-functions.md), multi-value strings behave as standard SQL arrays and can no longer be manipulated with non-array functions. -Druid serializes multi-value `VARCHAR` results as a JSON string of the array, if the value was not grouped on. If the -value was grouped, due to the implicit `UNNEST` behavior, all results will always be standard single value `VARCHAR`. -`ARRAY` typed results will be serialized into stringified JSON arrays if the context parameter `sqlStringifyArrays` is set, otherwise they -remain in their array format. +Druid serializes multi-value `VARCHAR` results as a JSON string of the array, if grouping was not applied on the value. +If the value was grouped, due to the implicit `UNNEST` behavior, all results will always be standard single value +`VARCHAR`. `ARRAY` typed results will be serialized into stringified JSON arrays if the context parameter +`sqlStringifyArrays` is set, otherwise they remain in their array format. ## NULL values diff --git a/docs/querying/sql-multivalue-string-functions.md b/docs/querying/sql-multivalue-string-functions.md index a3544992520b..bee77edbe19f 100644 --- a/docs/querying/sql-multivalue-string-functions.md +++ b/docs/querying/sql-multivalue-string-functions.md @@ -38,26 +38,26 @@ Druid supports string dimensions containing multiple values. This page describes the operations you can perform on multi-value string dimensions using [Druid SQL](./sql.md). See [SQL multi-value strings](./sql-data-types.md#multi-value-strings) and native [Multi-value dimensions](multi-value-dimensions.md) for more information. -All "array" references in the multi-value string function documentation can refer to multi-value string columns or -`ARRAY` types. Multi-value strings can also be converted to `ARRAY` types using `MV_TO_ARRAY`, see -[Multi-value string functions](sql-multivalue-string-functions.md), [`ARRAY` data type documentation](./sql-data-types.md#arrays), -and [array functions](./sql-array-functions.md) for additional details. +All array references in the multi-value string function documentation can refer to multi-value string columns or +`ARRAY` types. These functions are largely identical to the [array functions](./sql-array-functions.md), but use +`VARCHAR` types and behavior.Multi-value strings can also be converted to `ARRAY` types using `MV_TO_ARRAY`. For +additional details about `ARRAY` types, see [`ARRAY` data type documentation](./sql-data-types.md#arrays). -|Function|Notes| +|Function|Description| |--------|-----| |`MV_FILTER_ONLY(expr, arr)`|Filters multi-value `expr` to include only values contained in array `arr`.| |`MV_FILTER_NONE(expr, arr)`|Filters multi-value `expr` to include no values contained in array `arr`.| -|`MV_LENGTH(arr)`|Returns length of array expression.| -|`MV_OFFSET(arr, long)`|Returns the array element at the 0 based index supplied, or null for an out of range index.| -|`MV_ORDINAL(arr, long)`|Returns the array element at the 1 based index supplied, or null for an out of range index.| -|`MV_CONTAINS(arr, expr)`|Returns 1 if the array contains the element specified by `expr`, or contains all elements specified by `expr` if `expr` is an array, else 0.| -|`MV_OVERLAP(arr1, arr2)`|Returns 1 if arr1 and arr2 have any elements in common, else 0.| -|`MV_OFFSET_OF(arr, expr)`|Returns the 0 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| -|`MV_ORDINAL_OF(arr, expr)`|Returns the 1 based index of the first occurrence of `expr` in the array, or `-1` or `null` if `druid.generic.useDefaultValueForNull=false` if no matching elements exist in the array.| +|`MV_LENGTH(arr)`|Returns length of the array expression.| +|`MV_OFFSET(arr, long)`|Returns the array element at the 0-based index supplied, or null for an out of range index.| +|`MV_ORDINAL(arr, long)`|Returns the array element at the 1-based index supplied, or null for an out of range index.| +|`MV_CONTAINS(arr, expr)`|If `expr` is a scalar type, returns 1 if `arr` contains `expr`. If `expr` is an array, returns 1 if `arr` contains all elements of `expr`. Otherwise returns 0.| +|`MV_OVERLAP(arr1, arr2)`|Returns 1 if `arr1` and `arr2` have any elements in common, else 0.| +|`MV_OFFSET_OF(arr, expr)`|Returns the 0-based index of the first occurrence of `expr` in the array. If no matching elements exist in the array, returns `-1` or `null` if `druid.generic.useDefaultValueForNull=false`.| +|`MV_ORDINAL_OF(arr, expr)`|Returns the 1-based index of the first occurrence of `expr` in the array. If no matching elements exist in the array, returns `-1` or `null` if `druid.generic.useDefaultValueForNull=false`.| |`MV_PREPEND(expr, arr)`|Adds `expr` to `arr` at the beginning, the resulting array type determined by the type of the array.| |`MV_APPEND(arr1, expr)`|Appends `expr` to `arr`, the resulting array type determined by the type of the first array.| -|`MV_CONCAT(arr1, arr2)`|Concatenates 2 arrays, the resulting array type determined by the type of the first array.| -|`MV_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0 based index start(inclusive) to end(exclusive), or `null`, if start is less than 0, greater than length of arr or less than end.| +|`MV_CONCAT(arr1, arr2)`|Concatenates `arr2` to `arr1`. The resulting array type is determined by the type of `arr1`.| +|`MV_SLICE(arr, start, end)`|Returns the subarray of `arr` from the 0-based index start(inclusive) to end(exclusive), or `null`, if start is less than 0, greater than length of arr or greater than end.| |`MV_TO_STRING(arr, str)`|Joins all elements of `arr` by the delimiter specified by `str`.| |`STRING_TO_MV(str1, str2)`|Splits `str1` into an array on the delimiter specified by `str2`.| -|`MV_TO_ARRAY(str)`|Converts a multi-value string from a `VARCHAR` to an `ARRAY` instead.| +|`MV_TO_ARRAY(str)`|Converts a multi-value string from a `VARCHAR` to a `VARCHAR ARRAY`.| diff --git a/web-console/script/create-sql-docs.js b/web-console/script/create-sql-docs.js index 561db7e2cc46..794afb06e556 100755 --- a/web-console/script/create-sql-docs.js +++ b/web-console/script/create-sql-docs.js @@ -67,9 +67,10 @@ const readDoc = async () => { await fs.readFile('../docs/querying/sql-data-types.md', 'utf-8'), await fs.readFile('../docs/querying/sql-scalar.md', 'utf-8'), await fs.readFile('../docs/querying/sql-aggregations.md', 'utf-8'), + await fs.readFile('../docs/querying/sql-array-functions.md', 'utf-8'), await fs.readFile('../docs/querying/sql-multivalue-string-functions.md', 'utf-8'), await fs.readFile('../docs/querying/sql-json-functions.md', 'utf-8'), - await fs.readFile('../docs/querying/sql-operators.md', 'utf-8'), + await fs.readFile('../docs/querying/sql-operators.md', 'utf-8') ].join('\n'); const lines = data.split('\n'); From 339094bf5725e9ea6381bda6937f2b184207ea84 Mon Sep 17 00:00:00 2001 From: Clint Wylie Date: Mon, 17 Apr 2023 16:12:51 -0700 Subject: [PATCH 10/10] fix spelling --- docs/querying/sql-data-types.md | 2 +- docs/querying/sql-multivalue-string-functions.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/querying/sql-data-types.md b/docs/querying/sql-data-types.md index 78f7a5f8a6d1..4e6286d032d6 100644 --- a/docs/querying/sql-data-types.md +++ b/docs/querying/sql-data-types.md @@ -108,7 +108,7 @@ The behavior of Druid [multi-value string dimensions](multi-value-dimensions.md) their usage. When used with standard `VARCHAR` functions which expect a single input value per row, such as `CONCAT`, Druid will map -the function across all values in the row. If the row is null or empty, the function recieves `NULL` as its input. +the function across all values in the row. If the row is null or empty, the function receives `NULL` as its input. When used with the explicit [multi-value string functions](./sql-multivalue-string-functions.md), Druid processes the row values as if they were `ARRAY` typed. Any operations which produce null and empty rows are distinguished as diff --git a/docs/querying/sql-multivalue-string-functions.md b/docs/querying/sql-multivalue-string-functions.md index bee77edbe19f..d0d9040a2448 100644 --- a/docs/querying/sql-multivalue-string-functions.md +++ b/docs/querying/sql-multivalue-string-functions.md @@ -40,7 +40,7 @@ See [SQL multi-value strings](./sql-data-types.md#multi-value-strings) and nativ All array references in the multi-value string function documentation can refer to multi-value string columns or `ARRAY` types. These functions are largely identical to the [array functions](./sql-array-functions.md), but use -`VARCHAR` types and behavior.Multi-value strings can also be converted to `ARRAY` types using `MV_TO_ARRAY`. For +`VARCHAR` types and behavior. Multi-value strings can also be converted to `ARRAY` types using `MV_TO_ARRAY`. For additional details about `ARRAY` types, see [`ARRAY` data type documentation](./sql-data-types.md#arrays). |Function|Description|