update docs after legacy configs have been removed by clintropolis · Pull Request #17634 · apache/druid

clintropolis · 2025-01-16T11:23:46Z

Description

Update docs to reflect removal of legacy config options in #17609. Also made the services fail if any of these configs are set to the removed modes, with the help of a new PropertiesValidator that is baked into StartupInjectorBuilder, allowing for a centralized place to handle checking stuff about runtime.properties.

If set, service will do a log message that points to the migration guide and then exit:

Exception in thread "main" org.apache.druid.java.util.common.ISE: druid.generic.useDefaultValueForNull set to 'true', but has been removed, see https://druid.apache.org/docs/latest/release-info/migr-ansi-sql-null for details for how to migrate to SQL compliant behavior
	at org.apache.druid.guice.StartupInjectorBuilder$PropertiesValidator.validate(StartupInjectorBuilder.java:138)
	at org.apache.druid.guice.StartupInjectorBuilder.build(StartupInjectorBuilder.java:71)
	at org.apache.druid.cli.Main.main(Main.java:96)

Process finished with exit code 1

Currently this new validator only checks the 3 properties relevant to this PR to fail fast, but I can also imagine using this to track usage of deprecated properties and doing something like log.alert or something similar to both log and emit metrics to help monitor usage of deprecated properties slated for future removal.

techdocsmith

minor stylistic nits, otherwise docs changes LGTM

techdocsmith · 2025-01-16T20:06:18Z

-If `druid.generic.useDefaultValueForNull` is false, null values will be tracked in the `missingValueCount` field of the histogram.
-
-If `druid.generic.useDefaultValueForNull` is true, null values will be added to the histogram as the default 0.0 value.
+Null values will be tracked in the `missingValueCount` field of the histogram.


Suggested change

Null values will be tracked in the `missingValueCount` field of the histogram.

Druid tracks null values in the `missingValueCount` field of the histogram.

Nit: avoid future tense

techdocsmith · 2025-01-16T20:09:43Z

 | `initialValue` | The initial value of the accumulator for the `fold` (and `combine`, if `InitialCombineValue` is null) expression. | Yes |
 | `initialCombineValue` | The initial value of the accumulator for the `combine` expression. | No. Default `initialValue`. |
-| `isNullUnlessAggregated` | Indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | No. Defaults to the value of `druid.generic.useDefaultValueForNull`. |
+| `isNullUnlessAggregated` | Indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | No. Defaults to `true`. |


Suggested change

| `isNullUnlessAggregated` | Indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. | No. Defaults to `true`. |

| `isNullUnlessAggregated` | Sets the default output value to `null` when the aggregator does not process any rows. If true, the value is `null`. If false, Druid computes the value as the result of running the expressions with initial values. | No. Defaults to `true`. |

nit awkward passive.

techdocsmith · 2025-01-16T20:11:13Z

 |rint|rint(x) returns value that is closest in value to x and is equal to a mathematical integer|
 |round|round(x, y) returns the value of the x rounded to the y decimal places. While x can be an integer or floating-point number, y must be an integer. The type of the return value is specified by that of x. y defaults to 0 if omitted. When y is negative, x is rounded on the left side of the y decimal points. If x is `NaN`, x returns 0. If x is infinity, x will be converted to the nearest finite double. |
-|safe_divide|safe_divide(x,y) returns the division of x by y if y is not equal to 0. In case y is 0 it returns `null` or 0 if `druid.generic.useDefaultValueForNull=true` (legacy mode) |
+|safe_divide|safe_divide(x,y) returns the division of x by y if y is not equal to 0. In case y is 0 it returns `null`|


i flipped this around because its actually "returns null if y is 0"

techdocsmith · 2025-01-16T20:11:55Z

-## Logical operator modes
-In Druid 28.0 and later, `druid.expressions.useStrictBooleans=true` is set by default. Logical operations treat `null` values as "unknown" for SQL compatible behavior. _All boolean output functions_ will output 'homogeneous' `LONG` typed boolean values of `1` for `true` and `0` for `false`. 
+## Logical operator behavior
+Logical operations treat `null` values as "unknown" for SQL compatible behavior. _All boolean output functions_ will output `LONG` typed boolean values of `1` for `true` and `0` for `false`. 


Suggested change

Logical operations treat `null` values as "unknown" for SQL compatible behavior. _All boolean output functions_ will output `LONG` typed boolean values of `1` for `true` and `0` for `false`.

Logical operations treat `null` values as "unknown" for SQL compatible behavior. _All boolean output functions_ output `LONG` typed boolean values of `1` for `true` and `0` for `false`.

nit

…ions

* update docs after legacy configs have been removed * better runtime.properties validation with new StartupInjectorBuilder.PropertiesValidator to fail fast if removed properties are set

update docs after legacy configs have been removed

9a4af94

github-actions Bot added the Area - Documentation label Jan 16, 2025

clintropolis added this to the 32.0.0 milestone Jan 16, 2025

techdocsmith approved these changes Jan 16, 2025

View reviewed changes

better runtime.properties validation, update docs from review suggest…

05ca7cb

…ions

gianm approved these changes Jan 17, 2025

View reviewed changes

clintropolis merged commit e805c9a into apache:master Jan 17, 2025

clintropolis deleted the update-docs-sql-compat branch January 17, 2025 01:15

This was referenced Jan 17, 2025

[Backport] update docs after legacy configs have been removed #17638

Merged

remove json column v4 format writers #17625

Merged

clintropolis mentioned this pull request Mar 5, 2025

remove NullValueHandlingConfig, NullHandlingModule, NullHandling #17778

Merged

clintropolis mentioned this pull request Apr 4, 2025

remove some stale configs from example configs, minor doc update #17875

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

update docs after legacy configs have been removed#17634

update docs after legacy configs have been removed#17634
clintropolis merged 2 commits intoapache:masterfrom
clintropolis:update-docs-sql-compat

clintropolis commented Jan 16, 2025 •

edited

Loading

Uh oh!

techdocsmith left a comment

Uh oh!

techdocsmith Jan 16, 2025

Uh oh!

techdocsmith Jan 16, 2025

Uh oh!

techdocsmith Jan 16, 2025

Uh oh!

clintropolis Jan 17, 2025

Uh oh!

techdocsmith Jan 16, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

	Null values will be tracked in the `missingValueCount` field of the histogram.
	Druid tracks null values in the `missingValueCount` field of the histogram.

	\| `isNullUnlessAggregated` \| Indicates that the default output value should be `null` if the aggregator does not process any rows. If true, the value is `null`, if false, the result of running the expressions with initial values is used instead. \| No. Defaults to `true`. \|
	\| `isNullUnlessAggregated` \| Sets the default output value to `null` when the aggregator does not process any rows. If true, the value is `null`. If false, Druid computes the value as the result of running the expressions with initial values. \| No. Defaults to `true`. \|

	\|safe_divide\|safe_divide(x,y) returns the division of x by y if y is not equal to 0. In case y is 0 it returns `null`\|
	\|safe_divide\|safe_divide(x,y) returns the division of x by y if y is not equal to 0. Returns 0 if y is `null` \|

	Logical operations treat `null` values as "unknown" for SQL compatible behavior. _All boolean output functions_ will output `LONG` typed boolean values of `1` for `true` and `0` for `false`.
	Logical operations treat `null` values as "unknown" for SQL compatible behavior. _All boolean output functions_ output `LONG` typed boolean values of `1` for `true` and `0` for `false`.

Conversation

clintropolis commented Jan 16, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Uh oh!

techdocsmith left a comment

Choose a reason for hiding this comment

Uh oh!

techdocsmith Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

techdocsmith Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

techdocsmith Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

clintropolis Jan 17, 2025

Choose a reason for hiding this comment

Uh oh!

techdocsmith Jan 16, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

clintropolis commented Jan 16, 2025 •

edited

Loading