k8s discovery module: fix issue for druid.host being more than 63chars not permitted as k8s resource label value#10961
Conversation
…s not permitted as k8s resource label value
| } | ||
| int hash = str.hashCode(); | ||
| if (hash < 0) { | ||
| hash = -1 * hash; |
There was a problem hiding this comment.
nit: Just wondering if this is a best way to solve 63chars limitation using String.hascode(). As we know, String.hashCode() isn't unique, but it can't be(https://sigpwned.com/2018/08/10/string-hashcode-is-plenty-unique/). And hash = -1 * hash will double the probability of conflict.
In other words, what will happen if there is a conflict? Does the conflict will lead to a wrong SelfDiscovery? If not, I think String.hascode() is good enough.
There was a problem hiding this comment.
hash collision here has no impact on correctness , this is only used to get a suitable list of candidates. and, this is only used for finding a specific node by host:port for the health check.
There was a problem hiding this comment.
Thanks for your explanation :)
…s not permitted as k8s resource label value (apache#10961) * k8s discovery module: fix issue for druid.host being more than 63chars not permitted as k8s resource label value * update doc * fix test
4 participants
Description
Fixes a bug which shows up in specific deployments, that use k8s based discovery module, where
druid.hostgets set to something more than 63 characters which is the max length limit on kubernetes label values.Currently we put full
host:portas a k8s label inside pod spec, which can be of arbitrary length. This patch changes node discovery mechanism to work with a hash ofhost:portin the label value instead.It is the (2) issue described in #10752 (comment) . Since this issue will be encountered by users quickly so hoping to have this included in 0.21.0 release.
PS: Existing [unit/integration] tests cover the changes made here. It would further be exercised in a larger PR that introduces leader election improvements and integration tests similar to #10680 would be done later, that is blocked on a new release from kubernetes java client lib we use, that contains few leader election algorithm improvements.
This PR has: