-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[Improment](executor)Add workload group id in workload policy's property #33483
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thank you for your contribution to Apache Doris. Since 2024-03-18, the Document has been moved to doris-website. |
|
clang-tidy review says "All clean, LGTM! 👍" |
0bdf163 to
b5b3773
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
2 similar comments
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
5c0befd to
e9a4c41
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
1 similar comment
|
clang-tidy review says "All clean, LGTM! 👍" |
43c3f47 to
ff7466d
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
ff7466d to
33dc4c1
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
33dc4c1 to
b40bb8a
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run buildall |
|
TeamCity be ut coverage result: |
| return false; | ||
| } | ||
|
|
||
| public class PolicyProcNode { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个proc 功能,我们如果有了系统表,感觉可以不要了,直接删了吧。我们不维护了。
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
等加上policy的schema table时就删
| private volatile int priority; | ||
|
|
||
| @SerializedName(value = "wgId") | ||
| private long workloadGroupId; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what is the default value when upgrade from old version.
fe/fe-core/src/main/java/org/apache/doris/resource/workloadschedpolicy/WorkloadSchedPolicy.java
Outdated
Show resolved
Hide resolved
| new Column("Priority", ScalarType.createType(PrimitiveType.INT)), | ||
| new Column("Enabled", ScalarType.createType(PrimitiveType.BOOLEAN)), | ||
| new Column("Version", ScalarType.createType(PrimitiveType.INT))); | ||
| new Column("Version", ScalarType.createType(PrimitiveType.INT)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
需要修改对应的regression test。
- 新建policy的时候指定group name, group 不存在异常的处理。
- 修改已经存在的policy,指定group name。
- drop 已经存在的group,关联policy和不关联policy。
|
clang-tidy review says "All clean, LGTM! 👍" |
6 similar comments
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
clang-tidy review says "All clean, LGTM! 👍" |
|
run p0 |
|
run buildall |
TPC-H: Total hot run time: 38844 ms |
TPC-DS: Total hot run time: 184404 ms |
ClickBench: Total hot run time: 29.87 s |
|
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' |
|
TeamCity be ut coverage result: |
| private volatile int priority; | ||
|
|
||
| @SerializedName(value = "wgId") | ||
| private long workloadGroupId = -1; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
store as a list, because user may bind to multi workload groups in the future.
1e037e0 to
d55755c
Compare
|
clang-tidy review says "All clean, LGTM! 👍" |
d55755c to
37e1ba6
Compare
|
run buildall |
|
clang-tidy review says "All clean, LGTM! 👍" |
yiguolei
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
PR approved by at least one committer and no changes requested. |
|
PR approved by anyone and no changes requested. |
TPC-H: Total hot run time: 39047 ms |
TPC-DS: Total hot run time: 184892 ms |
ClickBench: Total hot run time: 30.32 s |
|
Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G' |
|
TeamCity be ut coverage result: |
xinyiZzz
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
Proposed changes
I think the similar workload should be within the same workload group, so a workload group may has many workload policy, but a workload policy should be only binded in one workload group.
I add a
workload_group_idproperty for workload schedule, then a policy with a valid workload group id can only match the workload in its own group.The default value of
workload_group_idis -1 ,it means all queries in the cluster may be limited by the policy.Drop workload group check
If a workload group has a related policy, it can not be dropped.
you need drop policy first or clear policy's workload group property.