-
Notifications
You must be signed in to change notification settings - Fork 3.7k
[Feature] Limit frequency of query and load in system level #7474
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
df13c4a to
2da67e8
Compare
|
Good idea, Impala’s AdmissionController does a similar thing, Introduction is here https://shimo.im/docs/6qxjctpyDHJgPwtw In #7198, I have counted the real memory consumption of each Query, which may be implemented based on your PR later |
|
Hi @tianhui5 , thanks for your contribution. |
e506f03 to
5b64c77
Compare
Thanks for remind me. I've add relative doc and unit test, please review this PR |
Nice work. |
5bb513d to
356bae1
Compare
fe/fe-core/src/main/java/org/apache/doris/http/rest/LoadAction.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/UserProperty.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/mysql/privilege/UserProperty.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/qe/ConnectProcessor.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/system/RateLimiterMgr.java
Outdated
Show resolved
Hide resolved
fe/fe-core/src/main/java/org/apache/doris/system/RateLimiterMgr.java
Outdated
Show resolved
Hide resolved
|
There are two main issues
|
356bae1 to
cf438dc
Compare
Yes, that seems to have too much RPC. I have rewrote the feature with statistic sychronize based on bdbje. Because I think it's fine to use bdbje to store some temperory statistics, and there is too much work to develop a middleware myself. Please review this patch, thanks for your time! |
d73a194 to
d73d4f2
Compare
|
Hi @tianhui5 Although we had some private discussions about this feature, we eventually needed a complete design and usage document to help all developers and users understand how this feature works and how to use it. It is recommended that the final design be fully described in the issue #7129 (and the previous design be marked as Deprecated). Also, I suggest a separate user document describing the functionality from a usage perspective, including the followings:
In the previous Doris development, we also had a lot of features that were difficult to maintain due to lack of documentation, and I hope to be able to improve the process in this area in future projects. Thanks! |
Done. |
|
We're closing this PR because it hasn't been updated in a while. |

Proposed changes
Fix #7129
Types of changes
Checklist