[Feature][Stream Load] Add "show stream load" to show stream load record

[weizuo93]

Proposed changes

Add the sql of show stream load to show the result of recent stream load jobs.

Types of changes

What types of changes does your code introduce to Doris?
Put an x in the boxes that apply

• [] Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• [] Breaking change (fix or feature that would cause existing functionality to not work as expected)
• [] Documentation Update (if none of the other choices apply)
• Code refactor (Modify the code structure, format the code, etc...)

Checklist

Put an x in the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.

• [] I have created an issue on (Fix #ISSUE) and described the bug/feature there in detail
• Compiling and unit tests pass locally with my changes
• [] I have added tests that prove my fix is effective or that my feature works
• [] If these changes need document changes, I have updated the document
• Any dependent changes have been merged

[kangkaisen]

@weizuo93 Hi, Thanks for your contribution.
I don't understand why we need to store the stream load meta both in FE and BE. why we couldn't get stream load info from FE meta?

[weizuo93]

@weizuo93 Hi, Thanks for your contribution.
I don't understand why we need to store the stream load meta both in FE and BE. why we couldn't get stream load info from FE meta?

@kangkaisen Our goal is to store the stream load meta in FE. The reason why the stream load meta is stored on BE first is to send the meta to FE in bulk to avoid calling RPC every time the stream load finished. After a certain period of time, stream load meta will be removed from BE.

[kangkaisen]

@weizuo93
Hi, I mean In LoadManager, dbIdToLabelToLoadJobs already has the stream load join info, why we still need to store stream load job meta in BE? in other words, I think we don't need to store stream load job meta in BE.

[weizuo93]

@weizuo93
Hi, I mean In LoadManager, dbIdToLabelToLoadJobs already has the stream load join info, why we still need to store stream load job meta in BE? in other words, I think we don't need to store stream load job meta in BE.

@kangkaisen As far as I know, there is no info about stream load in LoadManager and dbIdToLabelToLoadJobs. LoadManager only manages bulk load.

[kangkaisen]

@weizuo93 You are right. I am wrong. This PR could go ahead. (But store some load job meta in FE and store some load job meta in BE, which seems not very good)

[weizuo93]

@weizuo93 You are right. I am wrong. This PR could go ahead. (But store some load job meta in FE and store some load job meta in BE, which seems not very good)

@kangkaisen
Thanks for your comment. There is code overlap between this PR and PR #5452. After discussion with Mingyu Chen, we first push forward PR #5452. About storing stream load meta in BE, it's a tradeoff. If BE reports the stream load result to FE when each stream load job finished, it may result in too much stress for FE RPC. So the stream load results are temporarily stored on BE and then sent to FE by batch. After finished data sending, stream load meta will be removed from BE soon. Do you have any better idea?

[kangkaisen]

@weizuo93 Ok, Thanks for your explain. which is OK for me

[yangzhg]

show stream load is better, like show routine load

[weizuo93]

show stream load is better, like show routine load

OK, thank you.

[yangzhg]

LGTM

[yangzhg]

LGTM