[Feature] Add a http interface for single tablet migration between different disks #5101
Conversation
| curl -X GET http://be_host:webserver_port/tablet_migration?tablet_id=xxx&schema_hash=xxx&disk=xxx | ||
| ``` | ||
|
|
||
| 返回值就是tablet迁移结果，迁移成功会返回： |
There was a problem hiding this comment.
迁移可能是一个非常长的过程，如果是同步接口的话，可能会导致客户端hang住很长时间？
是否可以考虑做成一个异步接口，调用方只是触发一个 storage_migration的 task放入队列即返回？
然后可以通过接口来查询这个任务是否在运行？类似 http/action/compaction_action 那样?
morningman
added
kind/feature
| Submit the migration task: | ||
|
|
||
| ``` | ||
| curl -X GET http://be_host:webserver_port/tablet_migration?goal=run&tablet_id=xxx&schema_hash=xxx&disk=xxx |
There was a problem hiding this comment.
Recommended: /api/tablet_migration?.....
| Show the status of migration task: | ||
|
|
||
| ``` | ||
| curl -X GET http://be_host:webserver_port/tablet_migration?goal=status&tablet_id=xxx&schema_hash=xxx&disk=xxx |
There was a problem hiding this comment.
Recommended: /api/tablet_migration?.....
And only tablet_id and schema_hash is needed.
the result will return which disk it is moved to.
| } | ||
| } | ||
| { | ||
| std::unique_lock<std::mutex> lock(_migration_status_mutex); |
There was a problem hiding this comment.
Looks like you can move this block to the above code block, to avoid lock _migration_status_mutex twice?
There was a problem hiding this comment.
And the "submitted" status seems not necessary? It will be changed to "running" very soon. And it it submit failed, it will be removed from _migration_tasks.
There was a problem hiding this comment.
Looks like you can move this block to the above code block, to avoid lock
_migration_status_mutextwice?
OK.
There was a problem hiding this comment.
And the
"submitted"status seems not necessary? It will be changed to "running" very soon. And it it submit failed, it will be removed from _migration_tasks.
When doing tablet rebalance between disks, there may be many migration tasks being submitted and waiting in the queue. So "submitted" status may be useful.
| do { | ||
| std::unique_lock<std::mutex> lock(_migration_status_mutex); | ||
| std::map<MigrationTask, std::string>::iterator it_task = _migration_tasks.begin(); | ||
| for (; it_task != _migration_tasks.end(); it_task++) { |
There was a problem hiding this comment.
_migration_tasks is a map, so why not just find the task by key?
You can refer to https://www.cnblogs.com/xupeidong/p/11976671.html
There was a problem hiding this comment.
_migration_tasksis a map, so why not just find the task by key?
You can refer tohttps://www.cnblogs.com/xupeidong/p/11976671.html
OK. Thank you.
| LOG(WARNING) << "tablet migrate failed. tablet_id=" << tablet_id | ||
| << ", schema_hash=" << schema_hash << ", dest_disk=" << dest_disk | ||
| << ", status:" << res; | ||
| status = Status::InternalError("migration task failed"); |
There was a problem hiding this comment.
better to add res to the message in status, so that user can known what's wrong
There was a problem hiding this comment.
better to add
resto the message instatus, so that user can known what's wrong
That sounds reasonable.
weizuo93
force-pushed
the
single_tablet_migration_interface
branch
from
2b2f7de to
83d8ba2
Compare
…fferent disks (apache#5101) Based on PR apache#4475, this patch add a new feature for single tablet migration between different disks by http. Co-authored-by: weizuo <weizuo@xiaomi.com>
…fferent disks (apache#5101) Based on PR apache#4475, this patch add a new feature for single tablet migration between different disks by http. Co-authored-by: weizuo <weizuo@xiaomi.com>
2 participants
Proposed changes
Based on PR #4475, this patch add a new feature for single tablet migration between different disks by http.
Types of changes
What types of changes does your code introduce to Doris?
Put an
xin the boxes that applyChecklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your code.