diff --git a/applications/Chat/coati/ray/README.md b/applications/Chat/coati/ray/README.md
index f9133b049446..eaddc4f9f4f4 100644
--- a/applications/Chat/coati/ray/README.md
+++ b/applications/Chat/coati/ray/README.md
@@ -2,17 +2,17 @@
## Detach Experience Makers and Trainers
- We can completely separate the trainers and makers.
+ We can completely separate the trainers and makers.
-
+
- The experience maker performs inference, produces experience, and remotely delivers it to the trainer (1).
-- The trainer consumes experience to train models, and periodically transmits new model parameters to the maker (2.1, 2.2).
+- The trainer consumes experience to train models, and periodically transmits new model parameters to the maker (2.1, 2.2).
- Using an experience buffer to overlap transmission and computing.
-In this manner, each node will work continuously without model idle time, and different optimization strategies can be applied for inference and training to meet the needs of speed or storage. It is also helpful for scalability.
+In this manner, each node will work continuously without model idle time, and different optimization strategies can be applied for inference and training to meet the needs of speed or storage. It is also helpful for scalability.
`DetachedPPOTrainer` and `ExperienceMakerHolder` are Ray Actors (distinguished from Actor Model), representing Trainer and Experience Maker on the graph above, respectively.
@@ -24,9 +24,9 @@ See examples at `ColossalAI/application/Chat/examples/ray`
### Setup Makers
-- define makers' environment variables :
+- define makers' environment variables :
- ```python
+ ```python
env_info_makers = [{
'local_rank': '0',
'rank': str(rank),
@@ -36,8 +36,8 @@ See examples at `ColossalAI/application/Chat/examples/ray`
} for rank in range(num_makers)]
```
-- define maker models :
- ```python
+- define maker models :
+ ```python
def model_fn():
actor = get_actor_from_args(...)
critic = get_critic_from_args(...)
@@ -46,27 +46,27 @@ See examples at `ColossalAI/application/Chat/examples/ray`
return actor, critic, reward_model, initial_model
```
-- set experience_holder_refs :
+- set experience_holder_refs :
- ```python
+ ```python
experience_holder_refs = [
ExperienceMakerHolder.options(
- name=f"maker_{i}",
- num_gpus=1,
+ name=f"maker_{i}",
+ num_gpus=1,
max_concurrency=2
).remote(
detached_trainer_name_list=[f"trainer_{x}" for x in target_trainers(...)],
model_fn=model_fn,
- ...)
+ ...)
for i, env_info_maker in enumerate(env_info_makers)
]
```
- The names in the `detached_trainer_name_list` refer to the target trainers that the maker should send experience to.
- We set a trainer's name the same as a maker, by `.options(name="str")`. See below.
+ The names in the `detached_trainer_name_list` refer to the target trainers that the maker should send experience to.
+ We set a trainer's name the same as a maker, by `.options(name="str")`. See below.
### Setup Trainers
-- define trainers' environment variables :
+- define trainers' environment variables :
```python
env_info_trainers = [{
'local_rank': '0',
@@ -76,7 +76,7 @@ See examples at `ColossalAI/application/Chat/examples/ray`
'master_addr': master_addr
} for rank in range(num_trainers)]
```
-- define trainer models :
+- define trainer models :
```python
def trainer_model_fn():
@@ -88,8 +88,8 @@ See examples at `ColossalAI/application/Chat/examples/ray`
```python
trainer_refs = [
DetachedPPOTrainer.options(
- name=f"trainer{i}",
- num_gpus=1,
+ name=f"trainer{i}",
+ num_gpus=1,
max_concurrency=2
).remote(
experience_maker_holder_name_list=[f"maker{x}" for x in target_makers(...)],
@@ -98,22 +98,22 @@ See examples at `ColossalAI/application/Chat/examples/ray`
for i, env_info_trainer in enumerate(env_info_trainers)
]
```
- The names in `experience_maker_holder_name_list` refer to the target makers that the trainer should send updated models to.
+ The names in `experience_maker_holder_name_list` refer to the target makers that the trainer should send updated models to.
By setting `detached_trainer_name_list` and `experience_maker_holder_name_list`, we can customize the transmission graph.
-### Launch Jobs
-- define data_loader :
+### Launch Jobs
+- define data_loader :
```python
def data_loader_fn():
return = torch.utils.data.DataLoader(dataset=dataset)
```
-- launch makers :
+- launch makers :
```python
wait_tasks = []
for experience_holder_ref in experience_holder_refs:
wait_tasks.append(
- experience_holder_ref.workingloop.remote(data_loader_fn(),
+ experience_holder_ref.workingloop.remote(data_loader_fn(),
num_steps=experience_steps))
```
@@ -135,26 +135,26 @@ We can deploy different strategies to makers and trainers. Here are some notions
### 2 Makers 1 Trainer
-
+
### 2 Makers 2 Trainer2
-
+
### Maker Inference Quantization
-
+
### Tensor Parallel
-
+
-## TODO
+## TODO
- [ ] Support LoRA
- [ ] Support TP & PP
diff --git a/applications/Chat/coati/ray/assets/2m1t.png b/applications/Chat/coati/ray/assets/2m1t.png
deleted file mode 100644
index 9281943570d3..000000000000
Binary files a/applications/Chat/coati/ray/assets/2m1t.png and /dev/null differ
diff --git a/applications/Chat/coati/ray/assets/2m2t.png b/applications/Chat/coati/ray/assets/2m2t.png
deleted file mode 100644
index 94ca1c361985..000000000000
Binary files a/applications/Chat/coati/ray/assets/2m2t.png and /dev/null differ
diff --git a/applications/Chat/coati/ray/assets/2m2t_quantize.png b/applications/Chat/coati/ray/assets/2m2t_quantize.png
deleted file mode 100644
index 32f758bfd249..000000000000
Binary files a/applications/Chat/coati/ray/assets/2m2t_quantize.png and /dev/null differ
diff --git a/applications/Chat/coati/ray/assets/basic_structure.png b/applications/Chat/coati/ray/assets/basic_structure.png
deleted file mode 100644
index 38cc9ee58f0f..000000000000
Binary files a/applications/Chat/coati/ray/assets/basic_structure.png and /dev/null differ
diff --git a/applications/Chat/coati/ray/assets/tp_ddp_hybrid.png b/applications/Chat/coati/ray/assets/tp_ddp_hybrid.png
deleted file mode 100644
index 52cd8f875f84..000000000000
Binary files a/applications/Chat/coati/ray/assets/tp_ddp_hybrid.png and /dev/null differ