Skip to content

[tensor] Add interface for colo tesnor dp size#3227

Merged
FrankLeeeee merged 1 commit intohpcaitech:mainfrom
yhna940:chore/colotensor-interface
Mar 27, 2023
Merged

[tensor] Add interface for colo tesnor dp size#3227
FrankLeeeee merged 1 commit intohpcaitech:mainfrom
yhna940:chore/colotensor-interface

Conversation

@yhna940
Copy link
Copy Markdown
Contributor

@yhna940 yhna940 commented Mar 24, 2023

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

N/A

📝 What does this PR do?

This pull request updates the get_dp_world_size method of the ColoTensor class to provide the distributed data parallel (DP) world size of the tensor (Similar to get_tp_world_size method).

Changes

  1. Added the get_dp_world_size method to the ColoTensor class
  2. Updated the code that calls tensor.process_group.dp_world_size() to use the new method

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

@FrankLeeeee
Copy link
Copy Markdown
Contributor

Appreciate your contribution!

@FrankLeeeee FrankLeeeee merged commit 1a22904 into hpcaitech:main Mar 27, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants