Skip to content

[FX] refactor experimental tracer and adapt it with hf models#3157

Merged
FrankLeeeee merged 16 commits intohpcaitech:mainfrom
YuliangLiu0306:feature/integrate_tracer
Mar 22, 2023
Merged

[FX] refactor experimental tracer and adapt it with hf models#3157
FrankLeeeee merged 16 commits intohpcaitech:mainfrom
YuliangLiu0306:feature/integrate_tracer

Conversation

@YuliangLiu0306
Copy link
Copy Markdown
Contributor

@YuliangLiu0306 YuliangLiu0306 commented Mar 17, 2023

📌 Checklist before creating the PR

  • I have created an issue for this PR for traceability
  • The title follows the standard format: [doc/gemini/tensor/...]: A concise description
  • I have added relevant tags if possible for us to better distinguish different PRs

🚨 Issue number

part effort of #3160

📝 What does this PR do?

Summarize your work here.
if you have any plots/diagrams/screenshots/tables, please attach them here.

💥 Checklist before requesting a review

  • I have linked my PR to an issue (instruction)
  • My issue clearly describes the problem/feature/proposal, with diagrams/charts/table/code if possible
  • I have performed a self-review of my code
  • I have added thorough tests.
  • I have added docstrings for all the functions/methods I implemented

⭐️ Do you enjoy contributing to Colossal-AI?

  • 🌝 Yes, I do.
  • 🌚 No, I don't.

Tell us more if you don't enjoy contributing to Colossal-AI.

@YuliangLiu0306 YuliangLiu0306 changed the title Feature/integrate tracer [FX] refactor experimental tracer and adapt it with hf models Mar 17, 2023
@YuliangLiu0306 YuliangLiu0306 requested review from 1SAA, FrankLeeeee and ver217 and removed request for FrankLeeeee and ver217 March 20, 2023 02:39
@YuliangLiu0306 YuliangLiu0306 added Run Build and Test auto-parallel related to the auto-parallel feature labels Mar 20, 2023
@YuliangLiu0306 YuliangLiu0306 force-pushed the feature/integrate_tracer branch from 1b17c35 to 712c53f Compare March 20, 2023 08:24
@YuliangLiu0306
Copy link
Copy Markdown
Contributor Author

image

@Issues-translate-bot
Copy link
Copy Markdown

Bot detected the issue body's language is not English, translate it automatically. 👯👭🏻🧑‍🤝‍🧑👫🧑🏿‍🤝‍🧑🏻👩🏾‍🤝‍👨🏿👬🏿


image

Comment thread tests/kit/model_zoo/__init__.py Outdated
Comment thread tests/test_fx/test_tracer/test_timm_model/test_timm_model.py Outdated
@github-actions
Copy link
Copy Markdown
Contributor

The code coverage for the changed files is 36%.

Click me to view the complete report
Name                                                                         Stmts   Miss  Cover
------------------------------------------------------------------------------------------------
colossalai/_analyzer/_subclasses/_meta_registration.py                         227    209     8%
colossalai/_analyzer/_subclasses/_monkey_patch.py                               17      3    82%
colossalai/_analyzer/_subclasses/flop_tensor.py                                242    208    14%
colossalai/_analyzer/fx/__init__.py                                              3      0   100%
colossalai/_analyzer/fx/graph_module.py                                        119     62    48%
colossalai/_analyzer/fx/node_util.py                                            97     50    48%
colossalai/_analyzer/fx/passes/shape_prop.py                                    75     51    32%
colossalai/_analyzer/fx/tracer/__init__.py                                       2      0   100%
colossalai/_analyzer/fx/tracer/bias_addition.py                                 60     35    42%
colossalai/_analyzer/fx/tracer/custom_leaf_module.py                            22      9    59%
colossalai/_analyzer/fx/tracer/proxy.py                                         80     44    45%
colossalai/_analyzer/fx/tracer/symbolic_trace.py                                30     12    60%
colossalai/_analyzer/fx/tracer/tracer.py                                       222    174    22%
tests/kit/model_zoo/__init__.py                                                  3      0   100%
tests/kit/model_zoo/transformers/gpt.py                                         22      0   100%
tests/test_analyzer/test_fx/test_bias_addition.py                               71     40    44%
tests/test_fx/test_tracer/test_hf_model/hf_tracer_utils.py                      19     13    32%
tests/test_fx/test_tracer/test_hf_model/test_hf_albert.py                       15      5    67%
tests/test_fx/test_tracer/test_hf_model/test_hf_bert.py                         13      5    62%
tests/test_fx/test_tracer/test_hf_model/test_hf_gpt.py                          15      7    53%
tests/test_fx/test_tracer/test_hf_model/test_hf_opt.py                          13      5    62%
tests/test_fx/test_tracer/test_hf_model/test_hf_t5.py                           13      5    62%
tests/test_fx/test_tracer/test_timm_model/test_timm_model.py                    33     24    27%
tests/test_fx/test_tracer/test_torchaudio_model/test_torchaudio_model.py        12      5    58%
tests/test_fx/test_tracer/test_torchaudio_model/torchaudio_utils.py             17     14    18%
tests/test_fx/test_tracer/test_torchrec_model/test_deepfm_model.py              37      3    92%
tests/test_fx/test_tracer/test_torchrec_model/test_dlrm_model.py                39      4    90%
tests/test_fx/test_tracer/test_torchvision_model/test_torchvision_model.py      29      3    90%
------------------------------------------------------------------------------------------------
TOTAL                                                                         1547    990    36%

@FrankLeeeee FrankLeeeee merged commit f57d349 into hpcaitech:main Mar 22, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

auto-parallel related to the auto-parallel feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants