From 202f55c4e5596059a10851255003149335f2f738 Mon Sep 17 00:00:00 2001 From: FrankLeeeee Date: Wed, 11 Jan 2023 10:09:15 +0800 Subject: [PATCH 1/7] [example] improved the clarity yof the example readme --- examples/README.md | 48 +++++++++++++++++++++++++++++----------------- 1 file changed, 30 insertions(+), 18 deletions(-) diff --git a/examples/README.md b/examples/README.md index 53ab0896da0b..78facea5406d 100644 --- a/examples/README.md +++ b/examples/README.md @@ -1,28 +1,40 @@ -## Examples folder document +# Colossal-AI Examples ## Table of Contents - -## Example folder description +- [Colossal-AI Examples](#colossal-ai-examples) + - [Table of Contents](#table-of-contents) + - [Overview](#overview) + - [Folder Structure](#folder-structure) + - [Integrate Your Example With Testing](#integrate-your-example-with-testing) -This folder provides several examples using colossalai. The images folder includes model like diffusion, dreambooth and vit. The language folder includes gpt, opt, palm and roberta. The tutorial folder is for concept illustration, such as auto-parallel, hybrid-parallel and so on. +## Overview +This folder provides several examples accelerated by Colossal-AI. The `tutorial` folder is for everyone to quickly try out the different features in Colossal-AI. Other folders such as `images` and `language` include a wide range of deep learning tasks and applications. -## Integrate Your Example With System Testing +## Folder Structure -For example code contributor, to meet the expectation and test your code automatically using github workflow function, here are several steps: +```text +└─ examples + └─ images + └─ vit + └─ test_ci.sh + └─ train.py + └─ README.md + └─ ... + └─ ... +``` +## Integrate Your Example With Testing -- (must) Have a test_ci.sh file in the folder like shown below in 'File Structure Chart' -- The dataset should be located in the company's machine and can be announced using environment variable and thus no need for a separate terminal command. -- The model parameters should be small to allow fast testing. -- File Structure Chart +Regular checks are important to ensure that all examples run without apparent bugs and stay compatible with the latest API. +Colossal-AI runs workflows to check for examples on a on-pull-request and weekly basis. +When a new example is added or changed, the workflow will run the example to test whether it can run. +Moreover, Colossal-AI will run testing for examples every week. - └─examples - └─images - └─vit - └─requirements.txt - └─test_ci.sh +Therefore, it is essential for the example contributors to know how to integrate your example with the testing workflow. Simply, you can follow the steps below. + +1. Create a script called `test_ci.sh` in your example folder +2. Configure your testing parameters such as number steps, batch size in `test_ci.sh`, e.t.c. Keep these parameters small such that each example only takes several minutes. +3. Export your dataset path with the prefix `/data` and make sure you have a copy of the dataset in the `/data/scratch/examples-data` directory on the CI machine. Community contributors can contact us via slack to request for downloading the dataset on the CI machine. +4. Implement the logic such as dependency setup and example execution From 89f4253f4766553f8927f5aabefc7a2c3d5e65bb Mon Sep 17 00:00:00 2001 From: FrankLeeeee Date: Wed, 11 Jan 2023 10:22:47 +0800 Subject: [PATCH 2/7] polish workflow --- .github/workflows/auto_example_check.yml | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/.github/workflows/auto_example_check.yml b/.github/workflows/auto_example_check.yml index d9063bad9f33..d440dd782975 100644 --- a/.github/workflows/auto_example_check.yml +++ b/.github/workflows/auto_example_check.yml @@ -41,7 +41,7 @@ jobs: if [ "$x" = "[]" ]; then echo "anyChanged=false" >> $GITHUB_OUTPUT - echo "matrix=null" >> $GITHUB_OUTPUT + echo "matrix={\"directory\": ['null']}" >> $GITHUB_OUTPUT else dirs=$( IFS=',' ; echo "${res[*]}" ) echo "anyChanged=true" >> $GITHUB_OUTPUT @@ -54,7 +54,8 @@ jobs: if: | github.event.pull_request.draft == false && github.base_ref == 'main' && - github.event.pull_request.base.repo.full_name == 'hpcaitech/ColossalAI' && github.event_name == 'pull_request' + github.event.pull_request.base.repo.full_name == 'hpcaitech/ColossalAI' && github.event_name == 'pull_request' && + needs.detect-changed-example.outputs.anyChanged == 'true' name: Test the changed example needs: detect-changed-example runs-on: [self-hosted, gpu] From 52d93c13f150476c25f0bcfbfec03bb7f57fe7ae Mon Sep 17 00:00:00 2001 From: FrankLeeeee Date: Wed, 11 Jan 2023 10:28:00 +0800 Subject: [PATCH 3/7] polish workflow --- .github/workflows/auto_example_check.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/auto_example_check.yml b/.github/workflows/auto_example_check.yml index d440dd782975..867b61b04a18 100644 --- a/.github/workflows/auto_example_check.yml +++ b/.github/workflows/auto_example_check.yml @@ -41,7 +41,7 @@ jobs: if [ "$x" = "[]" ]; then echo "anyChanged=false" >> $GITHUB_OUTPUT - echo "matrix={\"directory\": ['null']}" >> $GITHUB_OUTPUT + echo "matrix={\"directory\": [\"null\"]}" >> $GITHUB_OUTPUT else dirs=$( IFS=',' ; echo "${res[*]}" ) echo "anyChanged=true" >> $GITHUB_OUTPUT From 32852ab5d8467a413493f70ba019720ae769d981 Mon Sep 17 00:00:00 2001 From: FrankLeeeee Date: Wed, 11 Jan 2023 10:29:49 +0800 Subject: [PATCH 4/7] polish workflow --- .github/workflows/auto_example_check.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/.github/workflows/auto_example_check.yml b/.github/workflows/auto_example_check.yml index 867b61b04a18..27494ae98d80 100644 --- a/.github/workflows/auto_example_check.yml +++ b/.github/workflows/auto_example_check.yml @@ -42,10 +42,12 @@ jobs: if [ "$x" = "[]" ]; then echo "anyChanged=false" >> $GITHUB_OUTPUT echo "matrix={\"directory\": [\"null\"]}" >> $GITHUB_OUTPUT + echo "false" else dirs=$( IFS=',' ; echo "${res[*]}" ) echo "anyChanged=true" >> $GITHUB_OUTPUT echo "matrix={\"directory\":$(echo "$dirs")}" >> $GITHUB_OUTPUT + echo "true" fi # If no file is changed, it will prompt an error and shows the matrix do not have value. From f32d1d2574c721eee629d0974a633d61c8161f32 Mon Sep 17 00:00:00 2001 From: FrankLeeeee Date: Wed, 11 Jan 2023 10:31:13 +0800 Subject: [PATCH 5/7] polish workflow --- .github/workflows/auto_example_check.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/auto_example_check.yml b/.github/workflows/auto_example_check.yml index 27494ae98d80..7a9cedb4a4ba 100644 --- a/.github/workflows/auto_example_check.yml +++ b/.github/workflows/auto_example_check.yml @@ -39,7 +39,7 @@ jobs: res=`python .github/workflows/scripts/example_checks/detect_changed_example.py --fileNameList $changedFileName` echo "All changed examples are $res" - if [ "$x" = "[]" ]; then + if [ "$res" = "[]" ]; then echo "anyChanged=false" >> $GITHUB_OUTPUT echo "matrix={\"directory\": [\"null\"]}" >> $GITHUB_OUTPUT echo "false" From 323db221b2cf014c2b12ad31c18a7c9f46e2aeb0 Mon Sep 17 00:00:00 2001 From: FrankLeeeee Date: Wed, 11 Jan 2023 10:34:47 +0800 Subject: [PATCH 6/7] polish workflow --- .github/workflows/auto_example_check.yml | 2 -- 1 file changed, 2 deletions(-) diff --git a/.github/workflows/auto_example_check.yml b/.github/workflows/auto_example_check.yml index 7a9cedb4a4ba..8f4d53ff779b 100644 --- a/.github/workflows/auto_example_check.yml +++ b/.github/workflows/auto_example_check.yml @@ -42,12 +42,10 @@ jobs: if [ "$res" = "[]" ]; then echo "anyChanged=false" >> $GITHUB_OUTPUT echo "matrix={\"directory\": [\"null\"]}" >> $GITHUB_OUTPUT - echo "false" else dirs=$( IFS=',' ; echo "${res[*]}" ) echo "anyChanged=true" >> $GITHUB_OUTPUT echo "matrix={\"directory\":$(echo "$dirs")}" >> $GITHUB_OUTPUT - echo "true" fi # If no file is changed, it will prompt an error and shows the matrix do not have value. From e809993abec28a8b0f974692523ddcb332050a76 Mon Sep 17 00:00:00 2001 From: FrankLeeeee Date: Wed, 11 Jan 2023 10:35:50 +0800 Subject: [PATCH 7/7] polish workflow --- .github/workflows/auto_example_check.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/workflows/auto_example_check.yml b/.github/workflows/auto_example_check.yml index 8f4d53ff779b..f88b6858e003 100644 --- a/.github/workflows/auto_example_check.yml +++ b/.github/workflows/auto_example_check.yml @@ -41,7 +41,7 @@ jobs: if [ "$res" = "[]" ]; then echo "anyChanged=false" >> $GITHUB_OUTPUT - echo "matrix={\"directory\": [\"null\"]}" >> $GITHUB_OUTPUT + echo "matrix=null" >> $GITHUB_OUTPUT else dirs=$( IFS=',' ; echo "${res[*]}" ) echo "anyChanged=true" >> $GITHUB_OUTPUT