[QEff.finetuning] Hf config update by tchawada · Pull Request #795 · quic/efficient-transformers

tchawada · 2026-02-17T06:34:16Z

Added QAIC validation to config_manager.py and introduced a default value for prompt_func in the Alpaca dataset configuration. Updated same in documentation.

To run integrated_test for DDP use following command:
QAIC_VISIBLE_DEVICES=0,1 torchrun --nproc-per-node=2 -m pytest -q QEfficient/finetune/experimental/tests/test_integrated.py

carry over patch quic#693 Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>

Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com> Co-authored-by: vtirumal <vtirumal@qti.qualcomm.com>

Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>

Updating README, custom script for 2-layer instruction for Wan Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Added step wise instructions for MULTI NODE Finetuning. --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>

Add support for multi-node Distributed Data Parallel (DDP) training to the QEfficient finetuning pipeline. This enables scaling training across multiple nodes while keeping the existing single-node behavior unchanged. Commands for DDP across 2 servers: For the Master Addr or the Primary Machine, use node-rank as 0: QAIC_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=2 --nproc-per-node=4 --seed 0 --node-rank=0 --master_addr=<MASTER_NODE_IP> --master_port=8000 -m QEfficient.cloud.finetune --device qaic --enable_ddp --model_name "meta-llama/Llama-3.2-1B" --dataset alpaca_dataset --train_batch_size 1 --val_batch_size 1 --num_epochs 1 --max_train_step 200 --max_eval_step 50 For Node 1, use node-rank as 1: QAIC_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=2 --nproc-per-node=4 --seed 0 --node-rank=1 --master_addr=<MASTER_NODE_IP> --master_port=8000 -m QEfficient.cloud.finetune --device qaic --enable_ddp --model_name "meta-llama/Llama-3.2-1B" --dataset alpaca_dataset --train_batch_size 1 --val_batch_size 1 --num_epochs 1 --max_train_step 200 --max_eval_step 50 --------- Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

QEfficient should ignore providing `-mdp-load-partition-config` when `-mdp-dump-partition-config` is provided in compiler_options of compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Handled the edge case where num samples in a dataset are less than 20. Corrected the dataset link in grammar_dataset.py Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

Since CCL is deactivated by default, the value of CCL lists (ccl_prefill and ccl_decode) should be None by default. In infer.py script the value of these lists wasn't None and it caused the problem of ccl activation by default. In this PR we addressed this issue. --------- Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>

In this PR: 1) We have modified the code to support PP+DDP on multi-server setup 2) Added preprocessing file for grammar dataset 3) Modified the naming convention for output dir to include the node rank of the server --------- Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

Added default NPI file for Gemma3. 1. Eliminates the need to provide NPI file as an extra argument by user. NPI file added as default, no need to provide it explicitly in the example script --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Amit Raj <amitraj@qti.qualcomm.com> Co-authored-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Co-authored-by: Amit Raj <amitraj@qti.qualcomm.com>

…WQ and FP8 models. (quic#735) Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>

Removed OpenGVLab/InternVL2_5-1B and OpenGVLab/InternVL3_5-1B test due to a compiler issue to unblock the CI --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Updated Qeff version to mainline --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Reverts quic#741 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com> Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>

The decode‑only GPT‑OSS model was failing when executing subfunctions due to somehow considering a dynamic dim value during reduced‑sum calculation. This caused incorrect tensor reduction and resulted in compilation errors. The fix replaces the reduction logic with an einsum-based computation, ensuring stable and deterministic summation regardless of dimension shape. --------- Signed-off-by: asmigosw <asmigosw@qti.qualcomm.com>

- updated the random sampling gold text, ids for InternVL2_5-1B Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Support to skip export, compilation if qpc already exists - Updated Flux, wan configs, pipelines with qpc_path changes --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

The SW issue came with prompt + generation length > SW. Fix 1. Cache updated with HybridSlidingWindowCache in cache utils --------- Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Fix gemma3 to support cb with new SW code Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

This PR fixes subfunction-based export issues for the following models: 1. `bigcode/starcoder` 2. `ibm-granite/granite-20b-code-base-8k` 3. `ibm-granite/granite-20b-code-instruct-8k` 4. `Qwen3-30B-A3B-Instruct-2507` 5. `Mixtral-8x7B` In addition, it updates the Causal LM subfunction test file to make it more robust and resilient across models. --------- Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

Updated the mainline version to 1.22.0.dev0 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

qaic-exec is going to be deprecated. Updated the code to use new qaic-compile for compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

- skip subfn handling in export utils for diffusers, we handle this in export() of diffuser models --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Co-authored-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

quic-akuruvil · 2026-02-26T08:31:06Z

Added QAIC validation to config_manager.py and introduced a default value for prompt_func in the Alpaca dataset configuration. Updated same in documentation.

To run integrated_test for DDP use following command: QAIC_VISIBLE_DEVICES=20,21 torchrun --nproc-per-node=2 -m pytest -q /home/tchawada/optimizer_task/final/QEff_tanisha/QEfficient/finetune/experimental/tests/test_integrated.py

Please use Qefficient relative path here and remove the absolute local path. QEfficient/finetune/experimental/tests/test_integrated.py

quic-akuruvil · 2026-02-26T08:50:02Z

+    logger.warning(f"Training loss: {train_result.training_loss:.4f}")
+    logger.warning(f"Evaluation loss: {eval_result['eval_loss']:.4f}")
+    assert abs(train_result.training_loss - eval_result["eval_loss"]) < TRAIN_EVAL_EPOCH_LOSS_DIFF_THRESHOLD
+


Also we used to do, in the mainline tests, a reference match with the Reference Loss values(Reference being the previous stable SDK- added in reference_data.py). Please include that assertion too.

efficient-transformers/tests/finetune/test_finetune.py

Line 322 in a6f2dd4

def test_finetune_assert(

quic-meetkuma · 2026-03-02T06:42:53Z


 # Optimizer configuration
 optimizers:
-  optimizer_name: "adamw"
+  optimizer_name: "AdamW"


Are we using AdamW everywhere in code as well?

quic-meetkuma · 2026-03-02T06:44:19Z

+    SEQ_CLS = "AutoModelForSequenceClassification"
+    SEQ_2_SEQ_LM = "AutoModelForSeq2SeqLM"
+
+


L42 till end of file.
If these constants are only used in test cases then it will be better to move it to tests directory's constants.py file.

quic-meetkuma · 2026-03-02T06:45:55Z

        default=1,
        metadata={"help": "Number of workers for the DataLoader."},
    )
+    remove_samples_with_empty_columns: bool = field(


This is an additional argument. In our case we will need this always. Any reason to expose it to user?

In dataset file we are taking this from kwargs, so I aaded this, should I remove this?

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil

Add a sample config here. Add in comments description of the config, eg: sft_single_device_config.yaml in QEfficient/finetune/experimental/configs/sample_config.yaml

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil · 2026-03-04T11:32:33Z

Is this(infer) tested and verified? @tchawada

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

To run integrated_test for DDP use following command: QAIC_VISIBLE_DEVICES=0,1 torchrun --nproc-per-node=2 -m pytest -q QEfficient/finetune/experimental/tests/test_integrated.py --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>

To run integrated_test for DDP use following command: QAIC_VISIBLE_DEVICES=0,1 torchrun --nproc-per-node=2 -m pytest -q QEfficient/finetune/experimental/tests/test_integrated.py --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

To run integrated_test for DDP use following command: QAIC_VISIBLE_DEVICES=0,1 torchrun --nproc-per-node=2 -m pytest -q QEfficient/finetune/experimental/tests/test_integrated.py --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

To run integrated_test for DDP use following command: QAIC_VISIBLE_DEVICES=0,1 torchrun --nproc-per-node=2 -m pytest -q QEfficient/finetune/experimental/tests/test_integrated.py --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

To run integrated_test for DDP use following command: QAIC_VISIBLE_DEVICES=0,1 torchrun --nproc-per-node=2 -m pytest -q QEfficient/finetune/experimental/tests/test_integrated.py --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>

ochougul and others added 30 commits January 6, 2026 14:57

General disagg fix for prefill-only model (quic#698)

f81ef6e

carry over patch quic#693 Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>

Adding Vae Decoder in Wan (quic#688)

c57392d

Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com> Co-authored-by: vtirumal <vtirumal@qti.qualcomm.com>

Evaluating the values of CCL lists for different scenarios (quic#710)

75367b1

Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>

Updating 2-layer instruction for Wan (quic#715)

1e63710

Updating README, custom script for 2-layer instruction for Wan Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Updated finetune docs for MULTI NODE Training (quic#717)

1ef9935

Added step wise instructions for MULTI NODE Finetuning. --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>

Updating MDP partition config: prioritizing dump over load (quic#720)

7a39933

QEfficient should ignore providing `-mdp-load-partition-config` when `-mdp-dump-partition-config` is provided in compiler_options of compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

Updated docs (quic#722)

08bce2c

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

HOTFIX: changes in alpaca and grammar dataset utils (quic#724)

8b00c1b

Handled the edge case where num samples in a dataset are less than 20. Corrected the dataset link in grammar_dataset.py Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

HOTFIX : Added support for repeat kv heads aligned Bias scaling for A…

1ec3975

…WQ and FP8 models. (quic#735) Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>

Removed OpenGVLab/InternVL2_5-1B and OpenGVLab/InternVL3_5-1B (quic#736)

e61a1a3

Removed OpenGVLab/InternVL2_5-1B and OpenGVLab/InternVL3_5-1B test due to a compiler issue to unblock the CI --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Qeff versioning (quic#741)

47a0fec

Updated Qeff version to mainline --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Revert "Qeff versioning" (quic#746)

3a8e5e9

Reverts quic#741 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Fix for Qwen 2.5 VL with subfunction (quic#733)

0ffa4ea

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

Fixed torch patch for subfunction with VLMs (quic#750)

32f30c0

Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>

Added support of subfunction for VLMs (quic#699)

eb74758

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com> Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>

Updating pytest config for InternVL (quic#758)

5a129c7

- updated the random sampling gold text, ids for InternVL2_5-1B Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Wan support to skip compilation (quic#734)

b777e8b

Support to skip export, compilation if qpc already exists - Updated Flux, wan configs, pipelines with qpc_path changes --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Fixing SW issue in Gemma3 (quic#740)

75bf976

The SW issue came with prompt + generation length > SW. Fix 1. Cache updated with HybridSlidingWindowCache in cache utils --------- Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

Fix documentation of Multinode FT (quic#764)

3751f7e

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Adding support for gemma3 in continous batching script for CI (quic#763)

27ebe8e

Fix gemma3 to support cb with new SW code Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

Mainline version update (quic#752)

f64f703

Updated the mainline version to 1.22.0.dev0 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Updated compile from qaic-exec to qaic-compile (quic#703)

1a3e09c

qaic-exec is going to be deprecated. Updated the code to use new qaic-compile for compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

quic-akuruvil reviewed Feb 26, 2026

View reviewed changes

quic-swatia mentioned this pull request Feb 26, 2026

[QEff. Finetuning]: Adding PP support in HF trainer stack #813

Merged

quic-rishinr added the fine-tuning label Mar 2, 2026

quic-meetkuma requested changes Mar 2, 2026

View reviewed changes

quic-akuruvil reviewed Mar 3, 2026

View reviewed changes

Comment thread QEfficient/cloud/finetune_experimental.py

Updated finetune_experimental.py

d254a29

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil reviewed Mar 3, 2026

View reviewed changes

tchawada added 3 commits March 3, 2026 16:05

Updated finetune_experimental.py

cf80242

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

Merge branch 'ft_experimental' into final_hf

6762d1a

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

Updated hf_finetune.md and config.md

07b5e54

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil reviewed Mar 4, 2026

View reviewed changes

Comment thread docs/source/hf_finetune.md

Updating hf_finetune.md

f77df70

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil reviewed Mar 4, 2026

View reviewed changes

Comment thread docs/source/hf_finetune.md Outdated

Copy link
Copy Markdown

Contributor

quic-akuruvil Mar 4, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this(infer) tested and verified? @tchawada

tchawada added 2 commits March 4, 2026 14:58

Added output dir structure in config.md

dce11f8

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

Added output dir structure in config.md

65b1693

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil merged commit 0c49669 into quic:ft_experimental Mar 4, 2026
3 checks passed

		SEQ_CLS = "AutoModelForSequenceClassification"
		SEQ_2_SEQ_LM = "AutoModelForSeq2SeqLM"

Conversation

tchawada commented Feb 17, 2026 • edited by quic-akuruvil Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

quic-akuruvil commented Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

quic-akuruvil Feb 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

quic-akuruvil left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

15 participants

tchawada commented Feb 17, 2026 •

edited by quic-akuruvil

Loading

quic-akuruvil commented Feb 26, 2026 •

edited

Loading

quic-akuruvil Feb 26, 2026 •

edited

Loading