[QEff.finetune]Test finetune by tchawada · Pull Request #826 · quic/efficient-transformers

tchawada · 2026-03-05T08:17:12Z

Modified test_finetune.py
Changed optimizer names

carry over patch quic#693 Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>

Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com> Co-authored-by: vtirumal <vtirumal@qti.qualcomm.com>

Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>

Updating README, custom script for 2-layer instruction for Wan Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Added step wise instructions for MULTI NODE Finetuning. --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>

Add support for multi-node Distributed Data Parallel (DDP) training to the QEfficient finetuning pipeline. This enables scaling training across multiple nodes while keeping the existing single-node behavior unchanged. Commands for DDP across 2 servers: For the Master Addr or the Primary Machine, use node-rank as 0: QAIC_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=2 --nproc-per-node=4 --seed 0 --node-rank=0 --master_addr=<MASTER_NODE_IP> --master_port=8000 -m QEfficient.cloud.finetune --device qaic --enable_ddp --model_name "meta-llama/Llama-3.2-1B" --dataset alpaca_dataset --train_batch_size 1 --val_batch_size 1 --num_epochs 1 --max_train_step 200 --max_eval_step 50 For Node 1, use node-rank as 1: QAIC_VISIBLE_DEVICES=0,1,2,3 torchrun --nnodes=2 --nproc-per-node=4 --seed 0 --node-rank=1 --master_addr=<MASTER_NODE_IP> --master_port=8000 -m QEfficient.cloud.finetune --device qaic --enable_ddp --model_name "meta-llama/Llama-3.2-1B" --dataset alpaca_dataset --train_batch_size 1 --val_batch_size 1 --num_epochs 1 --max_train_step 200 --max_eval_step 50 --------- Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

QEfficient should ignore providing `-mdp-load-partition-config` when `-mdp-dump-partition-config` is provided in compiler_options of compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Handled the edge case where num samples in a dataset are less than 20. Corrected the dataset link in grammar_dataset.py Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

Since CCL is deactivated by default, the value of CCL lists (ccl_prefill and ccl_decode) should be None by default. In infer.py script the value of these lists wasn't None and it caused the problem of ccl activation by default. In this PR we addressed this issue. --------- Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>

In this PR: 1) We have modified the code to support PP+DDP on multi-server setup 2) Added preprocessing file for grammar dataset 3) Modified the naming convention for output dir to include the node rank of the server --------- Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

Added default NPI file for Gemma3. 1. Eliminates the need to provide NPI file as an extra argument by user. NPI file added as default, no need to provide it explicitly in the example script --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Signed-off-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Amit Raj <amitraj@qti.qualcomm.com> Co-authored-by: Abukhoyer Shaik <abukhoye@qti.qualcomm.com> Co-authored-by: Amit Raj <amitraj@qti.qualcomm.com>

…WQ and FP8 models. (quic#735) Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>

Removed OpenGVLab/InternVL2_5-1B and OpenGVLab/InternVL3_5-1B test due to a compiler issue to unblock the CI --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Updated Qeff version to mainline --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Reverts quic#741 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com> Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>

The decode‑only GPT‑OSS model was failing when executing subfunctions due to somehow considering a dynamic dim value during reduced‑sum calculation. This caused incorrect tensor reduction and resulted in compilation errors. The fix replaces the reduction logic with an einsum-based computation, ensuring stable and deterministic summation regardless of dimension shape. --------- Signed-off-by: asmigosw <asmigosw@qti.qualcomm.com>

- updated the random sampling gold text, ids for InternVL2_5-1B Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Support to skip export, compilation if qpc already exists - Updated Flux, wan configs, pipelines with qpc_path changes --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

The SW issue came with prompt + generation length > SW. Fix 1. Cache updated with HybridSlidingWindowCache in cache utils --------- Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Fix gemma3 to support cb with new SW code Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

This PR fixes subfunction-based export issues for the following models: 1. `bigcode/starcoder` 2. `ibm-granite/granite-20b-code-base-8k` 3. `ibm-granite/granite-20b-code-instruct-8k` 4. `Qwen3-30B-A3B-Instruct-2507` 5. `Mixtral-8x7B` In addition, it updates the Causal LM subfunction test file to make it more robust and resilient across models. --------- Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

Updated the mainline version to 1.22.0.dev0 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

qaic-exec is going to be deprecated. Updated the code to use new qaic-compile for compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

- skip subfn handling in export utils for diffusers, we handle this in export() of diffuser models --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Co-authored-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

…etune

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil · 2026-03-05T11:40:38Z

@@ -21,9 +21,10 @@ model:
 # Dataset configuration
 dataset:
  dataset_type: "sft_dataset"
-  dataset_name: "yahma/alpaca-cleaned" # Dataset name from Hugging Face Hub
-  prompt_func: "QEfficient.finetune.experimental.preprocessing.alpaca_func:create_alpaca_prompt" # Function to create prompt from dataset fields


Keep another sft_single_device_alpaca_config.yaml also

quic-akuruvil · 2026-03-05T11:52:29Z


 ```

+### Step-by-Step Guide to run a fine-tuning job


why is this added again? Is it not already present?

Oh, during resolving conflicts it came twice, I remove it

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil · 2026-03-05T15:29:28Z

Looks Good!

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil · 2026-03-06T09:34:59Z

 **Single device using yaml file**
 ```bash
-python finetune_experimental.py configs/sft_single_device_config.yaml
+QAIC_VISIBLE_DEVICES=1 python QEfficient/cloud/finetune_experimental.py configs/sft_single_device_gsm8k_config.yaml


for configs also give full path starting from QEfficient

And use QAIC_visible_devices starting from 0 always

quic-akuruvil · 2026-03-06T09:36:54Z

 **Distributed (Using Accelerate)**
 ```bash
-accelerate launch --num_processes 4 finetune_experimental.py configs/sft_ddp_config.yaml
+QAIC_VISIBLE_DEVICES=1,2,3,4 accelerate launch --num_processes 4 -m QEfficient.cloud.finetune_experimental configs/sft_ddp_config.yaml


Start from 0

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

Modified test_finetune.py Changed optimizer names --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

Modified test_finetune.py Changed optimizer names --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

Modified test_finetune.py Changed optimizer names --------- Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com> Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com> Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

ochougul and others added 30 commits January 6, 2026 14:57

General disagg fix for prefill-only model (quic#698)

f81ef6e

carry over patch quic#693 Signed-off-by: Onkar Chougule <ochougul@qti.qualcomm.com>

Adding Vae Decoder in Wan (quic#688)

c57392d

Signed-off-by: Mohit Soni <mohisoni@qti.qualcomm.com> Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com> Co-authored-by: Mohit Soni <mohisoni@qti.qualcomm.com> Co-authored-by: vtirumal <vtirumal@qti.qualcomm.com>

Evaluating the values of CCL lists for different scenarios (quic#710)

75367b1

Signed-off-by: Vahid Janfaza <vjanfaza@qti.qualcomm.com>

Updating 2-layer instruction for Wan (quic#715)

1e63710

Updating README, custom script for 2-layer instruction for Wan Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Updated finetune docs for MULTI NODE Training (quic#717)

1ef9935

Added step wise instructions for MULTI NODE Finetuning. --------- Signed-off-by: Ann Kuruvilla <akuruvil@qti.qualcomm.com>

Updating MDP partition config: prioritizing dump over load (quic#720)

7a39933

QEfficient should ignore providing `-mdp-load-partition-config` when `-mdp-dump-partition-config` is provided in compiler_options of compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

Updated docs (quic#722)

08bce2c

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

HOTFIX: changes in alpaca and grammar dataset utils (quic#724)

8b00c1b

Handled the edge case where num samples in a dataset are less than 20. Corrected the dataset link in grammar_dataset.py Signed-off-by: Sharvari Medhe <smedhe@qti.qualcomm.com>

HOTFIX : Added support for repeat kv heads aligned Bias scaling for A…

1ec3975

…WQ and FP8 models. (quic#735) Signed-off-by: Dhiraj Kumar Sah <dhirajku@qti.qualcomm.com>

Removed OpenGVLab/InternVL2_5-1B and OpenGVLab/InternVL3_5-1B (quic#736)

e61a1a3

Removed OpenGVLab/InternVL2_5-1B and OpenGVLab/InternVL3_5-1B test due to a compiler issue to unblock the CI --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Qeff versioning (quic#741)

47a0fec

Updated Qeff version to mainline --------- Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Revert "Qeff versioning" (quic#746)

3a8e5e9

Reverts quic#741 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Fix for Qwen 2.5 VL with subfunction (quic#733)

0ffa4ea

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com>

Fixed torch patch for subfunction with VLMs (quic#750)

32f30c0

Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com>

Added support of subfunction for VLMs (quic#699)

eb74758

Signed-off-by: Abhishek Kumar Singh <sabhis@qti.qualcomm.com> Signed-off-by: abhishek-singh591 <sabhis@qti.qualcomm.com> Signed-off-by: Abhishek kumar singh <sabhis@qti.qualcomm.com>

Updating pytest config for InternVL (quic#758)

5a129c7

- updated the random sampling gold text, ids for InternVL2_5-1B Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Wan support to skip compilation (quic#734)

b777e8b

Support to skip export, compilation if qpc already exists - Updated Flux, wan configs, pipelines with qpc_path changes --------- Signed-off-by: vtirumal <vtirumal@qti.qualcomm.com>

Fixing SW issue in Gemma3 (quic#740)

75bf976

The SW issue came with prompt + generation length > SW. Fix 1. Cache updated with HybridSlidingWindowCache in cache utils --------- Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

Fix documentation of Multinode FT (quic#764)

3751f7e

Signed-off-by: Ann Kuruvilla <quic_akuruvil@quicinc.com>

Adding support for gemma3 in continous batching script for CI (quic#763)

27ebe8e

Fix gemma3 to support cb with new SW code Signed-off-by: Dipankar Sarkar <dipankar@qti.qualcomm.com>

Mainline version update (quic#752)

f64f703

Updated the mainline version to 1.22.0.dev0 Signed-off-by: Rishin Raj <rishinr@qti.qualcomm.com>

Updated compile from qaic-exec to qaic-compile (quic#703)

1a3e09c

qaic-exec is going to be deprecated. Updated the code to use new qaic-compile for compile API. --------- Signed-off-by: Asmita Goswami <asmigosw@qti.qualcomm.com>

tchawada added 3 commits March 4, 2026 14:58

Added output dir structure in config.md

dce11f8

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

Added output dir structure in config.md

65b1693

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

Adding test_finetune.py

5e506f6

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

tchawada requested review from ochougul, quic-amitraj, quic-hemagnih and quic-rishinr as code owners March 5, 2026 08:17

tchawada changed the title ~~Test finetune~~ [QEff.finetune]Test finetune Mar 5, 2026

tchawada added 3 commits March 5, 2026 09:36

Merge remote-tracking branch 'upstream/ft_experimental' into test_fin…

f541c74

…etune

Resolving lint error

9106bd5

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

changed dataset in config files

9e67a2d

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil reviewed Mar 5, 2026

View reviewed changes

Adding sft_single_device_alpaca_config.yaml

7fd1a90

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

tchawada added 2 commits March 6, 2026 05:09

Renamed a file sft_single_device_gsm8k_config.yaml

0d2483b

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

Solved Logger error

27008cc

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil reviewed Mar 6, 2026

View reviewed changes

updated hf_finetune.md

ab996f4

Signed-off-by: Tanisha Chawada <tchawada@qti.qualcomm.com>

quic-akuruvil merged commit d191e8a into quic:ft_experimental Mar 6, 2026
3 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[QEff.finetune]Test finetune#826

[QEff.finetune]Test finetune#826
quic-akuruvil merged 60 commits intoquic:ft_experimentalfrom
tchawada:test_finetune

tchawada commented Mar 5, 2026

Uh oh!

quic-akuruvil Mar 5, 2026

Uh oh!

tchawada Mar 5, 2026

Uh oh!

quic-akuruvil Mar 5, 2026

Uh oh!

tchawada Mar 5, 2026

Uh oh!

quic-akuruvil commented Mar 5, 2026 •

edited

Loading

Uh oh!

quic-akuruvil Mar 6, 2026

Uh oh!

quic-akuruvil Mar 6, 2026

Uh oh!

quic-akuruvil Mar 6, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

Conversation

tchawada commented Mar 5, 2026

Uh oh!

quic-akuruvil Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

tchawada Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

quic-akuruvil Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

tchawada Mar 5, 2026

Choose a reason for hiding this comment

Uh oh!

quic-akuruvil commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

quic-akuruvil Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

quic-akuruvil Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

quic-akuruvil Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

13 participants

quic-akuruvil commented Mar 5, 2026 •

edited

Loading