Skip to content

Added finetuning support for BERT based models on IMDB dataset.#292

Merged
quic-mamta merged 16 commits intoquic:mainfrom
quic-meetkuma:bert_ft
Apr 17, 2025
Merged

Added finetuning support for BERT based models on IMDB dataset.#292
quic-mamta merged 16 commits intoquic:mainfrom
quic-meetkuma:bert_ft

Conversation

@quic-meetkuma
Copy link
Copy Markdown
Contributor

@quic-meetkuma quic-meetkuma commented Feb 28, 2025 •
edited
Loading

  • Added support for SequenceClassification task for BERT based models.
  • Added support for IMDB dataset.
  • Introduced a new flag --task_type with default value as "generation". For BERT training, it can be set to "seq_classification".

@vbaddi vbaddi marked this pull request as draft February 28, 2025 10:43
@quic-meetkuma quic-meetkuma marked this pull request as ready for review March 28, 2025 09:10
quic-meetkuma
quic-meetkuma commented Mar 28, 2025
View reviewed changes
Comment thread QEfficient/finetune/utils/train_utils.py Outdated
quic-meetkuma
quic-meetkuma commented Mar 28, 2025
View reviewed changes
Copy link
Copy Markdown
Contributor Author

@quic-meetkuma quic-meetkuma left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Address minor comments along with the other reviewers' comments.

@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 9 times, most recently from 1911bc4 to 016331c Compare April 9, 2025 11:16
quic-mamta
quic-mamta reviewed Apr 9, 2025
View reviewed changes
Comment thread QEfficient/finetune/utils/config_utils.py Outdated
@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 2 times, most recently from e4c5045 to 977f881 Compare April 14, 2025 05:42
quic-swatia
quic-swatia reviewed Apr 15, 2025
View reviewed changes
Comment thread QEfficient/finetune/utils/train_utils.py Outdated
quic-swatia
quic-swatia reviewed Apr 15, 2025
View reviewed changes
Comment thread QEfficient/finetune/utils/train_utils.py Outdated
quic-mamta
quic-mamta reviewed Apr 15, 2025
View reviewed changes
Comment thread QEfficient/finetune/utils/config_utils.py
@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 2 times, most recently from e682111 to ad3c27f Compare April 15, 2025 13:06
quic-mamta
quic-mamta requested changes Apr 16, 2025
View reviewed changes
Comment thread QEfficient/finetune/utils/train_utils.py Outdated
Comment thread QEfficient/finetune/utils/train_utils.py Outdated
Comment thread QEfficient/finetune/utils/train_utils.py Outdated
Comment thread QEfficient/finetune/utils/train_utils.py Outdated
@quic-mamta quic-mamta requested a review from vbaddi April 16, 2025 05:46
@quic-meetkuma quic-meetkuma force-pushed the bert_ft branch 3 times, most recently from 3174547 to 8d201d0 Compare April 17, 2025 05:51
@quic-meetkuma
Added finetuning support for BERT based models on IMDB dataset.
214e85c 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Added finetuning support for BERT based models on IMDB dataset.
390bce8 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Added finetuning support for BERT based models on IMDB dataset.
a9637b1 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Added torchmetrics as dependency and fixed loss computation for ddp c…
2a09de2 
…ase.

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Fixed atol value.
41766dd 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Fixed collate fn for bs>1. It will work fine for bs>1 for llama as we…
74ede95 
…ll on single device.

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Fixed train accuracy computation
1047357 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Fixed formatting issues.
4e21d3b 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Fixed few comments. Need to rebase first and then test
5c154aa 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Removed collate_fn for samsum dataset as DataCollatorForSeq2Seq works…
e97fed8 
… for all the use cases supported.

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Ruff formatted code.
b3c707e 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Removed samsum collate_fn as it is dead code.
8d983f2 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Refactored evaluation fun and renamed variable names to generic names.
bf0f083 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Fixed metric value for json logging
9ef972f 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Minor fixes to variable names
2713fdc 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-meetkuma
Addressed few offline comments.
b6d0ce9 
Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
@quic-swatia quic-swatia self-requested a review April 17, 2025 11:49
@quic-mamta quic-mamta merged commit b1fb57d into quic:main Apr 17, 2025
4 of 5 checks passed
eplatero97 pushed a commit to eplatero97/efficient-transformers that referenced this pull request Apr 29, 2025
@quic-meetkuma @eplatero97
Added finetuning support for BERT based models on IMDB dataset. (quic…
0d236c1 
…#292)

- Added support for SequenceClassification task for BERT based models.
- Added support for IMDB dataset.
- Introduced a new flag --task_type with default value as "generation".
For BERT training, it can be set to "seq_classification".

---------

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
eplatero97 pushed a commit to eplatero97/efficient-transformers that referenced this pull request Apr 29, 2025
@quic-meetkuma @eplatero97
Added finetuning support for BERT based models on IMDB dataset. (quic…
40022aa 
…#292)

- Added support for SequenceClassification task for BERT based models.
- Added support for IMDB dataset.
- Introduced a new flag --task_type with default value as "generation".
For BERT training, it can be set to "seq_classification".

---------

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
eplatero97 pushed a commit to eplatero97/efficient-transformers that referenced this pull request Apr 29, 2025
@quic-meetkuma
Added finetuning support for BERT based models on IMDB dataset. (quic…
59d5182 
…#292)

- Added support for SequenceClassification task for BERT based models.
- Added support for IMDB dataset.
- Introduced a new flag --task_type with default value as "generation".
For BERT training, it can be set to "seq_classification".

---------

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
Signed-off-by: eplatero <quic_eplatero@quicinc.com>
eplatero97 pushed a commit to eplatero97/efficient-transformers that referenced this pull request Apr 29, 2025
@quic-meetkuma @eplatero97
Added finetuning support for BERT based models on IMDB dataset. (quic…
5a5e954 
…#292)

- Added support for SequenceClassification task for BERT based models.
- Added support for IMDB dataset.
- Introduced a new flag --task_type with default value as "generation".
For BERT training, it can be set to "seq_classification".

---------

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>
Signed-off-by: eplatero <quic_eplatero@quicinc.com>
eplatero97 pushed a commit to eplatero97/efficient-transformers that referenced this pull request Apr 29, 2025
@quic-meetkuma @eplatero97
Added finetuning support for BERT based models on IMDB dataset. (quic…
3ed8405 
…#292)

- Added support for SequenceClassification task for BERT based models.
- Added support for IMDB dataset.
- Introduced a new flag --task_type with default value as "generation".
For BERT training, it can be set to "seq_classification".

---------

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>

Signed-off-by: eplatero <quic_eplatero@quicinc.com>
eplatero97 pushed a commit to eplatero97/efficient-transformers that referenced this pull request Apr 29, 2025
@quic-meetkuma @eplatero97
Added finetuning support for BERT based models on IMDB dataset. (quic…
86750e9 
…#292)

- Added support for SequenceClassification task for BERT based models.
- Added support for IMDB dataset.
- Introduced a new flag --task_type with default value as "generation".
For BERT training, it can be set to "seq_classification".

---------

Signed-off-by: Meet Patel <quic_meetkuma@quicinc.com>

Signed-off-by: eplatero <quic_eplatero@quicinc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@quic-mamta quic-mamta quic-mamta approved these changes

@quic-swatia quic-swatia quic-swatia approved these changes

@quic-rishinr quic-rishinr Awaiting requested review from quic-rishinr

@ochougul ochougul Awaiting requested review from ochougul

@vbaddi vbaddi Awaiting requested review from vbaddi

Assignees

@quic-meetkuma quic-meetkuma

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@quic-meetkuma @quic-mamta @quic-swatia