Upgrade transformers to 4.48.0 for llama2#24302

Merged

xadupre merged 46 commits intomainfrom

syang/transformers-upgrade-llama2

Apr 18, 2025

Contributor

selenayang888 commented Apr 3, 2025 •

edited

Loading

Description

Upgrade Transformers to 4.48.0 for llama2, this version deprecated the old format of past_key_value, the current format is DynamicCache. So, we need to add patches to dynamo exporter in llama2.

Thanks to @xadupre who made the changes to add the patches to dynamo exporter, and implements patches to transformers 4.48.0 which don't export and convert dynamic_axes into dynamic shapes.

xadupre and others added 28 commits

March 28, 2025 15:01


          first draft to migrate to newer version of transformers


          add patches

31e82a9


          Merge branch 'main' of https://github.com/microsoft/onnxruntime into …

299f116

…llama2


          fix import

cdec2d0


          fix build and import

827d3bd


          build

18b649e


          fix lint

0e77ed4


          lint

4633a3e


          lint

b12287a


          rename

1b926cb


          lint

6646e61


          lint

a14b8b3


          remove args.dynamo

9f3a816


          fix issues

0c88e42


          copy inputs

8b60535


          fix shape

741285b


          fix validation

f8490a5


          Merge branch 'main' of https://github.com/microsoft/onnxruntime into …

dbe202c

…llama2


          add use_dynamo_export

ca43041


          lint

19d4dfb


          Merge branch 'llama2' of https://github.com/xadupre/onnxruntime into …

49fb806

…xadupre/llama


          fix requirements

8d3b0ba


          fix requitmeents

835b76e


          fix dynamic shapes

a0a8c21

2.6

f61c27b


          remove duplicated section

902c6af


          lint

e3188ad


          Merge remote-tracking branch 'origin/main' into syang/transformers-up…

88e16ca

…grade-llama2

github-advanced-security bot found potential problems

View reviewed changes

onnxruntime/python/tools/transformers/models/llama/llama_inputs.py Fixed Show fixed Hide fixed

onnxruntime/python/tools/transformers/models/llama/llama_parity.py Dismissed Show dismissed Hide dismissed

onnxruntime/python/tools/transformers/models/torch_export_patches/patches/patch_torch.py Dismissed Show dismissed Hide dismissed

onnxruntime/python/tools/transformers/models/torch_export_patches/patches/patch_torch.py Dismissed Show dismissed Hide dismissed

onnxruntime/python/tools/transformers/models/torch_export_patches/patches/patch_torch.py Dismissed Show dismissed Hide dismissed

onnxruntime/python/tools/transformers/models/torch_export_patches/patches/patch_transformers.py Dismissed Show dismissed Hide dismissed


          Merge remote-tracking branch 'origin/main' into syang/transformers-up…

e841c0b

…grade-llama2

github-actions bot reviewed

View reviewed changes

Contributor

github-actions bot left a comment

You can commit the suggested changes from lintrunner.

onnxruntime/python/tools/transformers/models/llama/llama_inputs.py Show resolved Hide resolved

onnxruntime/python/tools/transformers/models/llama/llama_parity.py Show resolved Hide resolved

onnxruntime/python/tools/transformers/models/llama/llama_parity.py Show resolved Hide resolved

github-advanced-security bot found potential problems

View reviewed changes

onnxruntime/python/tools/transformers/models/llama/llama_parity.py Fixed Show fixed Hide fixed


          adjust the format to resolve the comment

3e90875

github-actions bot reviewed

View reviewed changes

Contributor

github-actions bot left a comment

You can commit the suggested changes from lintrunner.

onnxruntime/python/tools/transformers/models/llama/llama_inputs.py Show resolved Hide resolved

onnxruntime/python/tools/transformers/models/llama/llama_parity.py Show resolved Hide resolved

onnxruntime/python/tools/transformers/models/llama/llama_parity.py Show resolved Hide resolved

xadupre and others added 5 commits

April 17, 2025 10:07


          Update onnxruntime/python/tools/transformers/models/llama/llama_parit…

a42dc0c

…y.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>


          Update onnxruntime/python/tools/transformers/models/llama/llama_parit…

307f53e

…y.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>


          Update onnxruntime/python/tools/transformers/models/llama/llama_input…

24cf39e

…s.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>


          add optree in the requirements

5fdc9d3


          implement string_type

2378aff

github-actions bot reviewed

View reviewed changes

Contributor

github-actions bot left a comment

You can commit the suggested changes from lintrunner.

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Show resolved Hide resolved

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Show resolved Hide resolved

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Show resolved Hide resolved

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Outdated Show resolved Hide resolved

xadupre and others added 4 commits

April 17, 2025 16:51


          Update onnxruntime/python/tools/transformers/models/torch_export_patc…

25caa82

…hes/__init__.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>


          Update onnxruntime/python/tools/transformers/models/torch_export_patc…

f2dd801

…hes/__init__.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>


          Update onnxruntime/python/tools/transformers/models/torch_export_patc…

09e25e0

…hes/__init__.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>


          Update onnxruntime/python/tools/transformers/models/torch_export_patc…

a26d857

…hes/__init__.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-actions bot reviewed

View reviewed changes

Contributor

github-actions bot left a comment

You can commit the suggested changes from lintrunner.

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Outdated Show resolved Hide resolved


          Update onnxruntime/python/tools/transformers/models/torch_export_patc…

4069a15

…hes/__init__.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

github-advanced-security bot found potential problems

View reviewed changes

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Fixed Show fixed Hide fixed

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Fixed Show fixed Hide fixed

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Fixed Show fixed Hide fixed

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Fixed Show fixed Hide fixed

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Fixed Show fixed Hide fixed

xadupre added 2 commits

April 17, 2025 17:50


          add missing import

c35b065


          add missing import

0aa3e16

github-actions bot reviewed

View reviewed changes

Contributor

github-actions bot left a comment

You can commit the suggested changes from lintrunner.

onnxruntime/python/tools/transformers/models/torch_export_patches/__init__.py Show resolved Hide resolved


          Update onnxruntime/python/tools/transformers/models/torch_export_patc…

ad386d1

…hes/__init__.py

Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

kunal-vaishnavi approved these changes

View reviewed changes

selenayang888 closed this

selenayang888 reopened this

tianleiwu approved these changes

View reviewed changes

tianleiwu reviewed

View reviewed changes

onnxruntime/python/tools/transformers/models/torch_export_patches/onnx_export_errors.py Show resolved Hide resolved

xadupre merged commit c19a496 into main

85 of 89 checks passed

xadupre deleted the syang/transformers-upgrade-llama2 branch

April 18, 2025 07:32

ashrit-ms pushed a commit that referenced this pull request


          Upgrade transformers to 4.48.0 for llama2 (#24302)

621dc80

### Description
Upgrade Transformers to 4.48.0 for llama2, this version deprecated the
old format of past_key_value, the current format is DynamicCache. So, we
need to add patches to dynamo exporter in llama2.

Thanks to @xadupre who made the changes to add the patches to dynamo
exporter, and implements patches to transformers 4.48.0 which don't
export and convert dynamic_axes into dynamic shapes.

---------

Co-authored-by: xadupre <xadupre@microsoft.com>
Co-authored-by: Xavier Dupré <xadupre@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>

intbf pushed a commit to intbf/onnxruntime that referenced this pull request


          Upgrade transformers to 4.48.0 for llama2 (microsoft#24302)

### Description
Upgrade Transformers to 4.48.0 for llama2, this version deprecated the
old format of past_key_value, the current format is DynamicCache. So, we
need to add patches to dynamo exporter in llama2.

Thanks to @xadupre who made the changes to add the patches to dynamo
exporter, and implements patches to transformers 4.48.0 which don't
export and convert dynamic_axes into dynamic shapes.

---------

Co-authored-by: xadupre <xadupre@microsoft.com>
Co-authored-by: Xavier Dupré <xadupre@users.noreply.github.com>
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
Signed-off-by: bfilipek <bartlomiej.filipek@intel.com>

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet