Skip to content

Support of AutoModel#192

Merged
ochougul merged 17 commits intoquic:mainfrom
quic-amitraj:rag
Dec 18, 2024
Merged

Support of AutoModel#192
ochougul merged 17 commits intoquic:mainfrom
quic-amitraj:rag

Conversation

@quic-amitraj
Copy link
Copy Markdown
Contributor

@quic-amitraj quic-amitraj commented Dec 6, 2024 •
edited
Loading

Adding support of AutoModel

  1. Added support for Embedding models with different architecture.
  2. Upgraded ONNX opset version from 13 to 14.
  3. Added tests to compare and validate outputs across PyTorch, ONNX, and AI 100 runtime.

@quic-amitraj quic-amitraj self-assigned this Dec 6, 2024
@quic-amitraj quic-amitraj marked this pull request as draft December 6, 2024 08:55
@quic-amitraj quic-amitraj added enhancement New feature or request wip Work in progress model-enablement labels Dec 7, 2024
@quic-amitraj quic-amitraj marked this pull request as ready for review December 9, 2024 18:39
@quic-amitraj
Copy link
Copy Markdown
Contributor Author

quic-amitraj commented Dec 9, 2024

Currently, the following models and architectures are validated with this PR:

  1. nomic-ai/nomic-embed-text-v1.5 - NomicBertModel
  2. sentence-transformers/multi-qa-mpnet-base-cos-v1 - MPNetForMaskedLM
  3. BAAI/bge-reranker-v2-m3 - XLMRobertaForSequenceClassification
  4. BAAI/bge-small-en-v1.5 - BertModel

@quic-amitraj quic-amitraj changed the title Adding support of embedding Models Adding support of AutoModel Dec 9, 2024
@quic-amitraj quic-amitraj changed the title Adding support of AutoModel Support of AutoModel Dec 9, 2024
@quic-amitraj quic-amitraj removed the wip Work in progress label Dec 9, 2024
ochougul
ochougul requested changes Dec 10, 2024
View reviewed changes
Comment thread QEfficient/generation/text_generation_inference.py Outdated
Comment thread QEfficient/generation/text_generation_inference.py Outdated
Comment thread tests/transformers/models/test_causal_lm_models.py
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py
ochougul
ochougul reviewed Dec 10, 2024
View reviewed changes
Comment thread QEfficient/transformers/models/modeling_auto.py
quic-amitraj and others added 6 commits December 11, 2024 21:48
@quic-amitraj
Docker-driven tests with latest SDKs (quic#180)
0b80dac 
* Added Docker support to the Jenkins tests

Signed-off-by: amitraj <quic_amitraj@quicinc.com>

* Addressed comments

Signed-off-by: amitraj <quic_amitraj@quicinc.com>

* updated qaic tests time upper limit to 60 minutes

Signed-off-by: Onkar Chougule <quic_ochougul@quicinc.com>

---------
Signed-off-by: amitraj <quic_amitraj@quicinc.com>

* Added support for Embedding moodels
@quic-amitraj
Added support for embedding models
e83b29a 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Lint & Format
be59287 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Added batch_size
1284155 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Docstring added
3f95df7 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Fix-1
74ffc16 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Comments Addressed-1
2fb41ad 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
quic-rishinr
quic-rishinr requested changes Dec 13, 2024
View reviewed changes
Comment thread QEfficient/transformers/models/modeling_auto.py
@quic-amitraj
Comments addressed-2
262f45e 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Lint and formatted
ba0258b 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Comments addressed-3
ba66c75 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Fix-2
38a4186 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
anujgupt-github
anujgupt-github reviewed Dec 16, 2024
View reviewed changes
Copy link
Copy Markdown
Contributor

@anujgupt-github anujgupt-github left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please add updates to the README also in the PR.

Comment thread QEfficient/generation/text_generation_inference.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread pyproject.toml Outdated
@anujgupt-github
Copy link
Copy Markdown
Contributor

anujgupt-github commented Dec 16, 2024

the C++ execution path also needs to be adapted for embedding models.
@asmigosw

@quic-amitraj
Comments addressed-4
4401fd6 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Minor fix-1
206c81a 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
fix-major
0f1f8bb 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
ochougul
ochougul requested changes Dec 18, 2024
View reviewed changes
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread QEfficient/transformers/models/modeling_auto.py Outdated
Comment thread tests/transformers/models/test_embedding_models.py Outdated
Comment thread tests/transformers/models/test_embedding_models.py Outdated
@quic-amitraj
fix-minor-2
6c9de4b 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
fix-minor-3
88e0fe6 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@quic-amitraj
Update ONNX_EXPORT_OPSET to 13
157142a 
Signed-off-by: amitraj <quic_amitraj@quicinc.com>
@ochougul ochougul merged commit 26e472e into quic:main Dec 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@anujgupt-github anujgupt-github anujgupt-github left review comments

@ochougul ochougul ochougul requested changes

@quic-rishinr quic-rishinr quic-rishinr requested changes

Assignees

@quic-amitraj quic-amitraj

Labels

enhancement New feature or request

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@quic-amitraj @anujgupt-github @ochougul @quic-rishinr