[QEff.Finetuning] Added support for SFTTrainer class along with tests

CONTRIBUTING.md

@@ -1,22 +1,160 @@
 ## Contributing to PROJECT
 
 Hi there!
-We’re thrilled that you’d like to contribute to this project.
+We're thrilled that you'd like to contribute to this project.
 Your help is essential for keeping this project great and for making it better.
 
-## Branching Strategy
 
-In general, contributors should develop on branches based off of `main` and pull requests should be made against `main`.
+## Submitting Your Contribution
 
-## Submitting a pull request
+Follow these steps to submit your example to the QEfficient repository:
 
 1. Please read our [code of conduct](CODE-OF-CONDUCT.md) and [license](LICENSE).
-1. Fork and clone the repository.
-1. Create a new branch based on `main`: `git checkout -b <my-branch-name> main`.
-1. Make your changes, add tests, and make sure the tests still pass.
-1. Commit your changes using the [DCO](http://developercertificate.org/). You can attest to the DCO by commiting with the **-s** or **--signoff** options or manually adding the "Signed-off-by".
-1. Push to your fork and submit a pull request from your branch to `main`.
-1. Pat yourself on the back and wait for your pull request to be reviewed.
+
+### 1. Fork and Clone the Repository
+
+First, fork the repository to your GitHub account, then clone your fork:
+
+```bash
+# Fork the repository on GitHub (click the "Fork" button)
+# Then clone your fork
+git clone git@github.com:YOUR_USERNAME/efficient-transformers.git
+cd efficient-transformers
+
+# Add upstream remote to keep your fork in sync
+git remote add upstream git@github.com:quic/efficient-transformers.git
+```
+
+### 2. Create a Feature Branch
+
+Create a descriptive branch for your changes:
+
+```bash
+# Update your main branch
+git checkout main
+git pull upstream main
+
+# Create a new branch
+git checkout -b <branch-name>
+```
+
+### 3. Make Your Changes
+
+When making changes to the codebase:
+
+- **Follow Existing Design Patterns**
+  - Review similar implementations before creating new code
+  - Maintain consistency with the project's architecture and coding style
+  - Reuse existing utilities and base classes where applicable
+
+- **Onboarding New Models**
+  - For adding new model support, refer to the comprehensive guide: `examples/onboarding_guide/causallm/`
+  - Follow the step-by-step process with code examples provided
+
+- **Testing is Mandatory**
+  - Add tests for all new features in the appropriate `tests/` subdirectory
+  - Run tests locally before pushing: `pytest tests/path/to/your/test.py -v`
+  - For model additions, verify all 4 pipeline stages (PyTorch HF → KV → ORT → AI 100) and make sure tokens are matching with refernce PyTorch HF 
+
+- **Documentation**
+  - **For New Features/Flags:**
+    - Document usage in `docs/source/<appropriate-page>` with feature description and usage examples
+    - Ensure documentation is clear enough for others to understand and use the feature
+  - **For New Models:**
+    - Test with basic inference scripts in the `examples/` folder
+    - If specific changes are needed, create a dedicated example file
+    - Update `docs/source/validate.md` with the model's HuggingFace card name and relevant details
+
+
+- **Code Quality Checks**
+  - Pre-commit hooks, DCO sign-off, and CI checks are covered in the following steps
+  - Ensure you complete steps 4-8 before finalizing your PR
+
+### 4. Run Pre-commit Checks
+
+Before committing, ensure your code passes all quality checks:
+
+```bash
+# Install pre-commit and ruff if not already installed
+pip install pre-commit
+pip install ruff
+
+# Run pre-commit on your changed files
+pre-commit run --files path/to/your/file1.py path/to/your/file2.py 
+
+# Run Ruff check 
+ruff check
+```
+
+**Important:** If pre-commit reports any failures:
+- Some issues will be auto-fixed (formatting, trailing whitespace, etc.)
+- For issues that aren't auto-fixed, manually correct them
+- Re-run `pre-commit run --files <files>` or `ruff check` until all checks pass
+
+### 5. Commit with Sign-off (DCO)
+
+All commits must be signed off to comply with the Developer Certificate of Origin (DCO):
+
+```bash
+# Stage your changes
+git add examples/your_domain/your_example.py
+git add examples/your_domain/README.md
+
+# Commit with sign-off
+git commit -s --author "Your Name <your.email@example.com>" -m "Add [model-name] support
+
+- Implements inference for [model-name]
+- Includes documentation and usage examples
+- Tested with [specific configurations]"
+```
+
+**Commit Message Guidelines:**
+- Use a clear, descriptive title 
+- Add a blank line, then detailed description if needed
+- Always include the `-s` flag for DCO sign-off
+
+### 6. Push to Your Fork
+
+Push your branch to your forked repository:
+
+```bash
+git push origin <branch-name>
+```
+
+### 7. Create a Pull Request
+
+1. Go to your fork on GitHub
+2. Click "Compare & pull request" for your branch
+3. Fill out the PR template with:
+   - **Title:** Clear, descriptive title (e.g., "Add Llama-3.2-Vision Support" or "Fix memory leak in KV cache")
+   - **Description:** 
+     - What changes were made and why
+     - What problem it solves or feature it adds
+     - Any special considerations or breaking changes
+     - Links to relevant documentation, issues, or model cards (if applicable)
+   - **Testing:** Describe how you tested your changes
+
+### 8. Ensure CI Checks Pass
+
+After creating the PR, verify that all automated checks pass:
+
+- ✅ **DCO Check:** Ensures all commits are signed off
+- ✅ **Lint Check:** Code style and formatting validation
+- ✅ **Tests:** Automated test suite (if applicable)
+
+If any checks fail:
+1. Review the error messages in the PR
+2. Make necessary fixes in your local branch
+3. Commit and push the fixes (with sign-off)
+4. The PR will automatically update and re-run checks
+
+### 9. Address Review Feedback
+
+Maintainers will review your PR and may request changes:
+- Make requested changes in your local branch
+- Commit with sign-off and push to update the PR
+- Respond to comments to facilitate discussion
+
 
 Here are a few things you can do that will increase the likelihood of your pull request to be accepted:
 

QEfficient/__init__.py

@@ -6,23 +6,64 @@
 # -----------------------------------------------------------------------------
 
 import os
-import warnings
-
-import QEfficient.utils.model_registery  # noqa: F401
-from QEfficient.utils import custom_format_warning
-from QEfficient.utils.logging_utils import logger
 
+# ----------------------------------------------------------------------------- #
 # For faster downloads via hf_transfer
 # This code is put above import statements as this needs to be executed before
 # hf_transfer is imported (will happen on line 15 via leading imports)
 os.environ["HF_HUB_ENABLE_HF_TRANSFER"] = "1"
+# DO NOT ADD ANY CODE ABOVE THIS LINE
+# Please contact maintainers if you must edit this file above this line.
+# ----------------------------------------------------------------------------- #
 # Placeholder for all non-transformer models registered in QEfficient
+import warnings  # noqa: I001
 
+import QEfficient.utils.model_registery  # noqa: F401
+from QEfficient.base import (
+    QEFFAutoModel,
+    QEFFAutoModelForCausalLM,
+    QEFFAutoModelForCTC,
+    QEFFAutoModelForImageTextToText,
+    QEFFAutoModelForSpeechSeq2Seq,
+    QEFFCommonLoader,
+)
+from QEfficient.compile.compile_helper import compile
+from QEfficient.diffusers.pipelines.flux.pipeline_flux import QEffFluxPipeline
+from QEfficient.diffusers.pipelines.wan.pipeline_wan import QEffWanPipeline
+from QEfficient.exporter.export_hf_to_cloud_ai_100 import qualcomm_efficient_converter
+from QEfficient.generation.text_generation_inference import cloud_ai_100_exec_kv
+from QEfficient.peft import QEffAutoPeftModelForCausalLM
+from QEfficient.transformers.transform import transform
+from QEfficient.utils import custom_format_warning
+from QEfficient.utils.logging_utils import logger
 
 # custom warning for the better logging experience
 warnings.formatwarning = custom_format_warning
 
 
+# Users can use QEfficient.export for exporting models to ONNX
+export = qualcomm_efficient_converter
+__all__ = [
+    "transform",
+    "export",
+    "compile",
+    "cloud_ai_100_exec_kv",
+    "QEFFAutoModel",
+    "QEFFAutoModelForCausalLM",
+    "QEFFAutoModelForCTC",
+    "QEffAutoPeftModelForCausalLM",
+    "QEFFAutoModelForImageTextToText",
+    "QEFFAutoModelForSpeechSeq2Seq",
+    "QEFFCommonLoader",
+    "QEffFluxPipeline",
+    "QEffWanPipeline",
+]
+
+
+# Conditionally import QAIC-related modules if the SDK is installed
+__version__ = "0.0.1.dev0"
+
+
 def check_qaic_sdk():
     """Check if QAIC SDK is installed"""
     try:
@@ -37,40 +78,5 @@ def check_qaic_sdk():
         return False
 
 
-# Conditionally import QAIC-related modules if the SDK is installed
-__version__ = "0.0.1.dev0"
-
-if check_qaic_sdk():
-    from QEfficient.base import (
-        QEFFAutoModel,
-        QEFFAutoModelForCausalLM,
-        QEFFAutoModelForCTC,
-        QEFFAutoModelForImageTextToText,
-        QEFFAutoModelForSpeechSeq2Seq,
-        QEFFCommonLoader,
-    )
-    from QEfficient.compile.compile_helper import compile
-    from QEfficient.exporter.export_hf_to_cloud_ai_100 import qualcomm_efficient_converter
-    from QEfficient.generation.text_generation_inference import cloud_ai_100_exec_kv
-    from QEfficient.peft import QEffAutoPeftModelForCausalLM
-    from QEfficient.transformers.transform import transform
-
-    # Users can use QEfficient.export for exporting models to ONNX
-    export = qualcomm_efficient_converter
-
-    __all__ = [
-        "transform",
-        "export",
-        "compile",
-        "cloud_ai_100_exec_kv",
-        "QEFFAutoModel",
-        "QEFFAutoModelForCausalLM",
-        "QEFFAutoModelForCTC",
-        "QEffAutoPeftModelForCausalLM",
-        "QEFFAutoModelForImageTextToText",
-        "QEFFAutoModelForSpeechSeq2Seq",
-        "QEFFCommonLoader",
-    ]
-
-else:
+if not check_qaic_sdk():
     logger.warning("QAIC SDK is not installed, eager mode features won't be available!")