The MASQuant code is difficult to use

[Nkniexin]

Thank you for proposing such an insightful work! However, when I tried to run the MASQuant code, I encountered several issues:

1. The conda environment is very difficult to configure. It would be helpful if the exact versions of the dependency packages could be clearly specified.

2. The codebase appears somewhat messy, and there seems to be a lot of code that is unrelated to multimodal models.

3. In the provided examples, I did not find any logic related to the use of LoRA, which seems inconsistent with the description in the paper.

4. Regarding the actual inference speed, it seems that the CUDA kernel implementation has not been provided.

[tiger-of-shawn]

Hi @Nkniexin ,Thank you for your interest and for raising these insightful questions! Your feedback is instrumental in helping us improve the project. Here are our responses to your points:

Environment Setup: You are right. We will upload a requirements.txt file soon to help everyone align their Conda environment.

Codebase Structure: Regarding the codebase, it is currently a large collection that houses our latest advancements in model pruning, quantization, and fine-tuning, with MASQuant being a key component. We designed it this way primarily to make the process of open-sourcing our research efforts easier and faster.

Use of LoRA: The logic for LoRA is detailed in our README.md file, specifically under the section "Cross-Modal Compensation Calibration and Evaluation". That section explains how to use LoRA for cross-modal compensation, calibration, and evaluation.

CUDA Kernel: We have good news regarding the kernel. We are finalizing the internal disclosure process and plan to push it to the repository soon. Once available, you will be able to reproduce the inference acceleration results mentioned in the paper.

Thank you again for your engagement and valuable suggestions! We will keep you updated on our progress in this issue.