-
Notifications
You must be signed in to change notification settings - Fork 693
[Features] support hugging face qwen3 dense and qwen2 model #3449
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
Thanks for your contribution! |
| @@ -0,0 +1,222 @@ | |||
| # Copyright (c) 2025 PaddlePaddle Authors. All Rights Reserved. | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这两个单测文件删了,用彬涵那个单测形式,你这里直接copyV0的整个文件改,有大量重复冗余代码,之后V0上新增单测,V1感知不到
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
| ) | ||
| param.copy_(loaded_weight, False) | ||
|
|
||
| def bias_loader(self, param, loaded_weight, loaded_shard_id: Optional[str] = None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个bias 可以和上面的weight 和一起吧,只是一个2维一个 1维
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已合并
| if self.nranks != 1: | ||
| hugging_face_format = self.fd_config.load_config.hugging_face_format | ||
| if hugging_face_format: | ||
| loaded_weight = loaded_weight.transpose([1, 0]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
bias是1维的吧
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
已删除这里判断
| if self.nranks > 1: | ||
| set_weight_attrs(self.linear.weight, {"output_dim": False}) | ||
|
|
||
| def weight_loader(self, param, loaded_weight, loaded_shard_id: Optional[str] = None): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这个是不是可以删了?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
| return fn | ||
|
|
||
|
|
||
| def slice_fn(weight_or_paramter, output_dim, start, end, step=1): |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
新增的函数需要对所有入参增加注解,比如 output_dim 比较难理解
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
done
| self.is_unified_ckpt = check_unified_ckpt(self.model) | ||
|
|
||
| self.override_name_from_config() | ||
| self.read_model_config() |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
这里model_format全部依赖自动判断了吗,如果某个模型config.json中没有torch_dtype,就无法被正确识别?也没法强制指定这个模型为torch模型?
Fastdeploy支持hugging face qwen2系列,qwen3 dense模型模型
使用示例:
目前仅在load_choices为default_v1时,才支持加载hugging face模型,我们根据模型config.json中是否有torch_dtype来判断是加载的hugging face模型还是paddle模型
服务方式:
离线方式: