[LLM] Add Yuan model #8654
[LLM] Add Yuan model #8654
Conversation
| Thanks for your contribution! |
| Lint问题可以参考link进行修复 |
| 我看lint的日志中引起black、isort、copyright_checker failed的文件已经被修改了，请问我还需要修改吗？或者这个错误具体指什么？我没看到引起错误的源文件？ |
具体是格式错误问题，PR中的文件需要满足要求格式。可以本地使用
辛苦拉一下最近代码即可，最新commit已经修复。 |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@ ## develop #8654 +/- ## =========================================== - Coverage 55.74% 55.42% -0.32% =========================================== Files 623 626 +3 Lines 97456 98057 +601 =========================================== + Hits 54323 54351 +28 - Misses 43133 43706 +573 ☔ View full report in Codecov by Sentry. |
| 请问目前的三个failed应该如何修改？ |
| # See the License for the specific language governing permissions and | ||
| # limitations under the License. | ||
| | ||
| """ Yuan model tools""" |
There was a problem hiding this comment.
这串代码建议封装成函数，避免出现2，24这样的魔鬼数字，明文模型路径等；
建议搞成参数传入；
There was a problem hiding this comment.
已修改。请问codecov中的warning应该如何修改？
There was a problem hiding this comment.
这个应该不影响代码合入，找commiter帮忙合入就可以了
.pre-commit-config.yaml Outdated
| repos: | ||
| # For Python files | ||
| - repo: https://github.com/psf/black.git | ||
| - repo: https://gitee.com/wygfzren/black.git |
There was a problem hiding this comment.
此文件请修改回原版本，会影响其他贡献者对代码进行格式化
paddlenlp/transformers/__init__.py Outdated
| from .deberta_v2.configuration import * | ||
| from .qwen2 import * | ||
| from .qwen2_moe import * | ||
| from .yuan.modeling import * |
There was a problem hiding this comment.
请使用 from .yuan import * 进行导入，并在yuan文件夹下的__init__.py中import 导入modeling和configuration
| from paddle.distributed import fleet | ||
| from paddle.nn import CrossEntropyLoss | ||
| | ||
| from paddlenlp.transformers.conversion_utils import ( |
There was a problem hiding this comment.
paddlenlp库内部函数和类，推荐使用相对路径import
| return q_embed, k_embed | ||
| | ||
| | ||
| class YuanPreTrainedModel(PretrainedModel): |
There was a problem hiding this comment.
此处需要修改模型名称为YuanPretrainedModel，否则在auto import时会产生报错。当前PaddleNLP导入规则为模型自定义名称（如Qwen2， Yuan） + 固定类型（如PretrainedModel, ForCausalLM）
| model_mappings.extend(layer_mappings) | ||
| | ||
| init_name_mappings(mappings=model_mappings) | ||
| # base-model prefix "LlamaModel" |
| if module._padding_idx is not None: | ||
| module.weight.data[module._padding_idx].zero_() | ||
| | ||
| def _set_gradient_checkpointing(self, module, value=False): |
There was a problem hiding this comment.
PaddleNLP中使用recompute来控制重计算，可参考llama相关重计算设置
| | ||
| hidden_states = inputs_embeds | ||
| | ||
| if self.gradient_checkpointing and self.training: |
There was a problem hiding this comment.
此处判断参数应为recompute，实现细节可参考llama代码
| 请问，test测试不通过应该如何修改？ |
非常感谢您的贡献。
|
1、已修改 |
5 participants
PR types
New features
PR changes
Models
Description
添加了源2.0的模型结构、配置等相关文件