New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

train能否支持baichuan，qwen等其他模型 #1

Open

wzwz180 opened this issue Dec 27, 2023 · 1 comment

wzwz180 commented Dec 27, 2023

/DiverseEvol/configs/config_example_kc_dolly.yml
的设置中model_name_or_path: decapoda-research/llama-7b-hf 如果换成其他模型的路径会报各种各样的错
train是否只支持llama模型的训练？

Collaborator

danielwusg commented Jan 1, 2024

感谢你的问题！

理论上可以支持其他模型，只需要将tokenizer和model相关的代码 (utils.py) 替换成对应模型的方法即可；另外在config.yml中也需要根据模型内部模块指定参数 fsdp_transformer_layer_cls_to_wrap。

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment