-
Notifications
You must be signed in to change notification settings - Fork 3k
【PaddleNLP No.1】 add pretrain.md #10506
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Thanks for your contribution! |
Codecov ReportAll modified and coverable lines are covered by tests ✅
❌ Your project status has failed because the head coverage (48.66%) is below the target coverage (58.00%). You can increase the head coverage or adjust the target coverage. Additional details and impacted files@@ Coverage Diff @@
## develop #10506 +/- ##
===========================================
- Coverage 48.66% 48.66% -0.01%
===========================================
Files 768 768
Lines 127103 127103
===========================================
- Hits 61860 61859 -1
- Misses 65243 65244 +1 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
@DrownFish19 pls review |
@DrownFish19 该目录下只有pretrain是rst格式,是否需要删除?(同步 |
@DrownFish19 pls review |
PR types
Others
PR changes
Docs
Description
1. 针对小白
大体框架与 .rst 一致,从新手小白用户的角度对文档进行润色。润色了一下文档的内容,适合小白入手大模型的预训练。
2. 克隆问题
由于少部分用户可能会碰到网络问题,加入了 gitee克隆和国内镜像 (个人测试过使用百度镜像和gitee均能克隆成功) 帮助小白能更快配好环境。 gitee具有滞后性
3. 模型支持列表
模型权重支持列表按照根目录的 README.md 进行更新。
4. 其他
添加 预训练成功后的打印信息 (以Qwen为例)。
修复部分失效链接:
use_flash_attention
需在 A100机器开启,支持的cuda版本最低11.8,不过最推荐的是官网最新cuda版本。5. docs/zh/llm/docs/pretrain.md
添加上面文档的相对路径
#9763