arslantu/realign
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity

init🎉:

Browse Source
This commit is contained in:
arslantu
2024-03-09 10:55:34 +08:00
commit bc182d09e0
30 changed files with 3022 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

4
train/configs/fsdp/internlm_fsdp_config.json Normal file
View File

@ -0,0 +1,4 @@
{
"fsdp_transformer_layer_cls_to_wrap": ["InternLMDecoderLayer"],
"limit_all_gathers": true
}

4
train/configs/fsdp/llama2_fsdp_config.json Normal file
View File

@ -0,0 +1,4 @@
{
"fsdp_transformer_layer_cls_to_wrap": ["LlamaDecoderLayer"],
"limit_all_gathers": true
}

4
train/configs/fsdp/qwen_fsdp_config.json Normal file
View File

@ -0,0 +1,4 @@
{
"fsdp_transformer_layer_cls_to_wrap": ["QWenBlock"],
"limit_all_gathers": true
}