Skip to content

Latest commit

 

History

History
 
 

examples

Folders and files

NameName
Last commit message
Last commit date

parent directory

..

detxoify_lm

detxoify_lm

 
 

msdp

msdp

 
 

sc21

sc21

 
 

README.md

README.md

 
 

evaluate_retriever_nq.sh

evaluate_retriever_nq.sh

 
 

evaluate_zeroshot_gpt.sh

evaluate_zeroshot_gpt.sh

 
 

finetune_mnli_distributed.sh

finetune_mnli_distributed.sh

 
 

finetune_race_distributed.sh

finetune_race_distributed.sh

 
 

finetune_retriever_distributed.sh

finetune_retriever_distributed.sh

 
 

merge_mp_bert.sh

merge_mp_bert.sh

 
 

pretrain_bert.sh

pretrain_bert.sh

 
 

pretrain_bert_distributed.sh

pretrain_bert_distributed.sh

 
 

pretrain_bert_distributed_with_mp.sh

pretrain_bert_distributed_with_mp.sh

 
 

pretrain_gpt.sh

pretrain_gpt.sh

 
 

pretrain_gpt3_175B.sh

pretrain_gpt3_175B.sh

 
 

pretrain_gpt_distributed.sh

pretrain_gpt_distributed.sh

 
 

pretrain_gpt_distributed_with_mp.sh

pretrain_gpt_distributed_with_mp.sh

 
 

pretrain_ict.sh

pretrain_ict.sh

 
 

pretrain_t5.sh

pretrain_t5.sh

 
 

pretrain_t5_distributed.sh

pretrain_t5_distributed.sh

 
 

pretrain_t5_distributed_with_mp.sh

pretrain_t5_distributed_with_mp.sh

 
 

run_text_generation_server_345M.sh

run_text_generation_server_345M.sh

 
 

run_text_generation_server_345M_8_tensor_parallel.sh

run_text_generation_server_345M_8_tensor_parallel.sh

 
 

README.md

Original examples by NVIDIA/Megatron-LM

This folder includes examples from the original NVIDIA/Megatron-LM repo. All of them do NOT have DeepSpeed technologies integrations, and some of them may not work due to changes in this Megatron-DeepSpeed repo. Thus we recommend you to go to ../examples_deepspeed/ folder which includes examples that have DeepSpeed technologies integrated and are tested by DeepSpeed team.