We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
CUDA_VISIBLE_DEVICES=0,1 llamafactory-cli train --stage sft --do_train True --model_name_or_path ./Qwen-1_8B-Chat --finetuning_type lora --quantization_bit 4 --template qwen --flash_attn auto --dataset_dir data --dataset self_coganition --cutoff_len 1024 --learning_rate 0.0002 --num_train_epochs 3.0 --max_samples 1000 --per_device_train_batch_size 2 --gradient_accumulation_steps 8 --lr_scheduler_type cosine --max_grad_norm 1.0 --logging_steps 5 --save_steps 100 --warmup_steps 0 --optim adamw_torch --packing False --report_to none --output_dir saves/Qwen-1.8B-Chat/lora/train_2024-05-13-18-20-06 --fp16 True --lora_rank 8 --lora_alpha 16 --lora_dropout 0.1 --lora_target c_attn --plot_loss True
训练正常结束,但加载模型之后,与之对话,出现乱码。
No response
The text was updated successfully, but these errors were encountered:
No branches or pull requests
Reminder
Reproduction
CUDA_VISIBLE_DEVICES=0,1 llamafactory-cli train --stage sft --do_train True --model_name_or_path ./Qwen-1_8B-Chat --finetuning_type lora --quantization_bit 4 --template qwen --flash_attn auto --dataset_dir data --dataset self_coganition --cutoff_len 1024 --learning_rate 0.0002 --num_train_epochs 3.0 --max_samples 1000 --per_device_train_batch_size 2 --gradient_accumulation_steps 8 --lr_scheduler_type cosine --max_grad_norm 1.0 --logging_steps 5 --save_steps 100 --warmup_steps 0 --optim adamw_torch --packing False --report_to none --output_dir saves/Qwen-1.8B-Chat/lora/train_2024-05-13-18-20-06 --fp16 True --lora_rank 8 --lora_alpha 16 --lora_dropout 0.1 --lora_target c_attn --plot_loss True
Expected behavior
训练正常结束,但加载模型之后,与之对话,出现乱码。
System Info
No response
Others
No response
The text was updated successfully, but these errors were encountered: