Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Training LLaVA MPT from scratch. Few Questions #1505

Open
chanangad opened this issue May 16, 2024 · 0 comments
Open

Training LLaVA MPT from scratch. Few Questions #1505

chanangad opened this issue May 16, 2024 · 0 comments

Comments

@chanangad
Copy link

Discussion

I noticed that the training scripts for MPT model have been removed and the latest code doesn't support pre-training and training with MPT base model. Was there a reason for removing the training scripts?

I also wanted to know that which conversation template was finally used for pre-training and instruction-tuning if anyone remembers. Was it the 'plain' template for pre-training and 'mpt' for instruction-tuning or something else?

Also was mm_use_im_start_end and mm_use_im_patch_token used in both the stages?

It'd be great if someone can help me with this,

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant