-
Notifications
You must be signed in to change notification settings - Fork 1.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Out-of-memory for default config. #409
Comments
I read report 1.1 and it does not state that only 80G of memory is required for training. Where did you see that? |
Honestly, I saw this response. Did I misunderstand something? |
This issue is stale because it has been open for 7 days with no activity. |
I found the problem! When I wanna use the pre-trained weight via huggingface, it will load the config file: https://huggingface.co/hpcai-tech/OpenSora-STDiT-v2-stage3/blob/main/config.json where, the "enable_flash_attn": false, is forbidden! |
This issue is stale because it has been open for 7 days with no activity. |
I am gonna close this issue since it appears to have been resolved by the question owner. |
Many thanks for open-sourcing this great project.
Currently, I meet the out-of-memory error when training.
I use the default training config in stage3.py and I have 2 A100 80G.
However, it raises the error, but in report 1.1, it says the default config is for 80G memory usage.
Currently, when I use 480p with 48 frames, it takes around 73GB.
The text was updated successfully, but these errors were encountered: