Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature]: Revamp the Transform hyperparameter loading #810

Open
hwpang opened this issue Apr 18, 2024 · 0 comments
Open

[Feature]: Revamp the Transform hyperparameter loading #810

hwpang opened this issue Apr 18, 2024 · 0 comments
Labels
enhancement a new feature request todo add an item to the to-do list
Milestone

Comments

@hwpang
Copy link
Contributor

hwpang commented Apr 18, 2024

Notes
From @davidegraff 's comment from #726

I feel like I'm broadly happy with this PR and there are some minor kinks to work out now. That said, I think there's two main extensions to this PR for the future:

  1. revamp the hyperparameter loading. The current implementation works and correctly stores the transforms in their respective modules, but it reintroduces duplicated memory that more efficient module checkpointing #444 attempted to address. In short, by saving the modules themselves as hyperparameters, their respective state dicts are stored twice in the checkpoint file: once with the scaler object itself and and once inside the entire model's state dict. Again, not a big deal considering these are O(24)−O(29), i.e, less than 1 KB each. There are only about five of them in the entire model, so this is practically a non-issue. However, it wouldn't be bad for someone to try and address this to maintain thematic consistency with the rest of the code, but it may be more trouble than it's worth...
@hwpang hwpang added the todo add an item to the to-do list label Apr 18, 2024
@davidegraff davidegraff changed the title [V2][TODO]: Revamp the hyperparameter loading [Feature]: Revamp the hyperparameter loading Apr 18, 2024
@davidegraff davidegraff added the enhancement a new feature request label Apr 18, 2024
@davidegraff davidegraff changed the title [Feature]: Revamp the hyperparameter loading [Feature]: Revamp the Transform hyperparameter loading Apr 18, 2024
@KnathanM KnathanM added this to the v2.1.0 milestone Apr 19, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement a new feature request todo add an item to the to-do list
Projects
None yet
Development

No branches or pull requests

3 participants