Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Break the dependency between torch.nn and torch.distributed #126347

Open
fegin opened this issue May 15, 2024 · 0 comments
Open

Break the dependency between torch.nn and torch.distributed #126347

fegin opened this issue May 15, 2024 · 0 comments
Assignees
Labels
oncall: distributed Add this issue/PR to distributed oncall triage queue triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@fegin
Copy link
Contributor

fegin commented May 15, 2024

馃殌 The feature, motivation and pitch

We are seeing several import issues when compiling distributed modules. And the root cause is there are circular dependencies between torch.nn and torch.distributed. Some examples:

  1. torch.nn will import DDP, which will rely on many torch.distributed modules.
  2. torch.nn will import pytorch/torch/_jit_internal.py, which rely on torch.distributed.rpc.

While right now we work around these circular dependency issues, but these issues recurrently happen when compiling the distributed modules. We need to ensure lazy dependencies on torch.distributed for torch.nn

Alternatives

No response

Additional context

No response

cc @mrshenli @pritamdamania87 @zhaojuanmao @satgera @gqchen @aazzolini @osalpekar @jiayisuse @H-Huang @kwen2501 @awgu @penguinwu @XilunWu @wanchaol @fduwjj @wz337 @tianyu-l @wconstab @yf225 @chauhang @d4l3k

@fegin fegin added oncall: distributed Add this issue/PR to distributed oncall triage queue triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels May 15, 2024
@fegin fegin self-assigned this May 15, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
oncall: distributed Add this issue/PR to distributed oncall triage queue triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests

1 participant