Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We鈥檒l occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dynamo ignores frame using yield #126360

Open
xmfan opened this issue May 16, 2024 · 1 comment
Open

Dynamo ignores frame using yield #126360

xmfan opened this issue May 16, 2024 · 1 comment
Labels
module: dynamo oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module

Comments

@xmfan
Copy link
Member

xmfan commented May 16, 2024

馃悰 Describe the bug

OptimizeContext is wrapped, and frame evaluation callback seems to be set to convert_frame. But there's no dynamo logs and the entire frame seems to run in eager.

import torch
from torch._dynamo import config

def fn():
    model = torch.nn.Sequential(
        torch.nn.Linear(400, 4),
        torch.nn.Sigmoid(),
    )

    for _ in range(3):
        x = torch.randn([100, 400])

        result = model(x).sum()
        result.backward()
        yield model[0].weight.grad
        yield model[0].bias.grad
        model.zero_grad()

opt_fn = torch.compile(fn)
outs = opt_fn()
res = list(outs)

Versions

main

cc @ezyang @msaroufim @bdhirsh @anijain2305 @chauhang @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng

@xmfan xmfan added oncall: pt2 module: dynamo triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module labels May 16, 2024
@ezyang
Copy link
Contributor

ezyang commented May 17, 2024

If you wrap model in a user callable I think it will work

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module: dynamo oncall: pt2 triaged This issue has been looked at a team member, and triaged and prioritized into an appropriate module
Projects
None yet
Development

No branches or pull requests

2 participants