[HOO][dynamo] Contextual hints for torch.compile backends #121639
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/121639
Note: Links to docs will display an error until the docs builds have been completed. ✅ No FailuresAs of commit 25012a4 with merge base d94db5f ( This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
Please seek CI approval before scheduling CIFlow labels |
|
The HOP feature seems useful in certain cases, but it's important to consider the potential uncertainties it could create and handle them carefully in different places. For example, when exporting, we need to serialize and deserialize the metadata, but hints can be of arbitrary type and may not be serializable. Additionally, we need to preserve the hint across transformations to avoid accidentally dropping it during the process, but the correct way of dealing with the hint might depend on the specific transformation and we can never be sure it's always correct because we don't know what kind of hint users can provide us. |
…D hints are required, autograd.Function should be used.
kbadz-pl
changed the title
janeyx99
added
the
triaged
|
Hello @ydwu4 @tugsbayasgalan - any update on the review or additional feedback? Thanks. |
Idea:
Described in ticket attached to this PR.
Design proposition:
Implemented hinted_context higher-order-op that allows to specify function, arguments and a hint.
The hint currently is a JSON string that is going to be propagated into all ops within specified function. Such hints can be nested to provide even more structure to the operations. In that case, JSON objects are going to be automatically merged and presented as a string, again.
Hints can be specified in two manners:
Provided test shows various usages, including nested annotations for both cases above. Each test runs on dummy backend that just prints the hint metadata from the nodes:
Fixes #126393
cc @voznesenskym @penguinwu @EikanWang @jgong5 @Guobing-Chen @XiaobingSuper @zhuhaozhe @blzheng @wenzhe-nrv @jiayisunx @chenyang78 @kadeng @chauhang