-
Notifications
You must be signed in to change notification settings - Fork 13.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Prevent start trigger initialization in scheduler #39585
base: main
Are you sure you want to change the base?
Prevent start trigger initialization in scheduler #39585
Conversation
d621388
to
657a1cf
Compare
657a1cf
to
926b3d3
Compare
maybe we should mark this experimental. trigger_kwargs being None vs {} does not seem like an explicit / obvious enough way to decide whether to start from trigger. why wouldn't we just look at the |
926b3d3
to
ec0a768
Compare
I'm ok with it. maybe wait for others' comment? if we're to mark it as experimental, where should I do so?
In the previous PR, @uranusjr and I discussed whether we could infer |
529ecc1
to
12fb69d
Compare
@@ -143,10 +143,12 @@ The ``self.defer`` call raises the ``TaskDeferred`` exception, so it can work an | |||
Triggering Deferral from Start |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TODO: address #38674 (comment)
445274d
to
dac10f5
Compare
4b0ac4a
to
5f99ce7
Compare
31fab0c
to
4be767c
Compare
…art_trigger_kwargs
…ss StartTriggerArgs
…her to start task execution from triggerer
4be767c
to
3dcace0
Compare
Why
During #38674, I introduced a logic that might run user code in the scheduler here. This PR intent to propose a new logic that could avoid this.
What
StartTriggerArgs
class which should containtrigger_cls
(or maybe it should be renamed astrigger_cls_path
?),tirgger_kwargs
,timeout
,next_method
which are what we need when we run an operator in deferrable mode.airflow.models.abstractoperator
might not be the best module to put this data class but might need some suggestionOperator authors will now need to set the
start_trigger_args
this one to start execution directly from triggerer.If the
trigger_kwargs
is set toNone
and not updated in__init__
then this task won't be deferred from starting. This is designed this way so that an operator can decide whether to start the execution from the triggerer. e.g.,^ Add meaningful description above
Read the Pull Request Guidelines for more information.
In case of fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
In case of a new dependency, check compliance with the ASF 3rd Party License Policy.
In case of backwards incompatible changes please leave a note in a newsfragment file, named
{pr_number}.significant.rst
or{issue_number}.significant.rst
, in newsfragments.