Orchestration tasks #8985
Unanswered
lucioladen
asked this question in
Q&A
Replies: 0 comments
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Hi to all!
I've been working with Celery since a while, and whenever I want to orchestrate complex workflows and expose them as a single task to be invoked from outside, I can't find the way to do it but using the task replacement feature.
To illustrate what I'm saying, suppose I have a task that translates text using different translation methods by separating a long texts in batches, each to be run in parallel task, then when all are finished, joining results to build the final translated text. So, this configures a chord (a group, followed by a callback).
Suppose I want to reuse this task by putting this task inside another chord that performs another tasks too.
In the sake of being short and not adding more code, lets say I want to execute this three times, each with a different translator. This is not an actual code of my app, but I do have similar challenges. I couldn't find another way to solve it other than using
self.replace(workflow)
, but I'm almost sure this is not a best practice and far from being the right way to do it.How would you recommend to reuse "workflows inside workflows" and/or splitting complex workflows like these into smaller tasks without having a large spaghetti code?
Beta Was this translation helpful? Give feedback.
All reactions