Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Display task dependencies in the grid view #35935

Closed
1 of 2 tasks
RNHTTR opened this issue Nov 28, 2023 · 4 comments
Closed
1 of 2 tasks

Display task dependencies in the grid view #35935

RNHTTR opened this issue Nov 28, 2023 · 4 comments
Labels
area:UI Related to UI/UX. For Frontend Developers. kind:feature Feature Requests

Comments

@RNHTTR
Copy link
Collaborator

RNHTTR commented Nov 28, 2023

Description

Over the last couple weeks I've come across a rather tricky problem a few times. One DAG run gets "stuck" in the queued state, while subsequent DAG runs will be stuck running (screenshot below). One of these issues was caused by max_active_runs being met when a task instance from a previously run DAG was cleared, and one of the tasks had depends_on_past=True. This caused the DAG run to be stuck in queued in perpetuity until it was realized that the task that wasn't getting scheduled needed the failed task in the preceding DAG run to be re-run, which in turn causes the stuck running DAG runs to be stuck in running. which caused quite a bit of confusion and stress.

image

Ideally there would be a new status (or a refresh of the no_status status), but a much simpler solution to this problem might be simply surfacing the full task instance details in the grid view, or at least adding a task's dependencies to the task instance details in the grid view.

Use case/motivation

I don't have a solid understanding of what underlies the UI, but IMO it'd be best to surface the full task instance details view in the task instance details sub-view of the grid view.

If that's too technically challenging for now or if there are other issues (for example, performance issues), it'd suffice to at least include the task's dependencies in the task instance details sub-view of the grid view.

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct

@RNHTTR RNHTTR added kind:feature Feature Requests needs-triage label for new issues that we didn't triage yet area:UI Related to UI/UX. For Frontend Developers. and removed needs-triage label for new issues that we didn't triage yet labels Nov 28, 2023
@bbovenzi
Copy link
Contributor

As part of #29856 we should bring the dependency info to the Grid view's TI details. Its a big dump of metadata though so I'd like to try to figure out a UX that makes it easier for a user to find relevant details.

@bbovenzi
Copy link
Contributor

Perhaps whenever a task has any pending state we highlight the dependencies and otherwise it is collapsed by default.

@RNHTTR
Copy link
Collaborator Author

RNHTTR commented Jan 19, 2024

Perhaps whenever a task has any pending state we highlight the dependencies and otherwise it is collapsed by default.

What do you mean by "pending state"? Like scheduled & queued? I don't think that's sufficient, because the really tricky case IMO is when depends_on_past=True. The task will be in the "no status" state, and it's super confusing to figure out why a DAG is still running, but no tasks are moving to scheduled. This just "got" me again:

image

I was debugging a depends_on_past=True issue, and it took me a few minutes to realize what was going on when the task wasn't moving to scheduled, and I'm already aware of this issue. I think the solution should be to highlight dependencies if

  1. dependencies exist and
  2. the task is in one of no status, scheduled, or queued.

@RNHTTR
Copy link
Collaborator Author

RNHTTR commented May 17, 2024

Closed by #38449

@RNHTTR RNHTTR closed this as completed May 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:UI Related to UI/UX. For Frontend Developers. kind:feature Feature Requests
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants