Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ordered Flow #397

Open
fawadasaurus opened this issue Aug 8, 2023 · 3 comments
Open

Ordered Flow #397

fawadasaurus opened this issue Aug 8, 2023 · 3 comments
Labels
enhancement New feature or request

Comments

@fawadasaurus
Copy link
Contributor

fawadasaurus commented Aug 8, 2023

Is your feature request related to a problem? Please describe.
I need to run an operation after other operations have completed.

component:
  kind: wick/component/composite@v1
  with:
    - name: db_url
      type: string
  operations:
  - name: migration_1
    uses:
      - name: COMPLETE_MIGRATION
        operation: candle_cloud_db::complete_migration
    flow:
    - candle_cloud_db::create_migrations_table.output -> drop
    - candle_cloud_db::create_users_table.output -> drop
    - candle_cloud_db::create_oauth_sessions_table.output -> drop
    - candle_cloud_db::create_oidc_claims_table.output -> drop
    - candle_cloud_db::create_tenants_table.output -> drop
    - candle_cloud_db::create_tenant_users_table.output -> drop
    - candle_cloud_db::create_deployments_table.output -> drop
    - candle_cloud_db::create_secrets_table.output -> drop
    - <>.name -> COMPLETE_MIGRATION.name
    - COMPLETE_MIGRATION.output -> <>.output

In this operation, I need to run create_tenant_users_table only after create_tenants_table and create_users_table completes.

Propose a solution
Make a _ component input. _ will not provide any value but just act as a means of informing the operation that prior operations must execute before the named instance operation executes.

In the following example, CREATE_TENANT_USERS_TABLE will only run after both create_users_table and create_tenants_table has completed.

component:
  kind: wick/component/composite@v1
  with:
    - name: db_url
      type: string
  operations:
  - name: migration_1
    uses:
      - name: CREATE_TENANT_USERS_TABLE
        operation: candle_cloud_db::create_tenant_users_table
      - name: COMPLETE_MIGRATION
        operation: candle_cloud_db::complete_migration
    flow:
    - candle_cloud_db::create_migrations_table.output -> drop
    - candle_cloud_db::create_users_table.output -> CREATE_TENANT_USERS_TABLE._
    - candle_cloud_db::create_oauth_sessions_table.output -> drop
    - candle_cloud_db::create_oidc_claims_table.output -> drop
    - candle_cloud_db::create_tenants_table.output -> CREATE_TENANT_USERS_TABLE._
    - CREATE_TENANT_USERS_TABLE.output -> drop
    - candle_cloud_db::create_deployments_table.output -> drop
    - candle_cloud_db::create_secrets_table.output -> drop
    - <>.name -> COMPLETE_MIGRATION.name
    - COMPLETE_MIGRATION.output -> <>.output

Describe alternatives you've considered
None

Additional context
None

@fawadasaurus fawadasaurus added enhancement New feature or request triage To be triaged labels Aug 8, 2023
@jsoverson jsoverson removed the triage To be triaged label Aug 9, 2023
@jsoverson
Copy link
Contributor

There does need to be a synchronization gate or some way of denoting order for operations with no/no related input/output. This proposal is very specific to inputless operations with dropped outputs.

To satisfy the case above, you could introduce inputs to the dependent SQL operations (even if they're unused in the query) to hold them from executing and pipe the upstream outputs to the unused inputs to act as a gate. It's not perfect, but it should work while we iterate on a sync solution.

@jsoverson
Copy link
Contributor

What about an implicit output (_done) and an implicit input (_start) that is only sent if connected and is only sent within a "flow" context when an operation has fully completed?

This would give syntax akin to above, but wouldn't require dropping the upstream output.

@fawadasaurus
Copy link
Contributor Author

That sounds good to me.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants