Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update policy to bump Task/Pipeline versions whenever a change is made #784

Open
bobcatfish opened this issue Jul 19, 2021 · 7 comments
Open

Update policy to bump Task/Pipeline versions whenever a change is made #784

bobcatfish opened this issue Jul 19, 2021 · 7 comments
Assignees
@jerop
Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness.

Comments

@bobcatfish
Copy link
Contributor

Expected Behavior

As a user of Tasks in the catalog, when I use Task (or Pipeline) and I specify a specific version X, I should be confident that the Task (or Pipeline) I use at time Y is the same as the version at time Y+1, etc.

That is: if anything that is user facing in a Task (or Pipeline) changes, the version of should be bumped.

Actual Behavior

We make changes to Tasks without bumping the version, for example:

Anyone grabbing the git-clone 0.1 or 0.4 Task BEFORE these changes would get something different AFTER these changes (and which version will be available in our OCI registry? I'm assuming the latest one, but not sure?)

My understanding of why these changes being made without bumping the version is b/c we view these as additive; however:

  1. without knowing exactly how users are using the Tasks/Pipelines, we can't know if these changes will break them (see steps to reproduce)
  2. with everything else we version (e.g. releases of Tekton Pipelines), we create a new version even if the changes are additive, or even in the case of bug fixes: we don't update v0.14.0 to contain a fix, we make a new release, v0.14.1, so that we can clearly communicate to our users that there has been a change

Steps to Reproduce the Problem

I want to demonstrate an example of how this can impact a user by describing how this impacted me recently.

I've been working on a custom task that runs a Pipeline as a TaskRun (tektoncd/experimental#770) and I created an example Pipeline which used git-clone 0.3.

  1. I installed git-clone with tkn hub install task git-clone (i didnt explicitly specify the version i needed but let's pretend i said 0.3 which is what i ended up working with)
  2. I created a Pipeline that used git-clone and used it to experiment
  3. I decided to create an end to end test that used the same git-clone - I wrote the TaskRun I expected (with the git-clone steps, params, etc. embedded in it) based on the git-clone task i had installed
  4. The last part of my test setup was to grab the git-clone 0.3 yaml from the catalog repo and add it to a testdata folder
  5. My tests failed! because the Task had changed since I grabbed it (the params -> env var change), my resulting TaskRun was now different

Additional Info

I'm wondering if one reason we're doing this is we've made it too expensive to bump a version - bumping a version means creating a whole new directory of READMEs, yamls, examples, etc. (not to mention it's hard to tell what's changed b/w versions).

If that's the case, it might be worth revisiting our approach. OR we could try something different, now that we're publishing to an OCI registry, like deleting the old version when we make a change.
@jerop
Copy link
Member

jerop commented Aug 2, 2021

/assign

@vdemeester
Copy link
Member

If that's the case, it might be worth revisiting our approach. OR we could try something different, now that we're publishing to an OCI registry, like deleting the old version when we make a change.

OCI bundle are not stable yet, so we can assume most of our user do not use them / have access to them. We can't rely on them completely until it's available for everyone.

@tekton-robot
Copy link

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale with a justification.
Stale issues rot after an additional 30d of inactivity and eventually close.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle stale

Send feedback to tektoncd/plumbing.

@tekton-robot tekton-robot added the lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. label Nov 1, 2021
@tekton-robot
Copy link

Stale issues rot after 30d of inactivity.
Mark the issue as fresh with /remove-lifecycle rotten with a justification.
Rotten issues close after an additional 30d of inactivity.
If this issue is safe to close now please do so with /close with a justification.
If this issue should be exempted, mark the issue as frozen with /lifecycle frozen with a justification.

/lifecycle rotten

Send feedback to tektoncd/plumbing.

@tekton-robot tekton-robot added lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. and removed lifecycle/stale Denotes an issue or PR has remained open with no activity and has become stale. labels Dec 1, 2021
@jerop
Copy link
Member

jerop commented Dec 16, 2021

/lifecycle frozen

we need to establish this policy and make it easy to make bumps - @vinamra28 and I are discussing this

draft: https://docs.google.com/document/d/1SGOkVrg0_s8ufbSTRl0Y4l_2zD_0QgUElEDZY0p8jSk/edit?usp=sharing

@tekton-robot tekton-robot added lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness. and removed lifecycle/rotten Denotes an issue or PR that has aged beyond stale and will be auto-closed. labels Dec 16, 2021
@waveywaves waveywaves mentioned this issue Dec 31, 2021
Merged
3 tasks
@ghost ghost mentioned this issue Jan 21, 2022
Merged
2 tasks
jerop added a commit to jerop/catalog that referenced this issue Jan 31, 2022
@jerop
Versioning Resources
fd96585 
Today, we don't always bump versions of resources when updating them.
The guideline has been to bump versions only when behavior changes,
but it's hard to figure out when the behavior has changed (a change
that could be trivial to one user could be meaningful to another).

Not bumping resource versions when changing them causes issues where
the resource definition becomes dependent on the time when it was applied
 by the user - which causes unexpected failures as described in
tektoncd#784.

This issue also came up as an issue where users cannot depend on the
Step indices because they can change:
tektoncd/community#572 (review).

In TEP-0003, we already proposed that a policy for versioning of resources:
https://github.com/tektoncd/community/blob/main/teps/0003-tekton-catalog-organization.md#versioning-resources

In Catalog Working Group on 01/13/2022, we revisted that policy and:
- agreed to follow the versioning policy
- make it easier to bump resources (see tektoncd/plumbing#994)

This change documents the versioning policy.

Fxes tektoncd#784
jerop added a commit to jerop/catalog that referenced this issue Jan 31, 2022
@jerop
Versioning Resources
3b9b684 
Today, we don't always bump versions of resources when updating them.
The guideline has been to bump versions only when behavior changes,
but it's hard to figure out when the behavior has changed (a change
that could be trivial to one user could be meaningful to another).

Not bumping resource versions when changing them causes issues where
the resource definition becomes dependent on the time when it was applied
 by the user - which causes unexpected failures as described in
tektoncd#784.

This issue also came up as an issue where users cannot depend on the
Step indices because they can change:
tektoncd/community#572 (review).

In TEP-0003, we already proposed that a policy for versioning of resources:
https://github.com/tektoncd/community/blob/main/teps/0003-tekton-catalog-organization.md#versioning-resources

In Catalog Working Group on 01/13/2022, we revisted that policy and:
- agreed to follow the versioning policy
- make it easier to bump resources (see tektoncd/plumbing#994)

This change documents the versioning policy in the contributions guide.

Fixes tektoncd#784
jerop added a commit to jerop/catalog that referenced this issue Jan 31, 2022
@jerop
Versioning Resources
61c5e39 
Today, we don't always bump versions of resources when updating them.
The guideline has been to bump versions only when behavior changes,
but it's hard to figure out when the behavior has changed (a change
that could be trivial to one user could be meaningful to another).

Not bumping resource versions when changing them causes issues where
the resource definition becomes dependent on the time when it was applied
 by the user - which causes unexpected failures as described in
tektoncd#784.

This issue also came up as an issue where users cannot depend on the
Step indices because they can change:
tektoncd/community#572 (review).

In TEP-0003, we already proposed that a policy for versioning of resources:
https://github.com/tektoncd/community/blob/main/teps/0003-tekton-catalog-organization.md#versioning-resources

In Catalog Working Group on 01/13/2022, we revisited that policy and:
- agreed to follow the versioning policy
- make it easier to bump resources (see tektoncd/plumbing#994)

This change documents the versioning policy in the contributions guide.

Fixes tektoncd#784
@jerop jerop mentioned this issue Jan 31, 2022
Closed
3 tasks
@jerop
Copy link
Member

jerop commented Feb 24, 2022

/lifecycle frozen

@jerop
Copy link
Member

jerop commented Jun 10, 2022

initial work addressing this is here: https://github.com/tektoncd/community/blob/main/teps/0110-decouple-catalog-organization-and-reference.md

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@jerop jerop

Labels
lifecycle/frozen Indicates that an issue or PR should not be auto-closed due to staleness.
Projects
Tekton Community Roadmap
Status: Todo
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Versioning Resources jerop/catalog
4 participants
@vdemeester @bobcatfish @jerop @tekton-robot