Release schedule

[smotornyuk]

Sorry for the grammar below and let's start.

It's likely that you have either searched or asked or thought about the CKAN release schedule before. And at some point, you realized, that there are no such things.
Patch releases happen when we have ~20-30 fixes merged into upstream or some serious security issue fixed. Usually, 3-5 months, but the main criterium - a bunch of fixes.
Minor releases happen... sometimes. Not too often.

you probably know it, but here are reasons, why we do things in that way

Reasons:

• There are not so many people involved in the process on a regular basis, so everyone has to put effort and spare some extra time for the release.
• Release process a bit complex: backport issues, update changelog, build images, test everything, update PyPI, update docker images, update deb-packages, build front-end assets...
• As releases do not happen often, we cannot afford any bug to be introduced as a part of the release.. yet we do it and customers have to wait till the next patch-release another 5 months before they can start using the new CKAN version and report even more issues
• ... feel free to add more

There is a way to change it:

Release schedule

• Minor CKAN release (v2.x, 2.y, 2.z) happens once in a 6 months
• 1 month before the release we create a corresponding branch and only fixes will get there. In that way, there will be no too immature features in the release.
• Customers(people who use CKAN) are notified(via documentation, probably. Or even a separate page on CKAN.org with an explanation of our release workflow) that the latest minor release may contain experimental features.
• In a half of year new minor release is out and the previous one is not considered as experimental anymore
• Every minor release supported for 18 months. I.e, just like now, 3 latest versions are supported

➕

• 🎆 NEW FEATURES. You love new features. You are contributing new features because you want and need them.
• There is no huge gap between versions. Right now there are a lot of changes in minor releases because these changes accumulated for a couple of years.

➖

• Removed features. We cannot just remove something from the codebase with this workflow. Any feature that is going to be removed, must be marked as deprecated and we can remove it only after 3 minor releases(18 months)

Yet, there is another problem(which we have even without these workflows). Bugs. What if the new release contains bugs?
In order to fix it, we have scheduled for the patch releases as well!

Patch-release schedule

• Patch release (v2.9.x, v2.9.y, v2.9.z) happens every two months.
• Patch release published for the last 3 minor releases.
• Whenever it's possible, PRs with the label Backport pending backported to all three minor releases. As minor releases are published more often, it's more likely that the majority of fixes(except for fixes of the new features) can be applied to all the previous releases

Is it even possible?

Yes, because every part of the release/patch-release process, except for backporting, can be done via GitHub actions.
As to backports, they:

• either applied at the moment, PR is merged into master, by the reviewer, who have merged it
• or by volunteer, if reviewer can't do it
• or by a dedicated person if we can afford it. You know, like the one who maintains demo.ckan.org or ckan.org.

Do we need it?

Yes!

• with smaller gaps between the versions it will be easier for customers to upgrade portal
• with a fixed schedule we can start removing deprecated code
• with frequent patch releases customers won't be afraid of bugs in the new minor releases
• customers are encouraged to use new CKAN versions because they can get new features and report bugs that will be fixed during the next two months and will be available for them in the nearest patch release. Even more, customers were encouraged to contribute by providing the fixes for the bugs discovered by themselves, Before it was easier to just report a bug and patch local CKAN because your contribution won't be available any soon. But now everyone can make CKAN better and see the result almost immediately.
• On your local projects, you probably have some sort of releases as well. They pass through the DEV, STAGING, something else. And then releases get to the PROD. If you discovered some bug on DEV, you can report it and there is a chance that the bug will be fixed before your code get to the PROD. Thus you get the best possible fix for your project.

[amercader]

Thanks for starting this discussion @smotornyuk . Here are some of my thoughts:

• I agree with the overall goal of having more frequent releases. Working towards the ideal of 2 minor releases (2.x) per year, and as many patch releases as we can. The smaller the releases the easier to manage backports and to upgrade for users
• I'm not sold on the experimental features. I think we should aim to merge stuff to master that is ready to be deployed with no major issues and any other bug or feature missing can be addressed via patch or minor releases as required
• Big +1 to automate everything. We should create actual issues for things we can automate so they can be picked up by us or the community. Having said that I think that might remove friction in terms of tasks to perform but not much in terms of time, as most of the time in a release is spent on backports, testing and changelog. Also deb packages, but these can very likely be automated.
• We can even explore automating backports. I know of some projects that use backport bots.
• In any case, I agree with what you said about trying to do the backports as part of the PR merging process.
• We should only support 2 minor versions (stable + previous one) or at lease only important security fixes on the 2nd previous before stable.

[frafra]

I also agree with both the proposal and your comment.
We need more minor releases, and more backports (which means we should also have more major releases too, otherwise it can become hard to backport in general).

[EricSoroos]

Re the changelog stuff, we're using Release Drafter in Pillow, and it's working reasonably well.

python-pillow/Pillow#5058

[amercader]

Re the changelog stuff, we're using Release Drafter in Pillow, and it's working reasonably well.

this looks great! it looks exactly as I ideally suggested but even better because we can edit Pull request titles before merging and we don't need to worry about adding further commits to the incoming branch

[btylerburton]

Hello, I don't see any other discussions open about the release schedule. Will you be putting out 2.10.2 this year, or are there plans to move directly to 3.x?

Working towards the ideal of 2 minor releases (2.x) per year,

[pdelboca]

Sadly we cannot commit to a fix date since CKAN is mostly a volunteer based work. What can I say is that we are pushing forward for a 2.11 release by the end of the year.

Is there anything in particular you are looking forward to be released in 2.10.2?

[btylerburton]

Thanks for the response @pdelboca. We are waiting on this fix in main to drop as this will allow us to address some core security vulnerabilities with Flask and Werkzeug...

#7463

[btylerburton]

@pdelboca did you have any insight into when this might drop?

[thegostev]

That'd be really nice to have a release cadence, whatever it is, say each 1-3 months so that everybody know that a waited fix or an update will see the world in a fixed time.

+1 for more automation. Dragan can contribute to it probably, after the current engagement on ElasticSearch. Let's discuss it!