Default branch changes

[bryevdv]

I noticed the default branch has been moved to branch-3.1. While I agree that there should be a branch-3.1 made at this time, I am not sure I agree that it should become the default so soon, given that there will definitely be more releases on branch-3.0 in the near future. @mattpap I assume this was you? In the future can we agree to have anyone open a GH Discussion prior to changing the default branch, in order to give everyone else a heads up (at least a day, say) and an opportunity to comment?

[bryevdv]

@bokeh/core I would actually like to change the default branch back to branch-3.0 for now:

• @mattpap is probably the only person currently working on that branch heavily at the moment
• two new contributors just today had confusion about where to land their PR and I had to rest one of them
• most of my immediate work (and I expect @tcmetzger and maybe @ianthomas23 work) will need to land on branch-3.0
• so far I have done most of the back/forward porting and I prefer to forward port in most instances.

[mattpap]

There should be no confusion where to put new changes, right now it's branch-3.1. Bokeh 3.0 is done. The next or actually the current thing is bokeh 3.1. The only thing that should happen for 3.0 is any fixes that would otherwise prevent bokeh 3.0 adoption and usage, and possibly changes that would extend its life (e.g. security updates) for people who won't be able to upgrade to later versions in 3.x series for some hopefully unlikely reason.

A good example is PR #12538, which fixes a problem with a dependency, which otherwise would make adoption harder for some users. The point is not to develop bokeh 3.0 further. All development efforts should be focused on 3.1 and possibly beyond. There should be no plan for 3.0.z nor an extensive discussion what could be added there or not. If 3.0 is broken, then we fix it and we release it. Any other changes go into branch-3.1, where they can sit for awhile, when they can be better evaluated and tested and finally released in bokeh 3.1, without the risk of breaking bokeh 3.0. This of course applies to future releases, so when bokeh 3.1 is released, then it's done, and we move all development efforts to 3.2, and only patch 3.1 if needed, and so on.

I think there's quite a bit of confusion related to what minor and patch releases are about, probably from the old days of 0.y.z releases. Unless we're working on a new major release, which hopefully isn't going to be anytime soon, then all development efforts should be put into minor releases, leaving patch releases for fixing and extending life of previous minor releases (or major, but a major release is a minor 0 release in a new series of releases) (as I mentioned in an earlier paragraph). Putting development efforts into patch releases, means that we spread thin our limited development resources and also waterdown impact of new minor releases. Such diversions also reduce the possibility of high cadence of new minor releases, due to time wasted on porting and unnecessary testing.

I would like to switch to a higher cadence release schedule, with an actual schedule, like every 2 months, with 6 weeks of development time before an RC and then 2 weeks for testing/documentation/release. Anything that wasn't finished in time, is moved to the next cycle without regrets, because we know there will be another one soon.

A few quick notes:

@mattpap is probably the only person currently working on that branch heavily at the moment

No, everyone should contribute to branch-3.1. Sure, I'm the most affected person, but that's just the nature and scale of my contributions, however Ian will be almost exclusively contributing to 3.1, and perhaps others form Holoviz team, although it will be usually me doing the implementation on bokeh's side on behalf of others.

two new contributors just today had confusion about where to land their PR and I had to rest one of them

New contributors have almost no business contributing to 3.0. Unless they fix critical issues with 3.0, all their contributions should land on branch-3.1 and then we can decide whether any of those contributions would need to be backported (in most cases they shouldn't be).

most of my immediate work (and I expect @tcmetzger and maybe @ianthomas23 work) will need to land on branch-3.0

Ian has almost no business to contribute to 3.0 anymore. An exception is working on critical regressions related to WebGL and color mapping or color bars. I'm not sure about Timo. Improvements to documentation contents, specifically related to 3.0, any clarifications, etc., should indeed be added to 3.0. However, general development of documentation should be viewed similarly to development and done for 3.1.

so far I have done most of the back/forward porting and I prefer to forward port in most instances.

The mixture of back and forward ports, as has been done so far, is chaos in my view. Based on what I wrote so far, it should be clear that I expect porting between version should be minimized. Additionally forward ports should be a rare event, either a fix to a previous version was merged to that version's branch to e.g. expedite the release of a new patch version of bokeh, or there was something useful developed when patching an older version, which could be useful in the current branch.

[bryevdv]

I think there's quite a bit of confusion related to what minor and patch releases are about

@mattpap the fact that there are differences of opinion does not constitute confusion on other's part. You are issuing edicts for things that have not been decided or agreed upon.

Point releases are used by many projects for ongoing bugfixing, and not merely for hotfixes which seems to be your position. I have specific goals and reasons for those goals that more frequent point releases would support. This will have to be a discussion.

Putting development efforts into patch releases, means that we spread thin our limited development resources

Also worth reminding: everyone gets to choose themselves, where to devote their own personal time.

My own position was stated more than a year ago and is more or less unchanged: #11280 (comment) Which I asked for feedback/discussion of at the time (but did not receive). Summarizing:

We should keep three levels of releases:

X = huge, very big, definitely breaking, especially visuals, strong need to telegraph "big event"
Y = regular feature development pace, may break a few things, but usually mostly for projects, not users
Z = rapid schedule, minor bugs, docs, small, much less risk

And overriding reason forth is to have smaller, less risky release on a shorter, more predictable schedule. Offhand, I would say roughly every ~8 weeks. This is for a few reasons:

• Our ability to quickly issue hotfixes is great, but it is in fact predicated on all our machinery working. We need to exercise that machinery with much more regularity (more releases) to keep it reliable and usable on short notice.
• There is an infinite trough of small bugs. Timelier releases gets fixes into users hands more quickly
• Better for new contributors (see below)

So what are the options for that?

• switch to CalVer. Huge work and risk, I don't think anyone has the stomach for that
• Issue minor releases for features 2-3x a year with 2-3 point releases for small bugfixes in between (my preference)
• Issue smaller minor releases much more often, which will make messaging/differentiating "this is a feature release" more difficult

Do you have other ideas compatible with a faster release cadence? If not do you want to simply issue more, smaller minor releases (~6x/year)? It's not my first preference but I would consider it. But we need to develop discipline around regular releases, one way or the other. The lumpy unpredictable "sometimes release cycles take a year" schedule is not something I want to continue.

As for specific points:

New contributors have almost no business contributing to 3.0.

For instance I disagree with this entirely. New contributor's contributions are typically small and low risk, and they should go in a release ASAP in order to afford a positive feedback cycle to them (hopefully encouraging more contributions). Poinr releases are perfect for most new contributor contributions. We've also both had the experience of seeing a new issue come up and fixing it immediately because ten minutes can clear it off the tracker. Those are also great candidates of ongoing point bugfix releases. The reporter also gets a positive experience with the project when their fix goes in very soon.

The mixture of back and forward ports, as has been done so far, is chaos in my view.

I don't disagree, which is in fact why I want to try to focus of forward ports. Forward ports are unavoidable when there is a point release (the SRIs and changelog at least need to be brought forward). There will almost aways need to be some point release, even under your stricter preferences. Therefore I would like forward ports to be the practiced norm and back-ports be the exception, or ideally avoided altogether. It's also trivial to get the relevant list of commits to consider via git query when forward porting, there is much more room for confusion or error in the other direction.

cc @bokeh/core

[ianthomas23]

I wanted to chime in with some high-level comments. I'm not particular concerned with the details of the versioning and release plans, as a relative newcomer to Bokeh core I can work within whatever constraints are agreed.

But I would very much like information like this to be clearly documented and easily available. As someone who works a little on a number of different OSS projects, each of which has different protocols and rules, it is very helpful if I can easily and quickly look up the current process and apply it correctly. Otherwise I end up remembering or guessing what the protocol is and accidentally applying that of project X to project Y. I expect this will help all contributors, particular new ones which are the ones I am most interested in helping.

Secondly, in situations where there is disagreement we should resort to a vote. We are a democracy, and every core contributors vote carries the same weight regardless of longevity and perceived contribution and commitment to the project.

[bryevdv]

Technically these intentions were ratified in BEP 6

Regular Bokeh development proceeds according to a series of minor x.y.0 releases, nominally targeted to occur every 8-10 weeks. Infrequent major x.0.0 releases occur whenever it is necessary to introduce large or breaking changes.

However, we have obviously not been hewing to what was laid out there.

I am really OK with anything that gets us on to an ~8 week cadence.

[bryevdv]

I guess this has fizzled out, so I am just going to state my intentions, which are:

I intend to make a release approximately every two months. If the current branch-x.y is ready for imminent release, I will use that. Otherwise, I will pick an appropriate selection of small bug fixes to backport to branch-x.(y-1) for a point release.

That still allows for advancing the default branch "sooner" rather than "later" and also adheres to the timeline of BEP 6. Hopefully that is a compromise everyone can live with. Otherwise if there are objections, then we should simply put things to a vote.

[bryevdv]

Specifically, BEP 6 states this:

Patch releases do not represent continued development. Patch releases generally only occur in the following situations:

• When it an important bug fix cannot wait for the next minor release to occur.
• When some extenuating concern mandates that an older line of development receive some specific critical update.

I would resolve this disagreement by amending to:

Patch releases do not represent continued development. Patch releases generally only occur in the following situations:

• When it an important bug fix cannot wait for the next minor release to occur.
• When some extenuating concern mandates that an older line of development receive some specific critical update.

Additionally, if the current base branch is not in a releasable state near to the expected 8-10 week release cycle, a point release with backports of minor bugfixes may be made.

[philippjfr]

Thanks for writing this up. The amended version sounds good to me. Overall I'd be very happy if we could hit the 8-10 week release cadence for minor releases but am skeptical it's always possible so the amendment at least ensures bug fixes can land in a release at a regular cadence, even if they do not fix some critical vulnerability. I also do not think this will "dilute" minor releases since enhancements and features should be at the forefront for those.

[philippjfr]

Also, to clarify, given this amendment are you therefore happy to immediately bump the main branch to the next minor release. If so should this be ratified and documented as the official policy?

[bryevdv]

Yes, and think it is already more or less in BEP 6. Happy to strengthen with additional suggestions, though.