Relationship between branch.protected attribute and protected_branches api?

[nickbroon]

Sometimes, but not always, when I create a branch and then protect it using the protected_branches api the branch.protected attribute will still be false. This seems inconsistent at best and can lead to problems with other code that checks the branch.protected attribute.

For example I'll see the branch protected attribute is false

$ gitlab --output json project-branch get --project-id group/project --name foo | jq
{
  "name": "foo",
  "commit": {
    "id": "9b6de136b5f0158c60844f85286a593cb70fb364",
    "short_id": "9b6de13",
    "created_at": "2023-11-23T15:56:19.000+00:00",
    "parent_ids": [
      "05c8c94ed407499279b2a552e7ee9bb03e859b7b"
    ],
    "title": "First commit to new branch",
    "message": "First commit to new branch",
    "author_name": "Nick Brown",
    "author_email": "nick@foo.com",
    "authored_date": "2023-11-23T15:56:19.000+00:00",
    "committer_name": "Nick Brown",
    "committer_email": "nick@foo.com",
    "committed_date": "2023-11-23T15:56:19.000+00:00",
    "trailers": {},
    "web_url": "https://gitlab.com/group/project/-/commit/9b6de136b5f0158c60844f85286a593cb70fb364"
  },
  "merged": false,
  "protected": false,
  "developers_can_push": false,
  "developers_can_merge": false,
  "can_push": true,
  "default": false,
  "web_url": "https://gitlab.com/group/project/-/tree/foo"
}

while there is definitely protect branch state:

$ gitlab --output json project-protected-branch get --project-id group/project --name foo | jq
{
  "id": 85715646,
  "name": "foo",
  "push_access_levels": [
    {
      "id": 100709939,
      "access_level": 0,
      "access_level_description": "No one",
      "deploy_key_id": null,
      "user_id": null,
      "group_id": null
    }
  ],
  "merge_access_levels": [
    {
      "id": 91311949,
      "access_level": 40,
      "access_level_description": "Maintainers",
      "user_id": null,
      "group_id": null
    }
  ],
  "allow_force_push": false,
  "unprotect_access_levels": [
    {
      "id": 40016310,
      "access_level": 40,
      "access_level_description": "Maintainers",
      "user_id": null,
      "group_id": null
    }
  ],
  "code_owner_approval_required": false,
  "inherited": false
}

I create the branch and protected branch state like this

        file.save(
            start_branch=project.default_branch,
            branch=f"{branch_name}",
            commit_message="First commit to new branch",
        )

        # A delay before using the protectedbranches API on the new branch
        # as while the 'protectedbranches' config is updated the 'branch.protected'
        # attributes is never set. Even if the branches.get() returns the branch.
        # Possibly some race in the GitLab backend?
        # But even with a 10 second sleep the problems is still present! :-(
        sleep(10)
        project.refresh()
        new_branch = project.branches.get(f"{branch_name}")

        protected_branch = project.protectedbranches.create(
            {
                "name": new_branch.name,
                "merge_access_level": gitlab.const.AccessLevel.MAINTAINER,
                "push_access_level": gitlab.const.AccessLevel.NO_ACCESS,
                "unprotect_access_level": gitlab.const.AccessLevel.MAINTAINER,
                "allow_force_push": False,
            }
        )

[nejch]

Hmm @nickbroon I'm not 100% but we're also seeing a lot of flakiness in our functional tests when it comes to files/commits, branches, and especially merge requests. It does sometimes take a bit of time for GitLab to return a state we really want, maybe in this case there's a lot of moving parts involved between GitLab/sidekiqs/Gitaly etc.

So I'd say that you might need to loop until a timeout waiting for the right condition sometimes. Have you tried that?

[nickbroon]

I guess my only option is call protectedbranches.create, poll the branch.protected attribute for a while and if doesn't become true then delete the protectedbranches and create it again, poll and repeat until the attribute changes. hmm... I suspect a bug in gitlab backend somewhere.

[nejch]

@nickbroon yes, I'd definitely say it's a GitLab issue or a server performance issue, there's nothing in our library that would affect the timing like that I think. If anything, the client waits longer in case it hits a 429 response.

One more idea. Protected branches are just strings or wildcard patterns, it has nothing to with specific branches technically. So maybe GitLab has some scheduled background jobs to sync the existing branches against the pattern you create and then returns the real state after processing them, and depending on this, you get a match or not.

So I'd definitely try to rely more on the protected branches API for the real status of a branch name or pattern.

[nickbroon]

Thanks for suggestion @nejch. I think should be able to adjust my code to just ignore the unreliable branch.protected attribute and just set/get the project.protectedbranches everywhere. I'll open a issue with GitLab to document my intermittent but frequent problem with branch.protected never updating in some cases.

[nickbroon]

I've created this issue: https://gitlab.com/gitlab-org/gitlab/-/issues/432856

[nejch]

Thanks a lot @nickbroon! Sometimes it's good to provide examples scripted with curl, otherwise they might assume it's an issue with our library, but let's see, I think the description is already clear on what's happening 👍