gnt-node modify --offline=no promotes node to master-candidate while master-capable flag is set to false

[rkojedzinszky]

We have cluster with a few nodes, 3 of them are marked as master-capable, the others are not. I suspect wrong behavior when changing offline flag on nodes:

node-1:~# gnt-node list -o name,master_capable,master_candidate
Node                               MasterCapable MasterC
node-1  Y             Y
node-2  Y             Y
node-3  Y             Y
node-4  N             N
node-11 N             N
node-12 N             N
node-13 N             N
node-14 N             N
node-15 N             N
node-1:~# gnt-node modify --offline=yes node-15 
Modified node node-15
 - offline -> True
node-1:~# gnt-node list -o name,master_capable,master_candidate
Node                               MasterCapable MasterC
node-1  Y             Y
node-2  Y             Y
node-3  Y             Y
node-4  N             N
node-11 N             N
node-12 N             N
node-13 N             N
node-14 N             N
node-15 N             N
node-1:~# gnt-node modify --offline=no --force node-15 
Mon Sep 12 09:25:43 2022  - INFO: Auto-promoting node to master candidate
Mon Sep 12 09:25:43 2022  - WARNING: Transitioning node from offline to online state without using re-add. Please make sure the node is healthy!
Modified node node-15
 - master_candidate -> True
 - offline -> False
node-1:~# gnt-node list -o name,master_capable,master_candidate
Node                               MasterCapable MasterC
node-1  Y             Y
node-2  Y             Y
node-3  Y             Y
node-4  N             N
node-11 N             N
node-12 N             N
node-13 N             N
node-14 N             N
node-15 N             Y
node-1:~# 

Documentation explicitly states:

Ganeti 2.4 introduces for this purpose a new master_capable flag, which (when unset) prevents nodes from being marked as master candidates, either manually or automatically.

https://docs.ganeti.org/docs/ganeti/3.0/html/admin.html#multi-site-model