In etcd version 3.4.13, how to restore the cluster from learner

[cuiHL]

etcd has 1 leader node, 2 follower nodes, and 1 learner node. When the leader node and follower node hang up, how to use the learner data to restore the cluster? When I use learner data to rebuild a new cluster, the error is as follows

The command to rebuild the cluster is as follows

etcd version：34.13

[ahrtr]

raft will raise panic "panic: removed all voters" when there isn't any voters, so it's expected behaviour.

Usually you should restore a cluster using one of the voting members' data. But if somehow all voting members' data are lost, and you only have a learner's data, then you have two choice,

1. Using command etcdutl snapshot restore ... to restore the cluster, refer to https://etcd.io/docs/v3.5/op-guide/recovery/#restoring-a-cluster

2. If you want to use flag --force-new-cluster to bootstrap a new one-member cluster, then you have to do some extra work programatically: you need to promote the learner offline.

   a). Figure out the learner ID

$ bbolt keys ./infra4.etcd/member/snap/db members
8211f1d0f64f3269
91bc3c398fb3c146
c0d999219b333896
fd422379fda50e48

$ bbolt get ./infra4.etcd/member/snap/db members 8211f1d0f64f3269
{"id":9372538179322589801,"peerURLs":["http://127.0.0.1:12380"],"name":"infra1"}

$ bbolt get ./infra4.etcd/member/snap/db members c0d999219b333896
{"id":13896306494845171862,"peerURLs":["http://127.0.0.1:42380"],"isLearner":true}

b). Write a simple program to promote the learner offline, refer to below,

etcd/server/etcdserver/api/membership/cluster.go

Lines 522 to 524 in 7a3ca53

	c.members[id].RaftAttributes.IsLearner = false
	c.updateMembershipMetric(id, true)
	c.be.MustSaveMemberToBackend(c.members[id])