...
This section contains more detailed information on managing terms. Terms are used to distinguish votes from different election cycles, and also to help servers detect when they are out of date with respect to the rest of the cluster. In general, if a candidate or leader finds itself out of date it immediately passivates, under the assumption that someone else will take over leadership.
- Each server stores two term identifiers:
- lastReign: the identifier of the highest-numbered term for which the server received a message from the leader.
- currentTerm: identifier of the current term. This is often the same as lastReign, but may be greater than lastReign if an election is underway. CurrentTerm indicates the lowest-numbered term in which this server will accept a leader.
- CurrentTerm and lastReign must be stored persistently on disk, so they can survive restarts of the server. Is this necessary? Suppose all servers go down simultaneously; is it okay if the terms reset to 0 when they come back up?
- When a server starts, it uses the saved value to initialize the current term; if there is no saved value then the current term starts at 0.
- Every message from server to server contains the term of the sender, plus an indication whether the sender is a candidate or leader (passive servers never issue requests). This value (call it senderTerm) is used to update lastReign and currentTerm and to detect out-of-data servers:
- senderTerm < currentTerm: reject the message with an indication that the sender is out of date; the rejection also includes currentTerm. When the sender receives the rejection it updates its currentTerm to match the one in the response, then passivates. This makes sense for leaders because it means their term of leadership is over. This also make sense for candidates because it means there is already a new election cycle with other active candidates; there is no need for the sender to participate.
- senderTerm == currentTerm: if the sender is a leader, then set lastReign to senderTerm.
- senderTerm > currentTerm: set currentTerm to senderTerm. If the message is from a leader, then also set lastReign to senderTerm. If the recipient is currently a leader or candidate, then it passivates.
- When a server switches from passive to candidate, it increments currentTerm to force a new election cyclea term number called currentTerm. This indicates the most recent term that has been used by this server or received in a message from another server.
- When a server starts, currentTerm is set to 0.
- Every message from server to server contains the term of the sender, plus an indication whether the sender is a candidate or leader (passive servers never issue requests). This value (call it senderTerm) is used to update currentTerm and to detect out-of-data servers:
- senderTerm < currentTerm: reject the message with an indication that the sender is out of date; the rejection also includes currentTerm. When the sender receives the rejection it updates its currentTerm to match the one in the response, then passivates. This makes sense for leaders because it means their term of leadership is over. This also make sense for candidates because it means there is already a new election cycle with other active candidates; there is no need for the sender to participate.
- senderTerm == currentTerm: if the sender is a leader, then set lastReign to senderTerm.
- senderTerm > currentTerm: set currentTerm to senderTerm. If the recipient is currently a leader or candidate, then it passivates.
- When a server switches from passive to candidate, it increments currentTerm to force a new election cycle.
- I do not believe that currentTerm needs to be saved on disk. When a server restarts, it sets currentTerm to 0. Servers start out in passive state; most likely the leader will contact them (which will update currentTerm) before the new server times out and becomes a candidate. If all servers crash simultaneously, it should be OK for the term numbers to restart at 1. If all servers but one crash and restart, and for some reason the remaining server is disconnected from the others, it's possible that the new servers will choose a leader with term 1, without contacting the remaining server. At some point they will eventually contact it, which will cause the currentTerms to update. The existing leader will passivate, and a new election cycle will eventually occur; at this point everyone will be caught up to the term number of the server that didn't crash.
State for each server
- currentTerm: number of the most recent term seen by this server.
- lastReign: number of the most recent term in which this server accepted a message from a leader.
- Vote: the server id and log length of the candidate who has received this server's vote for currentTerm (if any).
- Server id of this server.
- Log entries that have been accepted by the server.
- Id of the most recent log entry that has been accepted by all servers.
- Time of receipt of the last request from the leader.
- Cluster map: id and location of each server in the cluster, whether dead or alive.
...