Goal: Utilize the CPU time while write requests are waiting for replication Rpcs to return to do useful work (ie, handling other requests).
Ideally, we would like to handle new requests only as long as replication requests have not yet finished, and then prioritize running existing requests to completion as soon as they become unblocked.
Idea 1: Polling
Each worker thread handling a write request will issue its replication Rpcs, and then constantly yield until the responses come in.
If we assume FIFO ordering on the ready queue, then each new write request would need to wait for every existing write request to poll and then yield at least once before it gets a chance to run.
Idea 2: Condition Variable Broadcast
Each worker thread will sleep on a common CV when waiting for a response to an Rpc, and when any response is received, there is a broadcast on that CV, and all worker threads wake up and check if it was for them. If it was not for them, they go back to waiting on the CV.
For every Rpc response received, every worker thread will waste work by waking up to check if the response was for them.
Idea 3: Custom Wait Queue