Low-Latency RPC Scribe

Mogul: Two nics. If we configure a NIC for bandwidth or latency, may want two.
Can we really get a NIC that can be good at both at once?

Talk of UDP between slower than TCP due to more concentration on optimizing TCP.

Jeff keeps pushing the two NIC agenda - optimizing both dimensions on one NIC
really hard.

Keith Adams says syscalls very fast, but we're counting cache misses, so it
actually is worth considering.

Jeff mentioned RDMA.

Speculation about NIC hardware being improved with regards to latency.

• people seem to think certainly possible, but unsure whether it will
  get fast enough

Mendel & others on some NICs having complex microcode for multiple ring buffers,
virtualization, etc.

• big vendors, however (intel, broadcom), generally have functionality
  in silicon, not microcode – faster

Suggestions to use infiniband, or something other fancy, for prototyping.

Another suggestion to use our low layer networking and put cassandra, memcached,
etc on top to demonstrate low latency.

Question about retransmit rates expected

• nobody seems to have data

Jeff Mogul brings up incast problem

• overflowing buffers in intermediate switches bad, leads to secondary
  failures (synchronization possible)

Retransmit whole requests? If loss rare, why not?

• partial retransmits could reduce probability of incast problem

What happens when there's network congestion?

Flow control?

Jeff Dean: Use second core that shares same cache as the RPC handler core to
prefetch packet data into cache.

• no synch. needed

Keith Adams again: plug this stuff into memcached!