...
- Summary - throughput drops to 50% under high load.
- The throughput of the system is measured here against increasing
load. The load is in terms of read operations on 100 byte objects. - We notice that the throughput of the system drops by a factor of 2
for high loads. This is observed even though we are nowhere near the
network limits at this point. The measured outgoing throughput is
390217 ops/sec or 39M bytes/sec or 310M bits/sec which is well under
the expected 32Gbps limit. - The red, blue and green lines were measured with 24 RX buffers and
8, 32 and 64 TX buffers respectively. - The violet line was measured with 48 RX buffers and 8 TX
buffers. Notice that adding buffers to the pool on the receive side
allows the trasmit side to see a higher throughput - I do not
understand the reasons for this. - A set of further measurements are taken during the same experiment
and plotted on different graphs to aid understanding.
...
Average number of buffers returned by pollCompletionQueue across different Buffer Pool sizes
Analysis
- Summary - 3 empty buffers become available at a time under load.
Number of receive buffers also affects pollCompletionQueue(). - This average does not include calls when no buffers were returned.
This is an average when non-zero buffers were returned by pollCompletionQueue(). - The red, blue and green lines were measured with 24 RX buffers and
8, 32 and 64 TX buffers respectively. - The violet line was measured with 48 RX buffers and 8 TX buffers.
- An interesting trend that appears to be independent of number of
buffers in the pool. There is a drop in the average at the same load
irrespective of buffer-pool. - Why does doubling the number of receive buffers affect the number of
empty transmit buffers returned ? Compare Red against Violet. - Look at the average number of buffers returned under high load.
It appears as if this number is around 3. We expect this number to be
1 buffer returned under load where empty buffers are returned as soon
as they are available. The higher number indicates that maybe buffers
are being returned in sets of 3. What is the reason for this behavior ?
...