Page Comparison

...

This document defines a simple plan to evaluate the performance and scalability of RAMCloud. The first section lists the dimensions we need to take into consideration and the final measurements we should end up with. The second section talks about third section has ideas for simulated workload situations under which these measurements will could be made in the future.

Measurements of interest

The ordering indicates relative importance in the lists that follow. Only the first 2-3 are likely to be attempted to start things off.
At first these measurements could be made on a simple set-up with a single node running master/coordinator/backup/client.

Client's perspective

1. Latency under no-load
   Input parameters - Object Size, Type of operation (Object vs Table, Read vs Write), Capacity of RAMCloud in use, Master failures, Object access distribution (Zipf?)
   Output measured in - micro seconds
   Measured end-to-end from the perspective of the client.Latency under load
   Input parameters - Number of operations under load, Object Size, Type of operation (Object vs Table, Read vs Write), Capacity of RAMCloud in use, Master failures, Object access distribution (Zipf?)
   Output measured in - micro seconds
   A pre-defined workload is first imposed on the system in terms of number of simultaneous operations. Latency measurements are then made for each type of operation.
2. Throughput of operations
   Input parameters - Object Size distribution, Type of operation, Object access distribution, Capacity of RAMCloud in use, Master failures
   Output measured in - number of object reads per second, number of object writes per second
3. Throughput of IO
   Input parameters - Object Size distribution, Type of operation, Object access distribution, Capacity of RAMCloud in use, Master failures
   Output measured in - number of client bytes reads per second, number of client byte writes per second (ideally should only include real content size and not count the overhead imposed by RAMCloud structures)
4. Simultaneous Maximum simultaneous Connections
   Input parameters - Number of masters, Number of client processes, Number of client machines
   Output measured in - number of connections
5. Latency under load
   Input parameters - Number of operations under load, Object Size, Type of operation (Object vs Table, Read vs Write), Capacity of RAMCloud in use, Master failures, Object access distribution (Zipf?)
   Output measured in - micro seconds
   A pre-defined workload is first imposed on the system in terms of number of simultaneous operations. Latency measurements are then made for each type of operation.

Of internal interest

1. Number of rpcs
2. Internal Hash-table size
3. Actual DRAM used versus raw DRAM required for object contents

Graphs of interest

1. #
   #

Simulated workload types

1- Banking Website

...