Multikey object format

Summary of work done:

• Started by writing a new class to support multiple key objects and modifying the existing object class to reflect the same format in the log. Things got messy and confusing.
• Finally replaced the existing object class to support variable number of keys. Now the solution is much cleaner
• Most of the existing unit tests that used single keys had to be fixed to adhere to the new object format.
• Object class is used more extensively now
• Implemented a new sub-class of Buffer called ObjectBuffer that understands the new object format. It is essentially a wrapper around the Object class to restrict knowledge of the object format to one class.
• Clients can invoke operations on this subclass typically after the completion of a readKeysAndValue RPC to parse out specific fields - keys/value.

IMPORTANT:

To minimize overhead and the overall cost of write operations, the format of the object in the log is tied to the format of the object's value in the rpc payload. This prevents unnecessary copying/moving of data. The RPC payload will however not contain the object header which is written to the log. 

Summary of client side changes:

Writes:

• Separate write APIs for single key and multi key objects.
• multi-key write API takes keys in the following form array[<char *key, keyLen>]
• Still uses a single write RPC

Reads:

• Two variants:
  • read -- takes a Buffer argument and returns just the value of the object
  • readKeysAndValue --> takes an ObjectBuffer argument and returns information about keys and value

Multiwrite:

• Separate constructors in class MultiWriteObject for single key and multi-key objects
• Single RPC

Multiread:

• Currently, single RPC that returns keysAndValue
• PENDING - support for retuning just value (not clear of the use case)
  • new RPC
    • involves code duplication, new subclass of MultiOpObject that takes Tub<Buffer *> as a field instead of Tub<ObjectBuffer *>
  • only one RPC but additional argument which can be a field in MultiOpObject indicating whether or not this is multikey object
    • Caveat - scope for ambiguity. Clients may invoke operations on ObjectBuffer even though ObjectBuffer may contain just the 'value' of the object.

Some performance measurements:

• Ran clusterperf basic to measure overhead of the new structure/code
  • no perceivable difference in basic read and write performance.
• Summary of micro benchmarks to measure the performance of the 'getKey()' function

• configuration	Old format	New format
  Single key, key in contiguous memory	8.42 ns	6 - 7 ns
  key in a Buffer (single key)	25 ns	---
  key in a Buffer (50 secondary keys). Measure the first call to getKey()	---	25 ns
  key in a Buffer (50 secondary keys). Measure the second call to getKey() on the same object	---	11.75 ns

• The difference in performance between the first call to getKey() and a subsequent call to getKey() for the same object is due to the following optimization.
  • There are many inefficient methods to return a key from the object. What is important here is that we can expect this function to be very frequently, both by the client side code and the server side code
  • This also motivated a change in the object format storing cumulative key length values rather than lengths of each individual key.
  • The inefficiencies mainly arise if the object doesn't exist in contiguous memory and instead lies in a buffer that may not be contiguous.
  • Since, the object can be arbitrarily large, we don't want to copy all the keys and the value into contiguous memory upfront.
  • So, the first call to getKey() or any similar function that directly deals with keys like getKeyLength() etc will populate a structure that can store the number of keys in the object and all the cumulative key length values in contiguous memory. Notice that the size of this is roughly (2 * numKeys) bytes only.
  • For ever subsequent call to getKey(), the cumulative key length values already exist in memory. So the only thing that remains is to copy the key into contiguous memory based on its offset within the object