The intention of this page is to present experiments with non-CRUD data operations.
An aggregation operations adds up the values of a number of objects. When executing such an operation in RAMCloud three questions, among others, are of interest:
The experiments below are centered around the question about where to execute the operation. Three different scenarios are implemented:
for(uint64_t i = 0; i < range; ++i) { LogEntryHandle handle = objectMap.lookup(tableId, i); const Object* obj = handle->userData<Object>(); int *p; p = (int*) obj->data; sum += (uint64_t)*p; } |
/** * Aggregation Callback */ void aggregateCallback(LogEntryHandle handle, uint8_t type, void *cookie) { const Object* obj = handle->userData<Object>(); MasterServer *server = reinterpret_cast<MasterServer*>(cookie); int *p; p = (int*) obj->data; server->sum += (uint64_t)*p; } |
//** * Aggregation via traversing all SegmentEntries throughout the entire Log */ uint64_t Log::aggregate() { uint64_t sum = 0; //Iterate over all Segments in the Log foreach (ActiveIdMap::value_type& idSegmentPair, activeIdMap) { Segment* segment = idSegmentPair.second; //Iterate over all SegmentEntries in a Segment for (SegmentIterator i(segment); !i.isDone(); i.next()) { SegmentEntryHandle seh = i.getHandle(); //Check that it is an Object if (seh->type()==560620143) { const Object* obj = seh->userData<Object>(); int *p; p = (int*) obj->data; sum += *p; } } } return sum; } |
The benchmarks below have been executed using separate machines (out of the Stanford RAMCloud cluster) for client and server which are connected via Infiniband. After each run, the equality of the client-side and server-side calculated sum has been checked. This particular benchmark allows the following conclusions:
#number of objects |
client-side aggregation |
server-side aggregation |
server-side aggregation |
---|---|---|---|
10.000 |
75 ms |
2 ms |
238 ms |
100.000 |
766 ms |
23 ms |
251 ms |
1.000.000 |
7604 ms |
233 ms |
369 ms |
10.000.000 |
76515 ms |
2662 ms |
1444 ms |
100.000.000 |
770761 ms |
30049 ms |
18752 ms |