RAMCloud 0.2 Wrap-up (Jan 7 meeting)

Implement conditional operations for Diego's Filesystem
Figure out a performance demo
- Ideas:
  - Graph walks (Ryan)
  - Website-simulation using traces, e.g. wikipedia (Christos)
  - Social network graph using Facebook (mendel), NetFlix (Diego), something Jure does (Christos)

RAMCloud 0.2

Tables
Enough of this is done to have a sane interface for now. Don't waste much time on it, since table management is a future design issue. -Diego
Variable sized objects
Backup
Indexes
Reliable networking
1 copy
Key -> Obj lookup

Probably Punted

Variable sized tables
Recovery
Multiple masters
Cleaning

Design Issues

Object storage format

struct chunk_header {
    uint64_t checksum;
    uint64_t magic;
    uint64_t key;
    struct chunk_entry entries[0];
};
struct chunk_entry {
    uint64_t length;
    uint64_t index_id;
    char data[0];
};

Indexes
Key -> Object Lookup
Reliable networking

Buffer management

src/config.h.tmpl:31:#define MAX_RPC_LEN 2048
src/config.h.tmpl:33:#define RC_NUM_TABLES 256
src/config.h.tmpl:42:#define MAX_DATA_WRITE_LEN 16384
src/config.h.tmpl:45:#define HASH_NLINES 16384
src/shared/object.h:61:    char padding[1024];
src/client/client.c:41:enum { ERROR_MSG_LEN = 256 };
src/server/index.h:86:            assert(key.len < 256);
src/server/server.cc:520:    char keys_buf[1024];
src/server/server.cc:521:    uint64_t oids_buf[1024 / sizeof(uint64_t)];
src/server/index.cc:53:    char buf[100];
src/server/hashtable.h:22:#define NBUCKETS        5000
src/server/hashtable.cc:68:    const size_t maxmem = 1 * 1024 * 1024 * 1024;
src/server/server.h:75:            if (index_id == 65535) {
src/server/server.h:181:    Index *indexes[65536];
bindings/python/ramcloud.py:275:        buf = ctypes.create_string_buffer(65536)
bindings/python/ramcloud.py:277:        idx_buf = ctypes.create_string_buffer(65536)
bindings/python/ramcloud.py:425:            keys = ctypes.create_string_buffer(256 * limit)
bindings/haskell/RAMCloud.hs:75:    where max = 4096