...
Steve: A (key, version) tuple.
In a previous meeting we had used Object ID as a (table, primary key) tuple. Which is it?
A (key, version) tuple scoped to a table would identify a blob. If that's useful, maybe it should be a Blob ID.
Table
A table is a logical grouping of objects which share a set of indexes. Steve: A scoping of keys within the system. All objects live in An object lives in exactly one table and all indices are an index is associated with exactly one table. A table also scopes primary keys within the system.
Primary key
A primary key is a system-assigned 64-bit integer (which will never be reused?) . Together with a table, it identifies an object.Steve: The system-generated key that uniquely identifies an object within a table.
Version Number
A version_number of an object is an integer that refers to the revision of its blob. It is used for the overwrite request, which asserts the previous version number as a parameter: if the server finds the given version number is out of date, the overwrite request will be aborted. Steve: A number associated with a primary key that is system-incremented when When an object is modified, the version number will be incremented by the system.
Shard
A shard is a set of objects corresponding to a contiguous region of primary keys of a table – a subset of a "table's" key space. An object is a member of exactly one shard. Steve: A subset of a table's key space. Shards are sized for efficient disk access (i.e., they can be sucked into memory from a backup server's disk within a small amount of time on failureif the master server fails).
Block
A block is a fixed-size the main unit of memory allocation unit on a server. Blocks have a fixed-size, and much of a server's memory will be treated as an array of blocks. Blobs are stored in a set of blocks. While the blob need not start or end on block boundaries, it must fully utilize any blocks in the middle.Steve: A shardlet is composed of multiple fixed sized blocks. Blocks are the main unit of memory allocation in the system. They store one or more objects or portions of objects and are mapped into shardlets by an explicit table structure (inode)A block is mapped into the blob of an object by that object's inode. A blob may start partway through the first block and end partway through a last block. In either case, space may be shared with another object. A block then stores one or more blobs or portions of blobs.
Inode
An inode stores an object. This includes the blob's checksum, its version, and all of its index entries. It also includes the list mapping of blocks for the its blob, the start offset, and the length.
Steve: Each object is mapped to one or more blocks by an inode mapping. An object may start partway through the first block and end partway through a last block. In either case, space may be shared with another object. An inode contains metadata including an object checksum, version and index name, key tuples.
Shardlet
A shardlet is an array of constant size of blocks (akin to LFS segments). A shardlet will likely have a maximum size that is convenient efficient to write to disk, as a backup server will write a shardlet at a time sequentially to disk. A shardlet on a master server will be an array of pointers to blocks, while a shardlet on a backup server will probably be an array of blocks (directly).
Steve: Shards are broken up into shardlets, which are akin to LFS segments. Each shardlet is written to disk sequentially and sized such that they can be efficiently accessed in light of disk seek times.
Index
An index, made up of index entries, maps index keys to objects.
Steve: A lookup structure for arbitrary keys to object identifiers. Can be range-queryable
Index
An index is a lookup structure from arbitrary index keys to objects. An index is made up of index entries and can service range queries.
Index Key
An index key is a short, application-controlled string type used in index lookup queries and range queries on indexes. Steve: A typed index into an index structure (The types may include string, int, float, etc).
Index Entries
An index entry is an (index key, object) pair in a given index.
...