...
- How much metadata space is needed for all tables/applications?
- Up to 2^48 objects, 128 bit (16 byte) key, size (4-8 bytes), permissions or appid if not in address (2-4 bytes)
- (2^43)*16 = 128 TB
- How does metadata replication occur and what is the frequency?
- Lazily
- Not sufficient when a client discovers a host is down
- must update mappings in the new replicas at least very quickly
- May additionally want leases or heartbeat or something similar as in MapReduce to ensure enough copies of shards are maintained on failure even if the data is cold
Approaches
Mapping/Address Space Paritioning
...