/
Linearizable RPC & TX progress

Linearizable RPC & TX progress

Jul 30, 2015

TODO for SOSP Camera Ready

 

  • 1 M client benchmark / cstlee

  • Full-mix for TPC-C benchmark / seojin

  • Measure 1-level deeper for throughput (factor analysis?)

  • Clarify the term linearizability: make proposals. / collin

  • Talk about ultimate client / John??

  • Clarify exp setup for TPC-C benchmark.

  • Comparision with tell from VLDB.

  • Extra paragraph lease

 

Tentative schedule for both linearizability and TX.

Nov 3: Today

Nov 14: TX design draft

Nov 21: Linearizability implementation

Jan 27: TX Implementation

Feb 27: 1st draft & Measurements

Mar 28: Final

Transaction Components (Assumes 4 + 1 days / wk)

Client Driven Transaction

  • Prep Rpc and Handling - Seojin / DONE!

  • Client driven transactions "task" - Collin / DONE!

  • Transaction object & API - Collin /  DONE!

  • Multiple objects per prepare RPC - DONE!

Transaction Cleanup Coordinator (Recovery Coordinator)

  • Timer and Rpc for initiating recovery protocol. - Seojin / Half DONE.

  • Request Abort Rpc and Handling - Seojin / 2 day

  • Transaction Cleanup "task" - Collin / DONE!

  • Transaction Cleanup crash recovery - Collin / DONE!

Common Components (Mostly in Data Master)

  • Decision Rpc - seojin / DONE!

  • lock table - Collin / DONE!

  • Special GC rules for transaction entires (& lease expiration handling) - Seojin / 3 day

  • Transaction log records & recovery - Seojin / DONE!

Benchmark

Micro-benchmarks

  • Throughput and latency for TX varying # of objects and size of objects / READ vs. WRITE / # of servers. No contention.

  • Varied contention

  • (Conditional) Write latency for varied size of objects.

  • Crash recovery for client / server?

  • Scalability of leases?

Macro-benchmark

  • Run TPC-C and compare with other systems (eg. H-store)

  • YCSB?

  • Ask Ankita and SYang

Current Client Side TimeTrace:

(+ 21.5 ns): sendPrepareRpc find first object (+ 41.6 ns): sendPrepareRpc objectFinder.lookup (+ 186.1 ns): sendPrepareRpc prepareRpcs.emplace_back (+ 72.3 ns): sendPrepareRpc objectFinder.lookup and service locator comparison (+ 43.6 ns): appendOp append data to rpc (+ 94.1 ns): sendPrepareRpc break loop -- objectFinder.lookup and service locator comparison (+ 13.3 ns): send call (+ 14.7 ns): send get ackId (+ 456.2 ns): send session->sendRequest (+ 14.7 ns): sendPrepareRpc send return (+ 34.1 ns): processPrepareRpcs check for ready Rpcs (+ 100.6 ns): processPrepareRpcs prepareRpcs.erase ===================================================== (+1092.8 ns): total

 

Linearizability Components

  • increment handling

  • Coordinator RPC GetLeaseInfo w/ batching

 

Coordinator

  • Cluster Clock and Time support - Collin

  • Lease manager - Collin

Master

  • Backup's log reading during recovery - Collin (6days)

  • Recovery from log <logReplay> (6 days)

  • Handling of Linearizable RPC in MasterService - Seojin (3 days)

  • Durable logging - Seojin (3 days)

  • Log cleaning for rpc log entry - Collin (4 days)

  • In-memory client table (UnackedRpcResult) - Seojin

Client

  • Linearizable RPC handling. (using RpcTracker and ClusterClock) - Seojin (3 days)

  • Client lease - Collin (4 days)

  • In-memory RPC table (RpcTracker) - Seojin