/
SLIK Tasks

SLIK Tasks

Aug 11, 2015

  • EXPLORATION:

    • Stronger consistency for range lookups: quick and dirty mvcc w/o gc

    • Implement indexing using arrow transactions rather than ordered write / hints approach

  • MAKING SYSTEM USABLE:

    • Coordinator indexing recovery

    • Features:

      • Range lookup: ability to match all

      • Range lookup: ability to limit n

      • Range lookup: ability to reverse sort

      • Multiple types for index keys

      • Support indexed operations in transactions

      • Client library support for changing index schema / object schema (rewrite all objects in table to reflect index changes; this also allows reuse of index slot)

    • B+ tree implementation:

      • Better locking

      • Skip lists?

    • Split and migrate:

      • Automatic load balancing

      • Trim / clean up in B+ trees

    • Misc:

      • Clean dangling pointers (maybe reuse trim/clean-up mechanism from above)

    • Other JIRA Issues

  • TESTING:

    • Deploy and run real app(s) on SLIK

      • Check how Reddit is using Cassandra

    • Testing:

      • Add missing unit tests

      • Split and migrate

      • End-to-end system

      • Recovery

    • New benchmarks:

      • HyperDex: indexScalability

      • Throughput measurements for basic indexed ops

      • Indexed operation timelines

      • B+ tree microbenchmarks

        • Relation between tree fanout and insert/lookup perf

        • Time to insert/remove/lookup for varying params

      • YCSB (requires java bindings to support indexed ops)

      • TPC-H

      • Indexed operation perf affected by concurrent splitAndMigrate

      • Indexed operation perf affected by create/drop index

      • Memory footprint of indexes OR space overhead per indexed object

      • Compare w/ some RDBMS?

      • Latency under load

      • Recovery

    • Understand funny perf from current benchmarks

  • RELATED WORK