/
Data stores reference for Indexing

Data stores reference for Indexing

Dec 12, 2016

Ankita's quick guide (still a WIP) to key features in data stores, with focus on indexing support. (If you find something to be inaccurate or know another data store that does well on most/all the parameters, let me know!)

Data Store (& Link)

Indexed

In-mem

Dist/Scalable data, index

Durable

Latency

Throughput

Consistency

API

Additional comments

Data Store (& Link)

Indexed

In-mem

Dist/Scalable data, index

Durable

Latency

Throughput

Consistency

API

Additional comments

RAMCloud

y

y

y, y

y [n]

?

?

Linearizable

keys - val

 

MySQL SE

y

n

y, ? (replication)

y

 

 

Transaction safe, ACID compliant

relational

 

MySQL Cluster

y

n

y, ? (replication & partitioning)

y

 

 

 

SQL and NoSQL

 

Cassandra

y

cache

y, y*

y

 

 

BASE (Basically Available Soft-state Eventual Consistency) + Can choose consistency level

partitioned rows (similar to sql) + denormalization + materialized views

*: each node indexes data it holds locally

MongoDB

y

cache

 

y

 

 

 

document oriented storage: JSON-style docs with dynamic schemas

 

H-Store

? (1)

y

y, ?

 

 

 

 

row-based-relational

 

VoltDB

? (1)

y

y, ?

 

 

 

ACID for transactions; unclear otherwise.

relational

Commercial H-store

G-Store

n

 

 

 

 

 

 

 

 

LevelDB

n

 

 

 

 

 

 

key-val

 

Spanner

n

 

 

 

 

 

Externally-consistent

 

 

F1

y

n

y, ?

y

> mysql

 

Strong in gen, consistent global indexes

relational + sql

Uses Spanner; Google ads used MySQL. This made their db scalable.

PNUTS [paper]

kind-of*

n

y, -

y (2)

 

 

Relaxed

Basic relational

*: Optional secondary table lazily maintained; keyed on index key

DynamoDB

y

n (ssd)

y, ?

y (2)

single digit ms

 

Strong consistency on reads

tables, no fixed schemas. each item: diff num of attrs

 

BigTable

n

 

 

 

 

 

Strong

multi dimensional map which supports basic operations

 

Espresso

y

n

y, ?

y

 

 

Timeline-consistent

document oriented NoSQL; has secondary index

Uses MySQL/InnoDB as storage engine. Also uses Lucene+Databus+Helix

Postgres

? (1)

n

y, ?

y (2)

 

 

ACID compliant, MVCC

object-relational

 

COPS

n

 

 

 

 

 

Causal+

key-val

 

Eiger

n

 

 

 

 

 

Causal

column-store

 

Hyperdex [paper]

y

n

y, ?

 

 

 

Key ops are linearizable; Warp (commercial extension) has ACID transactions

key-val; rich datatypes

 

H-Base

n

 

y, -

 

 

 

Strictly consistent reads and writes

versioned, non-relational; has global and local indexes

Modeled after BigTable

Cloudera Impala

? (1)

n

y, ?

 

 

 

 

SQL interface

Massively parallel processing arch for perf w/ Hadoop Scalability

Redis

n

y

y, -

optional

 

 

 

key-val; keys can contain strings, hashes, lists, sets, sorted sets

 

CouchDB

y

 

y, ?

y

 

 

Eventually consistent

JSON

 

BigCouch

y

 

 

 

 

 

 

 

Commercial couchdb on steroids

Voldermort

n

cache

y

y

 

 

 

hash table

 

NuDB

 

 

 

 

 

 

 

relational?

 

Footnotes:

(1): Probably yes since they claim to be relational / sql.

(2): Probably yes since it is not in-memory.

Sources:

Websites/papers linked in the first column and official blog/wikis.