clusterperf November 12, 2012

clusterperf output as measured on  November 12, 2012 (rc1-rc20)

Notes:

  • 100-byte reads are faster, due to Alex Mordkovich's tweak to the Infiniband driver.
  • Multi-read is faster, due to Stephen Yang's improvements to the batching mechanism.
  • New benchmark readDist computes a cumulative distribution of read times.
  • Many changes to the underlying system:
    • Refactoring of log management code
    • New RPC wrapper mechanism
    • New ServerList implementation
    • Many changes to recovery

 

basic.read100          4.9 us     read single 100B object with 30B key
basic.readBw100       19.5 MB/s   bandwidth reading 100B object with 30B key
basic.read1K           6.7 us     read single 1KB object with 30B key
basic.readBw1K       142.7 MB/s   bandwidth reading 1KB object with 30B key
basic.read10K         10.1 us     read single 10KB object with 30B key
basic.readBw10K      947.6 MB/s   bandwidth reading 10KB object with 30B key
basic.read100K        46.5 us     read single 100KB object with 30B key
basic.readBw100K       2.0 GB/s   bandwidth reading 100KB object with 30B key
basic.read1M         420.6 us     read single 1MB object with 30B key
basic.readBw1M         2.2 GB/s   bandwidth reading 1MB object with 30B key
basic.write100        15.3 us     write single 100B object with 30B key
basic.writeBw100       6.2 MB/s   bandwidth writing 100B object with 30B key
basic.write1K         19.2 us     write single 1KB object with 30B key
basic.writeBw1K       49.7 MB/s   bandwidth writing 1KB object with 30B key
basic.write10K        39.1 us     write single 10KB object with 30B key
basic.writeBw10K     244.1 MB/s   bandwidth writing 10KB object with 30B key
basic.write100K      246.4 us     write single 100KB object with 30B key
basic.writeBw100K    387.0 MB/s   bandwidth writing 100KB object with 30B key
basic.write1M          2.3 ms     write single 1MB object with 30B key
basic.writeBw1M      420.9 MB/s   bandwidth writing 1MB object with 30B key
broadcast            174.8 us     broadcast message to 9 slaves
netBandwidth         681.7 MB/s   many clients reading from Cdifferent servers
netBandwidth.max      18.5 MB/s   fastest client
netBandwidth.min      12.3 MB/s   slowest client
readNotFound          13.6 us     read object that doesn't exist
# Gauges impact of asynchronous writes on synchronous writes.
# Write two values. The size of the first varies over trials
# (its size is given as 'firstObjectSize'). The first write is
# either synchronous (if firstWriteIsSync is 1) or asynchronous
# (if firstWriteIsSync is 0). The response time of the first
# write is given by 'firstWriteLatency'. The second write is
# a 100 B object which is always written synchronously (its 
# response time is given by 'syncWriteLatency'
# Both writes use a 30 B key.
# Generated by 'clusterperf.py writeAsyncSync'
#
# firstWriteIsSync firstObjectSize firstWriteLatency(us) syncWriteLatency(us)
#----------------------------------------------------------------------------
                 0             100                   6.9                 18.2
                 0            1000                   8.2                 18.6
                 0           10000                  15.7                 30.4
                 0          100000                  98.2                193.3
                 0         1000000                 934.2               1314.1
                 1             100                  15.6                 15.9
                 1            1000                  19.5                 15.9
                 1           10000                  36.3                 16.5
                 1          100000                 257.7                 18.4
                 1         1000000                2236.7                 24.6
# RAMCloud multiRead performance for 100 B objects with 30 byte keys
# located on a single master.
# Generated by 'clusterperf.py multiRead_oneMaster'
#
# Num Objs    Num Masters    Objs/Master    Latency (us)    Latency/Obj (us)
#----------------------------------------------------------------------------
         1              1              1            5.4                5.4
         2              1              2            6.9                3.4
         3              1              3            7.8                2.6
         4              1              4            8.9                2.2
         5              1              5           10.2                2.0
         6              1              6           10.9                1.8
         7              1              7           11.8                1.7
         8              1              8           12.6                1.6
         9              1              9           13.3                1.5
        10              1             10           14.3                1.4
        20              1             20           21.4                1.1
        30              1             30           28.7                1.0
        40              1             40           35.9                0.9
        50              1             50           42.7                0.9
        60              1             60           49.5                0.8
        70              1             70           56.8                0.8
        80              1             80           64.0                0.8
        90              1             90           71.0                0.8
       100              1            100           78.9                0.8
       200              1            200          111.1                0.6
       300              1            300          143.6                0.5
       400              1            400          177.5                0.4
       500              1            500          212.2                0.4
       600              1            600          249.5                0.4
       700              1            700          290.8                0.4
       800              1            800          331.8                0.4
       900              1            900          376.0                0.4
      1000              1           1000          418.3                0.4
      2000              1           2000          854.8                0.4
      3000              1           3000         1257.2                0.4
      4000              1           4000         1671.4                0.4
      5000              1           5000         2090.4                0.4
# RAMCloud multiRead performance for 100 B objects with 30 byte keys
# with one object located on each master.
# Generated by 'clusterperf.py multiRead_oneObjectPerMaster'
#
# Num Objs    Num Masters    Objs/Master    Latency (us)    Latency/Obj (us)
#----------------------------------------------------------------------------
         1              1              1            5.5                5.5
         2              2              1            7.0                3.5
         3              3              1            7.8                2.6
         4              4              1            8.9                2.2
         5              5              1            9.8                2.0
         6              6              1           11.1                1.8
         7              7              1           12.3                1.8
         8              8              1           13.3                1.7
         9              9              1           14.3                1.6
        10             10              1           15.6                1.6
        11             11              1           19.8                1.8
        12             12              1           22.5                1.9
        13             13              1           24.4                1.9
        14             14              1           25.0                1.8
        15             15              1           29.2                1.9
        16             16              1           30.6                1.9
        17             17              1           31.5                1.9
        18             18              1           33.3                1.8
        19             19              1           35.9                1.9
        20             20              1           37.1                1.9
# RAMCloud multiRead performance for an approximately fixed number
# of 100 B objects with 30 byte keys
# distributed evenly across varying number of masters.
# Generated by 'clusterperf.py multiRead_general'
#
# Num Objs    Num Masters    Objs/Master    Latency (us)    Latency/Obj (us)
#----------------------------------------------------------------------------
      5000              1           5000         2077.3                0.4
      5000              2           2500         2165.2                0.4
      4998              3           1666         2575.8                0.5
      5000              4           1250         2870.6                0.6
      5000              5           1000         2369.8                0.5
      4998              6            833         3454.3                0.7
      4998              7            714         3582.6                0.7
      5000              8            625         3658.4                0.7
      4995              9            555         3612.1                0.7
      5000             10            500         2488.2                0.5
      4994             11            454         3727.0                0.7
      4992             12            416         3749.5                0.8
      4992             13            384         3843.4                0.8
      4998             14            357         3867.5                0.8
      4995             15            333         3967.6                0.8
      4992             16            312         3972.4                0.8
      4998             17            294         3994.5                0.8
      4986             18            277         4107.3                0.8
      4997             19            263         4067.2                0.8
      5000             20            250         4311.1                0.9
# RAMCloud multiRead performance for an approximately fixed number
# of 100 B objects with 30 byte keys
# distributed evenly across varying number of masters.
# Requests are issued in a random order.
# Generated by 'clusterperf.py multiRead_generalRandom'
#
# Num Objs    Num Masters    Objs/Master    Latency (us)    Latency/Obj (us)
#----------------------------------------------------------------------------
      5000              1           5000         2401.4                0.5
      5000              2           2500         2587.6                0.5
      4998              3           1666         2781.8                0.6
      5000              4           1250         3055.5                0.6
      5000              5           1000         3165.7                0.6
      4998              6            833         3278.5                0.7
      4998              7            714         3268.5                0.7
      5000              8            625         3695.7                0.7
      4995              9            555         3984.4                0.8
      5000             10            500         3773.6                0.8
      4994             11            454         3999.1                0.8
      4992             12            416         3986.7                0.8
      4992             13            384         4203.2                0.8
      4998             14            357         4485.6                0.9
      4995             15            333         4664.4                0.9
      4992             16            312         4750.8                1.0
      4998             17            294         4653.6                0.9
      4986             18            277         5657.2                1.1
      4997             19            263         6063.6                1.2
      5000             20            250         6608.6                1.3
# Cumulative distribution of time for a single client to read a
# single 100-byte object from a single server.  Each line indicates
# that a given fraction of all reads took at most a given time
# to complete.
# Generated by 'clusterperf.py readDist'
#
# Time (usec)  Cum. Fraction
#---------------------------
    0.00       0.000
    4.53       0.000
    4.61       0.010
    4.63       0.020
    4.64       0.030
    4.64       0.040
    4.64       0.050
    4.64       0.060
    4.65       0.070
    4.65       0.080
    4.65       0.090
    4.65       0.100
    4.65       0.110
    4.65       0.120
    4.66       0.130
    4.66       0.140
    4.66       0.150
    4.66       0.160
    4.66       0.170
    4.67       0.180
    4.67       0.190
    4.67       0.200
    4.68       0.210
    4.68       0.220
    4.69       0.230
    4.69       0.240
    4.69       0.250
    4.70       0.260
    4.70       0.270
    4.71       0.280
    4.71       0.290
    4.71       0.300
    4.72       0.310
    4.72       0.320
    4.72       0.330
    4.72       0.340
    4.72       0.350
    4.72       0.360
    4.73       0.370
    4.73       0.380
    4.73       0.390
    4.73       0.400
    4.73       0.410
    4.73       0.420
    4.73       0.430
    4.73       0.440
    4.73       0.450
    4.74       0.460
    4.74       0.470
    4.74       0.480
    4.74       0.490
    4.74       0.500
    4.74       0.510
    4.74       0.520
    4.74       0.530
    4.74       0.540
    4.74       0.550
    4.75       0.560
    4.75       0.570
    4.75       0.580
    4.75       0.590
    4.75       0.600
    4.75       0.610
    4.75       0.620
    4.76       0.630
    4.76       0.640
    4.76       0.650
    4.76       0.660
    4.77       0.670
    4.77       0.680
    4.77       0.690
    4.78       0.700
    4.78       0.710
    4.79       0.720
    4.80       0.730
    4.80       0.740
    4.81       0.750
    4.82       0.760
    4.83       0.770
    4.84       0.780
    4.86       0.790
    4.88       0.800
    4.89       0.810
    4.91       0.820
    4.95       0.830
    4.99       0.840
    5.27       0.850
    5.35       0.860
    5.39       0.870
    5.41       0.880
    5.43       0.890
    5.45       0.900
    5.50       0.910
    5.73       0.920
    5.85       0.930
    5.91       0.940
    5.96       0.950
    5.98       0.960
    6.04       0.970
    6.44       0.980
    7.10       0.990
    9.05       0.999
   58.21       0.9999
   99.22       1.000
# RAMCloud read performance for 100 B objects
# with keys of various lengths.
# Generated by 'clusterperf.py readVaryingKeyLength'
#
# Key Length      Latency (us)     Bandwidth (MB/s)
#----------------------------------------------------------------------------
           1              5.2                 0.2
           5              5.1                 0.9
          10              5.1                 1.9
          15              5.1                 2.8
          20              5.2                 3.7
          25              5.2                 4.6
          30              5.2                 5.5
          35              5.3                 6.3
          40              5.3                 7.1
          45              5.3                 8.1
          50              5.3                 9.0
          55              5.0                10.6
          60              5.2                10.9
          65              5.2                11.8
          70              5.0                13.3
          75              5.0                14.2
          80              5.1                15.0
          85              5.4                15.1
          90              5.3                16.1
          95              5.5                16.6
         100              5.4                17.5
         200              6.3                30.5
         300              6.5                44.0
         400              6.9                54.9
         500              7.1                67.3
         600              6.8                83.7
         700              7.1                93.8
         800              7.0               108.7
         900              7.1               120.9
        1000              7.2               132.2
        2000              7.9               241.1
        3000              8.6               333.6
        4000              9.3               412.2
        5000             10.1               471.0
        6000             11.4               500.8
        7000             12.0               555.1
        8000             12.7               601.9
        9000             13.0               658.9
       10000             13.6               699.5
       20000             21.9               872.7
       30000             29.5               968.7
       40000             39.9               956.0
       50000             48.0               994.1
       60000             56.0              1021.3
# RAMCloud write performance for 100 B objects
# with keys of various lengths.
# Generated by 'clusterperf.py writeVaryingKeyLength'
#
# Key Length      Latency (us)     Bandwidth (MB/s)
#----------------------------------------------------------------------------
           1             14.6                 0.1
           5             14.9                 0.3
          10             14.5                 0.7
          15             14.5                 1.0
          20             14.8                 1.3
          25             14.7                 1.6
          30             16.1                 1.8
          35             16.7                 2.0
          40             16.4                 2.3
          45             16.1                 2.7
          50             15.8                 3.0
          55             16.0                 3.3
          60             16.1                 3.6
          65             16.4                 3.8
          70             16.9                 4.0
          75             16.6                 4.3
          80             16.3                 4.7
          85             15.9                 5.1
          90             15.6                 5.5
          95             16.0                 5.7
         100             16.6                 5.7
         200             17.7                10.8
         300             18.2                15.7
         400             18.2                20.9
         500             19.5                24.4
         600             20.4                28.0
         700             20.5                32.6
         800             20.5                37.2
         900             20.7                41.5
        1000             21.7                44.0
        2000             25.7                74.2
        3000             29.8                95.9
        4000             33.4               114.2
        5000             37.7               126.4
        6000             41.7               137.1
        7000             47.7               139.9
        8000             51.0               149.7
        9000             55.3               155.2
       10000             59.2               161.2
       20000            108.4               175.9
       30000            151.9               188.4
       40000            204.4               186.6
       50000            257.8               185.0
       60000            311.1               183.9
# RAMCloud read performance as a function of load (1 or more
# clients all reading a single 100-byte object with 30-byte key
# repeatedly).
# Generated by 'clusterperf.py readLoaded'
#
# numClients  readLatency(us)  throughput(total kreads/sec)
#----------------------------------------------------------
    1            5.0               199
    2            5.3               375
    3            5.5               548
    4            6.0               670
    5            7.0               713
    6            8.3               720
    7            9.8               712
    8           11.4               702
    9           12.8               705
   10           14.4               696
   11           15.7               701
   12           17.1               702
   13           18.4               707
   14           20.2               693
   15           21.3               705
   16           22.5               711
   17           23.9               711
   18           25.4               708
   19           27.0               704
   20           30.1               665
# RAMCloud read performance when 1 or more clients read
# 100-byte objects with 30-byte keys chosen at random from
# 10 servers.
# Generated by 'clusterperf.py readRandom'
#
# numClients  throughput(total kreads/sec)  slowest(ms)  reads > 10us
#--------------------------------------------------------------------
  1                  185                      1.65          0.4%
  2                  350                      1.12          0.3%
  3                  499                      3.36          0.4%
  4                  707                      0.96          0.4%
  5                  868                      1.01          0.5%
  6                 1024                      1.88          0.5%
  7                 1204                      1.98          0.5%
  8                 1374                      0.72          0.5%
  9                 1573                      0.52          0.6%
 10                 1714                      1.56          0.7%
 11                 1846                      0.79          0.7%
 12                 2022                      1.63          0.6%
 13                 2134                      0.87          0.8%
 14                 2070                     13.69          0.8%
 15                 2355                      1.54          0.9%
 16                 2446                      4.01          1.0%
 17                 2731                      0.94          1.1%
 18                 2841                      0.94          1.2%
 19                 2816                      3.33          1.3%
 20                 3042                      1.37          1.3%
 21                 3042                      2.94          1.7%
 22                 3382                      0.60          1.9%
 23                 3283                      3.21          2.2%
 24                 2936                     17.41          2.3%
 25                 3678                      1.89          2.6%
 26                 3582                      3.25          3.2%
 27                 3810                      1.89          3.6%
 28                 3942                      2.01          4.2%
 29                 3933                      1.90          4.8%
 30                 4220                      0.95          5.4%
 31                 4187                      1.96          6.1%
 32                 4284                      1.97          7.2%
 33                 4392                      2.03          8.4%
 34                 4424                     10.01          9.2%
 35                 4267                      8.37          10.5%
 36                 3259                     27.18          12.4%
 37                 3610                     20.92          13.1%
 38                 4502                      4.40          14.4%
 39                 4704                      2.32          15.4%
 40                 4749                      2.35          16.4%
 41                 4764                      4.30          17.6%
 42                 4964                      2.83          17.7%
 43                 4476                     11.76          18.5%
 44                 4848                      3.82          20.8%
 45                 4991                      4.09          21.9%
 46                 4707                      6.16          23.7%
 47                 3714                     35.84          24.9%
 48                 5349                      2.16          25.4%
 49                 5090                      3.78          25.5%
 50                 5023                     10.02          27.4%