clusterperf November 12, 2012
- John Ousterhout
Owned by John Ousterhout
clusterperf output as measured on November 12, 2012 (rc1-rc20)
Notes:
- 100-byte reads are faster, due to Alex Mordkovich's tweak to the Infiniband driver.
- Multi-read is faster, due to Stephen Yang's improvements to the batching mechanism.
- New benchmark readDist computes a cumulative distribution of read times.
- Many changes to the underlying system:
- Refactoring of log management code
- New RPC wrapper mechanism
- New ServerList implementation
- Many changes to recovery
basic.read100 4.9 us read single 100B object with 30B key
basic.readBw100 19.5 MB/s bandwidth reading 100B object with 30B key
basic.read1K 6.7 us read single 1KB object with 30B key
basic.readBw1K 142.7 MB/s bandwidth reading 1KB object with 30B key
basic.read10K 10.1 us read single 10KB object with 30B key
basic.readBw10K 947.6 MB/s bandwidth reading 10KB object with 30B key
basic.read100K 46.5 us read single 100KB object with 30B key
basic.readBw100K 2.0 GB/s bandwidth reading 100KB object with 30B key
basic.read1M 420.6 us read single 1MB object with 30B key
basic.readBw1M 2.2 GB/s bandwidth reading 1MB object with 30B key
basic.write100 15.3 us write single 100B object with 30B key
basic.writeBw100 6.2 MB/s bandwidth writing 100B object with 30B key
basic.write1K 19.2 us write single 1KB object with 30B key
basic.writeBw1K 49.7 MB/s bandwidth writing 1KB object with 30B key
basic.write10K 39.1 us write single 10KB object with 30B key
basic.writeBw10K 244.1 MB/s bandwidth writing 10KB object with 30B key
basic.write100K 246.4 us write single 100KB object with 30B key
basic.writeBw100K 387.0 MB/s bandwidth writing 100KB object with 30B key
basic.write1M 2.3 ms write single 1MB object with 30B key
basic.writeBw1M 420.9 MB/s bandwidth writing 1MB object with 30B key
broadcast 174.8 us broadcast message to 9 slaves
netBandwidth 681.7 MB/s many clients reading from Cdifferent servers
netBandwidth.max 18.5 MB/s fastest client
netBandwidth.min 12.3 MB/s slowest client
readNotFound 13.6 us read object that doesn't exist
# Gauges impact of asynchronous writes on synchronous writes.
# Write two values. The size of the first varies over trials
# (its size is given as 'firstObjectSize'). The first write is
# either synchronous (if firstWriteIsSync is 1) or asynchronous
# (if firstWriteIsSync is 0). The response time of the first
# write is given by 'firstWriteLatency'. The second write is
# a 100 B object which is always written synchronously (its
# response time is given by 'syncWriteLatency'
# Both writes use a 30 B key.
# Generated by 'clusterperf.py writeAsyncSync'
#
# firstWriteIsSync firstObjectSize firstWriteLatency(us) syncWriteLatency(us)
#----------------------------------------------------------------------------
0 100 6.9 18.2
0 1000 8.2 18.6
0 10000 15.7 30.4
0 100000 98.2 193.3
0 1000000 934.2 1314.1
1 100 15.6 15.9
1 1000 19.5 15.9
1 10000 36.3 16.5
1 100000 257.7 18.4
1 1000000 2236.7 24.6
# RAMCloud multiRead performance for 100 B objects with 30 byte keys
# located on a single master.
# Generated by 'clusterperf.py multiRead_oneMaster'
#
# Num Objs Num Masters Objs/Master Latency (us) Latency/Obj (us)
#----------------------------------------------------------------------------
1 1 1 5.4 5.4
2 1 2 6.9 3.4
3 1 3 7.8 2.6
4 1 4 8.9 2.2
5 1 5 10.2 2.0
6 1 6 10.9 1.8
7 1 7 11.8 1.7
8 1 8 12.6 1.6
9 1 9 13.3 1.5
10 1 10 14.3 1.4
20 1 20 21.4 1.1
30 1 30 28.7 1.0
40 1 40 35.9 0.9
50 1 50 42.7 0.9
60 1 60 49.5 0.8
70 1 70 56.8 0.8
80 1 80 64.0 0.8
90 1 90 71.0 0.8
100 1 100 78.9 0.8
200 1 200 111.1 0.6
300 1 300 143.6 0.5
400 1 400 177.5 0.4
500 1 500 212.2 0.4
600 1 600 249.5 0.4
700 1 700 290.8 0.4
800 1 800 331.8 0.4
900 1 900 376.0 0.4
1000 1 1000 418.3 0.4
2000 1 2000 854.8 0.4
3000 1 3000 1257.2 0.4
4000 1 4000 1671.4 0.4
5000 1 5000 2090.4 0.4
# RAMCloud multiRead performance for 100 B objects with 30 byte keys
# with one object located on each master.
# Generated by 'clusterperf.py multiRead_oneObjectPerMaster'
#
# Num Objs Num Masters Objs/Master Latency (us) Latency/Obj (us)
#----------------------------------------------------------------------------
1 1 1 5.5 5.5
2 2 1 7.0 3.5
3 3 1 7.8 2.6
4 4 1 8.9 2.2
5 5 1 9.8 2.0
6 6 1 11.1 1.8
7 7 1 12.3 1.8
8 8 1 13.3 1.7
9 9 1 14.3 1.6
10 10 1 15.6 1.6
11 11 1 19.8 1.8
12 12 1 22.5 1.9
13 13 1 24.4 1.9
14 14 1 25.0 1.8
15 15 1 29.2 1.9
16 16 1 30.6 1.9
17 17 1 31.5 1.9
18 18 1 33.3 1.8
19 19 1 35.9 1.9
20 20 1 37.1 1.9
# RAMCloud multiRead performance for an approximately fixed number
# of 100 B objects with 30 byte keys
# distributed evenly across varying number of masters.
# Generated by 'clusterperf.py multiRead_general'
#
# Num Objs Num Masters Objs/Master Latency (us) Latency/Obj (us)
#----------------------------------------------------------------------------
5000 1 5000 2077.3 0.4
5000 2 2500 2165.2 0.4
4998 3 1666 2575.8 0.5
5000 4 1250 2870.6 0.6
5000 5 1000 2369.8 0.5
4998 6 833 3454.3 0.7
4998 7 714 3582.6 0.7
5000 8 625 3658.4 0.7
4995 9 555 3612.1 0.7
5000 10 500 2488.2 0.5
4994 11 454 3727.0 0.7
4992 12 416 3749.5 0.8
4992 13 384 3843.4 0.8
4998 14 357 3867.5 0.8
4995 15 333 3967.6 0.8
4992 16 312 3972.4 0.8
4998 17 294 3994.5 0.8
4986 18 277 4107.3 0.8
4997 19 263 4067.2 0.8
5000 20 250 4311.1 0.9
# RAMCloud multiRead performance for an approximately fixed number
# of 100 B objects with 30 byte keys
# distributed evenly across varying number of masters.
# Requests are issued in a random order.
# Generated by 'clusterperf.py multiRead_generalRandom'
#
# Num Objs Num Masters Objs/Master Latency (us) Latency/Obj (us)
#----------------------------------------------------------------------------
5000 1 5000 2401.4 0.5
5000 2 2500 2587.6 0.5
4998 3 1666 2781.8 0.6
5000 4 1250 3055.5 0.6
5000 5 1000 3165.7 0.6
4998 6 833 3278.5 0.7
4998 7 714 3268.5 0.7
5000 8 625 3695.7 0.7
4995 9 555 3984.4 0.8
5000 10 500 3773.6 0.8
4994 11 454 3999.1 0.8
4992 12 416 3986.7 0.8
4992 13 384 4203.2 0.8
4998 14 357 4485.6 0.9
4995 15 333 4664.4 0.9
4992 16 312 4750.8 1.0
4998 17 294 4653.6 0.9
4986 18 277 5657.2 1.1
4997 19 263 6063.6 1.2
5000 20 250 6608.6 1.3
# Cumulative distribution of time for a single client to read a
# single 100-byte object from a single server. Each line indicates
# that a given fraction of all reads took at most a given time
# to complete.
# Generated by 'clusterperf.py readDist'
#
# Time (usec) Cum. Fraction
#---------------------------
0.00 0.000
4.53 0.000
4.61 0.010
4.63 0.020
4.64 0.030
4.64 0.040
4.64 0.050
4.64 0.060
4.65 0.070
4.65 0.080
4.65 0.090
4.65 0.100
4.65 0.110
4.65 0.120
4.66 0.130
4.66 0.140
4.66 0.150
4.66 0.160
4.66 0.170
4.67 0.180
4.67 0.190
4.67 0.200
4.68 0.210
4.68 0.220
4.69 0.230
4.69 0.240
4.69 0.250
4.70 0.260
4.70 0.270
4.71 0.280
4.71 0.290
4.71 0.300
4.72 0.310
4.72 0.320
4.72 0.330
4.72 0.340
4.72 0.350
4.72 0.360
4.73 0.370
4.73 0.380
4.73 0.390
4.73 0.400
4.73 0.410
4.73 0.420
4.73 0.430
4.73 0.440
4.73 0.450
4.74 0.460
4.74 0.470
4.74 0.480
4.74 0.490
4.74 0.500
4.74 0.510
4.74 0.520
4.74 0.530
4.74 0.540
4.74 0.550
4.75 0.560
4.75 0.570
4.75 0.580
4.75 0.590
4.75 0.600
4.75 0.610
4.75 0.620
4.76 0.630
4.76 0.640
4.76 0.650
4.76 0.660
4.77 0.670
4.77 0.680
4.77 0.690
4.78 0.700
4.78 0.710
4.79 0.720
4.80 0.730
4.80 0.740
4.81 0.750
4.82 0.760
4.83 0.770
4.84 0.780
4.86 0.790
4.88 0.800
4.89 0.810
4.91 0.820
4.95 0.830
4.99 0.840
5.27 0.850
5.35 0.860
5.39 0.870
5.41 0.880
5.43 0.890
5.45 0.900
5.50 0.910
5.73 0.920
5.85 0.930
5.91 0.940
5.96 0.950
5.98 0.960
6.04 0.970
6.44 0.980
7.10 0.990
9.05 0.999
58.21 0.9999
99.22 1.000
# RAMCloud read performance for 100 B objects
# with keys of various lengths.
# Generated by 'clusterperf.py readVaryingKeyLength'
#
# Key Length Latency (us) Bandwidth (MB/s)
#----------------------------------------------------------------------------
1 5.2 0.2
5 5.1 0.9
10 5.1 1.9
15 5.1 2.8
20 5.2 3.7
25 5.2 4.6
30 5.2 5.5
35 5.3 6.3
40 5.3 7.1
45 5.3 8.1
50 5.3 9.0
55 5.0 10.6
60 5.2 10.9
65 5.2 11.8
70 5.0 13.3
75 5.0 14.2
80 5.1 15.0
85 5.4 15.1
90 5.3 16.1
95 5.5 16.6
100 5.4 17.5
200 6.3 30.5
300 6.5 44.0
400 6.9 54.9
500 7.1 67.3
600 6.8 83.7
700 7.1 93.8
800 7.0 108.7
900 7.1 120.9
1000 7.2 132.2
2000 7.9 241.1
3000 8.6 333.6
4000 9.3 412.2
5000 10.1 471.0
6000 11.4 500.8
7000 12.0 555.1
8000 12.7 601.9
9000 13.0 658.9
10000 13.6 699.5
20000 21.9 872.7
30000 29.5 968.7
40000 39.9 956.0
50000 48.0 994.1
60000 56.0 1021.3
# RAMCloud write performance for 100 B objects
# with keys of various lengths.
# Generated by 'clusterperf.py writeVaryingKeyLength'
#
# Key Length Latency (us) Bandwidth (MB/s)
#----------------------------------------------------------------------------
1 14.6 0.1
5 14.9 0.3
10 14.5 0.7
15 14.5 1.0
20 14.8 1.3
25 14.7 1.6
30 16.1 1.8
35 16.7 2.0
40 16.4 2.3
45 16.1 2.7
50 15.8 3.0
55 16.0 3.3
60 16.1 3.6
65 16.4 3.8
70 16.9 4.0
75 16.6 4.3
80 16.3 4.7
85 15.9 5.1
90 15.6 5.5
95 16.0 5.7
100 16.6 5.7
200 17.7 10.8
300 18.2 15.7
400 18.2 20.9
500 19.5 24.4
600 20.4 28.0
700 20.5 32.6
800 20.5 37.2
900 20.7 41.5
1000 21.7 44.0
2000 25.7 74.2
3000 29.8 95.9
4000 33.4 114.2
5000 37.7 126.4
6000 41.7 137.1
7000 47.7 139.9
8000 51.0 149.7
9000 55.3 155.2
10000 59.2 161.2
20000 108.4 175.9
30000 151.9 188.4
40000 204.4 186.6
50000 257.8 185.0
60000 311.1 183.9
# RAMCloud read performance as a function of load (1 or more
# clients all reading a single 100-byte object with 30-byte key
# repeatedly).
# Generated by 'clusterperf.py readLoaded'
#
# numClients readLatency(us) throughput(total kreads/sec)
#----------------------------------------------------------
1 5.0 199
2 5.3 375
3 5.5 548
4 6.0 670
5 7.0 713
6 8.3 720
7 9.8 712
8 11.4 702
9 12.8 705
10 14.4 696
11 15.7 701
12 17.1 702
13 18.4 707
14 20.2 693
15 21.3 705
16 22.5 711
17 23.9 711
18 25.4 708
19 27.0 704
20 30.1 665
# RAMCloud read performance when 1 or more clients read
# 100-byte objects with 30-byte keys chosen at random from
# 10 servers.
# Generated by 'clusterperf.py readRandom'
#
# numClients throughput(total kreads/sec) slowest(ms) reads > 10us
#--------------------------------------------------------------------
1 185 1.65 0.4%
2 350 1.12 0.3%
3 499 3.36 0.4%
4 707 0.96 0.4%
5 868 1.01 0.5%
6 1024 1.88 0.5%
7 1204 1.98 0.5%
8 1374 0.72 0.5%
9 1573 0.52 0.6%
10 1714 1.56 0.7%
11 1846 0.79 0.7%
12 2022 1.63 0.6%
13 2134 0.87 0.8%
14 2070 13.69 0.8%
15 2355 1.54 0.9%
16 2446 4.01 1.0%
17 2731 0.94 1.1%
18 2841 0.94 1.2%
19 2816 3.33 1.3%
20 3042 1.37 1.3%
21 3042 2.94 1.7%
22 3382 0.60 1.9%
23 3283 3.21 2.2%
24 2936 17.41 2.3%
25 3678 1.89 2.6%
26 3582 3.25 3.2%
27 3810 1.89 3.6%
28 3942 2.01 4.2%
29 3933 1.90 4.8%
30 4220 0.95 5.4%
31 4187 1.96 6.1%
32 4284 1.97 7.2%
33 4392 2.03 8.4%
34 4424 10.01 9.2%
35 4267 8.37 10.5%
36 3259 27.18 12.4%
37 3610 20.92 13.1%
38 4502 4.40 14.4%
39 4704 2.32 15.4%
40 4749 2.35 16.4%
41 4764 4.30 17.6%
42 4964 2.83 17.7%
43 4476 11.76 18.5%
44 4848 3.82 20.8%
45 4991 4.09 21.9%
46 4707 6.16 23.7%
47 3714 35.84 24.9%
48 5349 2.16 25.4%
49 5090 3.78 25.5%
50 5023 10.02 27.4%