Foreword

These experiments explore the write latencies of several SSD's in our cluster. They are conducted in a similar style to Diego's Intel 530 Performance, but were performed at a later time with a standard unix tool (dd) instead of a custom C program. These tests also extend the original experiments by looking into the effects of different file systems vs. raw drive performance. 

Summary

• Write Cache: Disabling the write cache improves latency, but sacrifices bandwidth in most cases.
  
• Drives: Intel drives have the lowest latencies (<1ms) with the fastest configuration being in RAW with Write Cached Disabled (48µs write latency!!)
  • Note: The intel drive also saw a bandwidth increase with write cache disabled
• File Systems: XFS has lower write latencies than EXT4 (1-2x faster).
• TRIM: TRIM makes minimal difference in the SSD write latencies (although it may be because the drive is completely empty at the beginning of the benchmark)

Setup

We tested 4 SSDs and 1 HDD (for comparison) while varying the drive's file system, TRIM option, and write cache. Each configuration was benchmarked with the following dd command and the output in seconds is interpreted as millisecond latencies:

dd if=/dev/zero bs=512 count=1000 oflag=direct,sync of=$TARGET

where $TARGET specfies the device (i.e. /dev/sdb if raw) or a file on the device (i.e. /mnt/sdb/testFile).

The drive's file system, write cache, and TRIM options were varied with the following table of commands

The entire experiment can be run with the following script

WARNING: THE DRIVE IS SPECIFIED IN THE SCRIPT, CHANGE IT TO THE CORRECT ONE OR /dev/sdb WILL BE DESTROYED

benchmark.sh

Hardware

Results

Raw

Pivot Table

Here is a pivot table summarizing the results. Since the TRIM command didn't seem to make much of a difference, they are not included in the results below.

All times are in milliseconds and +WC means with Write Cached Enabled, -WC means with Write Cached Disabled.

Write Cache Effects

Final comparison between the bandwidth + latency of enabling/disbling the Write Cache on a RAW device

The Maximum Bandwidth calculation was generated from: 

dd if=/dev/zero bs=128M count=10 oflag=direct,sync of=$DRIVE