Page Comparison

...

I'm optimizing a system in which synchronous disk writes are a key factor in performance. My current drives are too slow. I have a single Intel 520 drive that performs well enough (<1ms), but I need five drives. I bought five Intel 530 SSDs in hopes they would perform like the Intel 520, but instead they take 10ms to write a single byte to disk under various versions of Linux. Curiously, if I connect the Intel 530 drives over a USB-to-SATA adapter instead of using SATA directly, they're much faster (~200us per write). What's wrong?

Microbenchmark

Here's the original microbenchmark, which calls write() and fdatasync() on a single byte of data 1000 times: https://gist.github.com/ongardie/9177853 I run this as "time ./bench" and divide the wall time by 1000 to get the approximate average time per write.

I normally run this on ext4. (I've tried it on a raw device as well; see negative results section below).

A mu

Machines

These are the machines I've tried. The Intel 530 over SATA takes 10ms per write on each of these, and other SSDs take <1ms per write on each of these.

...

Many people have suggested that writing 1 byte is not efficient and writing more bytes should be faster. I tried it both with write-caching on and off. 

Image Added

I was using the same string of random a-z characters for each write. The code used in this benchmark is slightly modified to allow specifying the write size (up to 8KB), see: https://gist.github.com/ongardie/9218611 (oops, this code had a bug, it only called posix_fallocate for the first 1000 KBs, so most of the 4KB and 8KB areas were not pre-allocated. I should re-run those).

In these tests, no drive is faster at writing more than 1 byte than it is at writing 1 byte. Most drives take about 2-4x as long to write 8KB as they do to write 1 byte"x1" machine for some of the tests; the other machines were idle.