Making friends with your new SSD – a simple baseline

It seems with lots of customers they should focus on a very simple baseline to see how the CPU, memory and IO controller (SATA, SAS, PCIe) function with their new Intel SSD. So what are the simplest steps?

  1. Read the specs first,  writing and reading tests then give you an idea of what you want to get close to achieving, in this case with a raw test can we get somewhat close to the specifications of 400-500 MB/sec of an Intel SSD when we are copying some blocks to the SSD in directIO mode.
  2. Attach your SSD to the best interface that you have with the least hardware or protocol overhead as possible on a given system, a SATA3 interface will be better than a SATA2 interface, a PCIe interface will be best.
  3. Test it simply and test just one of your new drives as RAID, Spans, software RAID, controller firmware's, fancy controllers, these will all add complexity and deviations. User space software like database processes, will make things amazingly more complex. So if things are going wrong stop, and start with the following three tests.

Direct Write Test

First off remember system OS'es, and controllers, are always setup for caching and buffering, so use DirectIO, here is an example syntax that should work on most flavors of Linux/Unix, or try Cygwin on Windows to achieve the same. So to me it seems amazingly universal but you tell me, is there a more universal tool than "dd" out there that's more powerful and more standard for a fast memory to device copy test.

$ time dd if=/dev/zero of=tstfile bs=2048k count=512 oflag=direct
2048+0 records in
2048+0 records out
1073741824 bytes (1.1 GB) copied, 3.53208 s, 304 MB/s

real 0m3.619s
user 0m0.187s
sys 0m0.217s

Direct Read Test

You can reverse things to see what the read test looks like, things should be faster on reads in all cases with respect to the SSD:

$ time dd of=/dev/zero if=tstfile bs=2048k count=512 iflag=direct
2048+0 records in
2048+0 records out
1073741824 bytes (1.1 GB) copied, 3.45007 s, 311 MB/s

real 0m3.463s
user 0m0.015s

Here I am running a simple test using dd a tool that's been around for copy blocks between block devices for decades. I am testing a memory structure out to a file on the new SSD, you can condition the SSD first, but this is a simple (sanity) baseline to be sure you have partitioned the device properly with no alignment issues and you are getting a good test.

What happens when I turn off the direct flag, now I see memory and system caching take over and I get performance more along the line of the 6 GB/s SATA3 interface, but of course there is overhead in the copy process and in the software layers so we never get perfect interface speed, but you should be close (50-90%).

Here is where SATA3 speed is taking over because we are now read caching, you will find read caching works more simply, with somewhat better performance than write caching, and changing "anything" will change results but you want to baseline what is going on and understand caching here versus directio. That's the purpose here, is my system "fast enough" and am I setup to use my block devices as well as they are able.

Read Cache Test

(testing the DRAM memory, and system in this case)

$ time dd of=/dev/zero if=tstfile bs=2048k count=512
2048+0 records in
2048+0 records out
1073741824 bytes (1.1 GB) copied, 0.280394 s, 3.8 GB/s

real 0m0.296s
user 0m0.000s
sys 0m0.296s

I hope this helps you baseline your drive and system, what tool do you use to quick  baseline a new SSD or block mode device?

I tend to see the best "near specification" results when the file copy block size is set to 2048k, or 2MB. But I have not exhaustively looked into that.

Other tools of reference:

hdparm (a simple benchmark tool on Linux)  - T tests the buffering of your system , -t tests a buffered read copy (to the NAND)

fio (a Linux-based brute force load tool capable of achieving specifications for us hardware vendors)

iometer (A windows-based  brute force load tool that actually creates the Intel specifications)

Would it be useful if Intel published a blog on how we benchmark with IOmeter and fio?

Remember that the more tools you use, and to compare against the better off you are as long as those tools are standard and proven, I think everyone considers dd about as proven as it comes.

Published on Categories Archive

About Frank Ober

Frank Ober is a Data Center Solutions Architect in the Non-Volatile Memory Group of Intel. He joined 3 years back to delve into use cases for the emerging memory hierarchy after a 25 year Enterprise Applications IT career, spanning, SAP, Oracle, Cloud Manageability and other domains. He regularly tests and benchmarks Intel SSDs against application and database workloads, and is responsible for many technology proof point partnerships with Intel software vendors.