*Updated* Benchmark Performance SSD Testing: AHCI vs IDE (or perhaps not)
** Update **
Since this article went live, we've been in contact with Benchmark Reviews and would like to point out that they have just posted an updated article using additional multi-threaded benchmarks. I'm leaving this piece up as a rebuttal to the original work, which had relied only on HDTune for the results.
** End Update **
My attention was recently directed to an article over at Benchmark Reviews. The piece intended to point out SSD performance differences when testing with different SATA protocol modes (AHCI vs. IDE). There's only one problem - their testing did not "demonstrate the difference between AHCI and IDE" at all.
The first flaw I discovered in their testing was almost immediately apparent:
- They used only a single benchmark - HDTune. HDTune is not a multi-threaded benchmark. As such, it is not capable of issuing parallel requests to the drive under test. AHCI employs Native Command Queueing (NCQ), but NCQ can not function unless requests are issued in parallel. End result - the chosed benchmark application was incapable of demonstrating the performance gains normally seen with AHCI.
- HDTune has a history of being notoriously bad for benching drives. Before the most recent update, it only accessed the first 1024GB of a drive for random access tests, meaning it would effectively short stroke drives larger than 1TB, giving seek times an artificial (and incorrect) performance boost. It's also not particularly great for benching SSD's. Write tests don't appear to be aligned to page or block boundaries, so those tests give a *very* inconsistent output. These effects are so bad that we limit its use to read testing only (burst, sequential, access time).
Breaking down the different SSD controller types tested brings an additional flaw to light, in that 2 of the 3 drives are not suitable for this type of comparative test:
- Indilinx: Indilinx controllers, while very good, take almost no advantage of NCQ at all. In the rare case that it does, it's only up to a queue depth of 4 and only with our web server benching profile under IOMeter (see chart below).
- JMicron: The new JMicron controller is leaps and bounds above their older offerings, but it does not see any performance advantage with NCQ.
- Sandforce: Of the three controller types used, Sandforce scales nicely when hit with parallel IO's and will easily scale past 10,000 IOPS in such a case. These results could not be seen simply because the testing was done with HDTune.
- Intel: The article excluded Intel. In the article comments, the author attempted to explain his exclusion of the market leader from the tests: "I didn't feel that I needed to test every SSD controller available in order to demonstrate the difference between AHCI and IDE." Lets take a step back here. The goal of the article is to determine the difference between AHCI and IDE modes, yet the single best example of an SSD to demonstrate such a thing was excluded from the results.
Here we can see the AHCI / NCQ scaling of the Sandforce (Vertex LE) and Intel SSD's. This chart also demonstrates how the Indilinx (Vertex 1.30), Samsung (Summit), and JMicron (SiliconEdge) take minimal to no advantage of parallel IO, as noted by their IOPS remaining constant regardless of Queue Depth:
IOMeter output shows AHCI / NCQ scaling of many SSD controller types.
Performance seen under IDE-only mode would be equal to Queue Depth = 1.