NVMe was a great thing to happen to SSDs. The per-IO reduction in latency and CPU overhead was more than welcome, as PCIe SSDs were previously using the antiquated AHCI protocol, which was a carryover from the SATA HDD days. With NVMe came additional required support in Operating Systems and UEFI BIOS implementations. We did some crazy experiments with arrays of these new devices, but we were initially limited by the lack of native hardware-level RAID support to tie multiple PCIe devices together. The launch of the Z170 chipset saw a remedy to this, by including the ability to tie as many as three PCIe SSDs behind a chipset-configured array. The recent C600 server chipset also saw the addition of RSTe capability, expanding this functionality to enterprise devices like the Intel SSD P3608, which was actually a pair of SSDs on a single PCB.
Most Z170 motherboards have come with one or two M.2 slots, meaning that enthusiasts wanting to employ the 3x PCIe RAID made possible by this new chipset would have to get creative with the use of interposer / adapter boards (or use a combination of PCI and U.2 connected Intel SSD 750s). With the Samsung 950 Pro available, as well as the slew of other M.2 SSDs we saw at CES 2016, it’s safe to say that U.2 is going to push back into the enterprise sector, leaving M.2 as the choice for consumer motherboards moving forward. It was therefore only a matter of time before a triple-M.2 motherboard was launched, and that just recently happened - Behold the Gigabyte Z170X-SOC Force!
This new motherboard sits at the high end of Gigabyte’s lineup, with a water-capable VRM cooler and other premium features. We will be passing this board onto Morry for a full review, but this piece will be focusing on one section in particular:
I have to hand it to Gigabyte for this functional and elegant design choice. The space between the required four full length PCIe slots makes it look like it was chosen to fit M.2 SSDs in-between them. I should also note that it would be possible to use three U.2 adapters linked to three U.2 Intel SSD 750s, but native M.2 devices makes for a significantly more compact and consumer friendly package.
With the test system set up, let’s get right into it, shall we?
Subject: Storage | November 26, 2015 - 07:13 PM | Jeremy Hellstrom
Tagged: M.2, pcie, sata 6Gbs, Silverstone, ECM20
That's right, no matter if you run iOS, Linux or Windows if you have an M key style M.2 port the Silverstone ECM20 add-on card will give you both a PCI-e M.2 slot and a SATA 6Gbs M.2 slot. The board itself is a mounting point, no controller but simply a way to transfer data from a PCI-e M.2 card or a mount for a SATA style card, you provide the cable. The simplcity ensures that your transfer speeds will match what you would expect from a native slot as the tests at Benchmark Reviews show. At less than $20 it is a great way to expand your high speed storage capacity.
"The m.2 form factor is becoming popular for SSDs due to its small size, and, in PCI-E guise, superior performance. As our recent test of the Samsung 950 Pro m.2 SSD has shown, PCI-E m.2 SSDs offer performance many times that of the very best SATA SSDs, so if you’re looking for a storage upgrade, m.2 is definitely the way to go."
Here are some more Storage reviews from around the web:
- ADATA XPG SX930 and Premier SP550 240GB SATA 6G SSD Review @HiTech Legion
- HyperX Predator 240GB M.2 PCIe SSD Review @ Hardware Asylum
- Understanding M.2 RAID NVMe Boot and 2/3x M.2 NVME RAID0 Tested @ The SSD Review
- Synology DS715 2-bay NAS @ techPowerUp
- QNAP TS-563 Network Attached Storage @ Modders-Inc
- ASUSTOR AS1004T 4-bay NAS Review @ Madshrimps
- Seagate Enterprise NAS 8TB SATA III HDD Review @ NikKTech
Introduction, Specifications and Packaging
What's better than an 18-channel NVMe PCIe Datacenter SSD controller in a Half Height Half Length (HHHL) package? *TWO* 18-channel NVMe PCIe Datacenter controllers in a HHHL package! I'm sure words to this effect were uttered in an Intel meeting room some time in the past, because such a device now exists, and is called the SSD DC P3608:
The P3608 is essentially a pair of P3600's glued together on a single PCB, much like how some graphics cards merge a pair of GPUs to act with the performance of a pair of cards combined into a single one:
What is immediately impressive here is that Intel has done this same trick within 1/4 of the space (HHHL compared to a typical graphics card). We can only imagine the potential of a pair of P3600 SSDs, so lets get right into the specs, disassembly, and testing!
Subject: Storage | September 22, 2015 - 02:39 AM | Allyn Malventano
Tagged: vnand, V-NAND, ssd, Samsung, pcie, NVMe, M.2 2280, M.2, 950 PRO, 512GB, 256GB
Samsung’s new product launching will be called the 950 PRO. This will be an M.2 2280 form factor product running at PCIe 3.0 x4. Equipped with Samsung’s 32-layer V-NAND and using the NVMe protocol enabled by a new UBX controller, the 950 PRO will be capable of up to an impressive 300,000 random read IOPS. Random writes come in at 110,000 IOPS and sequential throughputs are expected to be 2.5 GB/sec reads and 1.5 GB/sec for writes. Available capacities will be 256GB and 512GB.
- 256GB - $199.99 ($0.78/GB)
- 512GB - $349.99 ($0.68/GB)
- 1TB - (early next year with the switch to 48-layer V-NAND)
The 950 PRO will be shipping with a 5-year warranty rated at 200 terabytes written for the 256GB model and 400 TBW for the 512GB. That works out to just over 100GB per day for both capacities.
These hit retail in October and we currently have samples in hand for testing.
(for those curious, both capacities only have components on the front side of the PCB)
Subject: Storage | August 19, 2015 - 09:41 PM | Allyn Malventano
Tagged: IDF 2015, ocz, revodrive, RevoDrive 400, M.2, HHHL, pcie, NVMe, ssd
While roaming around at IDF, Ryan spotted a couple of new OCZ parts that were strangely absent from Flash Memory Summit:
You are looking at what is basically a Toshiba NVMe PCIe controller and flash, tuned for consumer applications and packaged/branded by OCZ. The only specific we know about it is that the scheduled release is in the November time frame. No specifics on performance yet but it should easily surpass any SATA SSD, but might fall short of the quad-controller-RAID RevoDrive 350 in sequentials.
As far as NVMe PCIe SSDs go, I'm happy to see more and more appearing on the market from every possible direction. It can only mean good things as it will push motherboard makers to perfect their UEFI boot compatibility sooner rather than later.
More to come on the RevoDrive 400 as November is just around the corner!
Subject: Storage | August 19, 2015 - 09:26 PM | Allyn Malventano
Tagged: ssd, pcie, NVMe, kingston, IDF 2015
**Edit** There was some speculation about which controller was in this SSD. It has since been solved. Here's a shot of the top of the PCB:
Now lets compare that with a shot I caught at FMS 2015 last week:
...from the Phison booth. I hadn't wirtten up my Phison post yet but this new Kingston SSD is most certainly going to be using the Phison E7 controller. Here's the placard stating some high level specs:
We saw a draft copy of Kingston’s HyperX Predator at CES 2014. That demo unit was equipped with a SandForce 3700 series controller, but since SandForce never came through on that part, Kingston had to switch gears and introduce the HyperX Predator with a Marvell 88SS9293 controller. The Marvell part was very capable, and the HyperX Predator turned out to be an attractive and performant PCIe SSD. The one catch was that Marvell’s controller was only an AHCI part, while newer NVMe-based SSDs were quickly pushing the Predator down in our performance results.
Kingston’s solution is a newer generation PCIe SSD, this time equipped with NVMe:
We have very little additional information about this new part, though we can tell from the above image that the flash was provided by Toshiba (toggle mode). They also had Iometer running:
We were not sure of the exact workload being run, but those results are in line with the specs we saw listed on Silicon Motion’s SM2260, seen last week at Flash Memory Summit.
We’ll keep track of the development of this new part and hope to see it in a more disclosed form at CES 2016. Kingston's IDF 2015 press blast appears after the break.
Subject: Storage | August 14, 2015 - 04:44 PM | Allyn Malventano
Tagged: FMS 2015, silicon motion, SM2260, SM2256, SM2246EN, pcie, NVMe, ssd, controller
We’ve reviewed a few Silicon Motion SSDs in the past (Angelbird | Corsair Force LX | Crucial BX100), and I have always been impressed with their advances in SSD controller technology. Their SM2246EN SATA controller was launched two years ago, and strived to be a very efficient and performant unit. Based on our reviews that turned out to be true, and this allowed Silicon Motion to slide into the void left by SandForce, who repeatedly delayed their newer developments and forced the many companies who were sourcing their parts to look elsewhere.
The many SSDs using Silicon Motion’s SM2246EN controller.
Silicon motion pushed this further with their SM2256, which we first saw at the 2014 Flash Memory Summit and later saw driving SLC/TLC hybrid flash at this past Consumer Electronics Show. While the SM2256 makes its way into more and more products, I was glad to see an important addition to their lineup at this year’s FMS:
Finally we see Silicon Motion doing a PCIe controller! This is the SM2260, seen here in the M.2 form factor…
…and here in SATA Express. While the latter will likely not be as popular due to the more limited PCIe lanes present in SATA Express, I’m sure we will see this controller appearing in many PCIe devices very soon. The stated performance figures may be a bit shy of currently comparing SSDs like the Intel SSD 750 and Samsung SM951, but with the recent introduction of Z170 motherboards and RST PCIe RAID, it is now easier to RAID a smaller capacity pair of these devices, increasing the performance of slower units. Further, the point of the SM2260 is likely to get a low cost NVMe PCIe SSD controller into the hands of SSD makers, which can only mean good things for those looking to make the move away from SATA.
I’ve included Silicon Motion’s FMS press blast after the break.
Subject: Storage | August 13, 2015 - 08:12 PM | Allyn Malventano
Tagged: FMS 2015, ssd, sata, SAS, pcie, NVMe, novachips, HLNAND, flash
It turns out Samsung wasn’t the only company to have 16TB SSDs at Flash Memory Summit after all:
Now that I’ve got your attention, Novachips is an SSD company that does not make their own flash, but I would argue that they make other peoples flash better. They source flash memory wafers and dies from other companies, but they package it in a unique way that enables very large numbers of flash dies per controller. This is handy for situations where very large capacities per controller are needed (either physically or logically).
Normally there is a limit to the number of dies that can communicate on a common bus (similar limits apply to DRAM, which is why some motherboards are picky with large numbers of DIMMs installed). Novachips gets around this with an innovative flash packaging method:
The 16-die stack in the above picture would normally just connect out the bottom of the package, but in the Novachips parts, those connections are made to a microcontroller die also present within the package. This part acts as an interface back to the main SSD controller, but it does so over a ring bus architecture.
To clarify, those 800 or 1600 MB/sec figures on the above slide are the transfer rates *per ring*, and Novachips controller is 8-channels, meaning the flash side of the controller can handle massive throughputs. Ring busses are not limited by the same fanout requirements seen on parallel addressed devices, which means there is no practical limit to the number of flash packages connected on a single controller channel, making for some outrageous amounts of flash hanging off of a single controller:
That’s a lot of flash on a single card (and yes, the other side was full as well).
The above pic was taken at last years Flash Memory Summit. Novachips has been making steady progress on controller development as well. Here is a prototype controller seen last year running on an FPGA test system:
…and this year that same controller had been migrated to an ASIC:
It’s interesting to see the physical differences between those two parts. Note that both new and old platforms were connected to the same banks of flash. The newer photo showed two complete systems – one on ONFi flash (IMFT Intel / Micron) and the other on Toggle Mode (Toshiba). This was done to demonstrate that Novachips HLNAND hardware is compatible with both types.
Novachips also had NVMe PCIe hardware up and running at the show.
Novachips was also showing some impressive packaging in their SATA devices:
At the right was a 2TB SATA SSD, and at the left was a 4TB unit. Both were in the 7mm form factor. 4TB is the largest capacity SSD I have seen in that form factor to date.
Novachips also makes an 8TB variant, though the added PCB requires 15mm packaging.
All of this means that it is not always necessary to have huge capacity per die to achieve a huge capacity SSD. Imagine very high capacity flash arrays using this technology, connecting a single controller to a bank of Toshiba’s new QLC archival flash or Samsung’s new 256Gbit VNAND. Then imagine a server full of those PCIe devices. Things certainly seem to be getting big in the world of flash memory, that’s for sure.
Even more Flash Memory Summit posts to follow!
Subject: Storage | August 6, 2015 - 06:37 PM | Allyn Malventano
Tagged: SSD 750, ssd, pcie, NVMe, Intel
A new 800GB SKU of the Intel SSD 750 Series of PCIe SSDs was hinted at with the Skylake launch press materials, and it appears to have been a reality:
They may not be on the shelves yet, but appearing on ARK is a pretty good indicator that these are coming soon. We don't have pricing yet, but I would suspect a cost/GB closer to the 1.2TB model than to the 400GB model, which should come in at around $700. Performance sees a slight hit for the 800GB model, likely since this is an 'uneven' number of dies for the design of the SSD DC P3500 line it was based on.
Which would you prefer - a single 800GB or a pair of 400GB SSD 750's in a RAID (now that it is possible)?
Subject: Storage | June 8, 2015 - 04:04 PM | Allyn Malventano
Tagged: U.2, ssd, SFF-8639, pcie, NVMe, Intel, computex 2015, computex
Intel has announced that the SSD Form Factor Working Group has finally come up with a name to replace the long winded SFF-8639 label currently applied to 2.5" devices that connect via PCIe.
As Hardwarezone peeked in the above photo, the SFF-8639 connector will now be called U.2 (spoken 'U dot 2'). This appropriately corresponds with the M.2 connector currently used in portable and small form factor devices today, just with a new letter before the dot.
An M.2 NVMe PCIe device placed on top of a U.2 NVMe PCIe device.
Just as how the M.2 connector can carry SATA and PCIe signaling, the U.2 connector is an extension of the SATA / SAS standard connectors:
Not only are there an additional 7 pins between the repurposed SATA data and power pins, there are an additional 40 pins on the back side. These can carry up to PCIe 3.0 x4 to the connected device. Here is what those pins look like on a connector itself:
Further details about the SFF-8639 / U.2 connector can be seen in the below slide, taken from the P3700 press briefing:
With throughputs of up to 4 GB/sec and the ability to employ the new low latency NVMe protocol, the U.2 and M.2 standards are expected to quickly overtake the need for SATA Express. An additional look at the U.2 standard (then called SFF-8639), as well as a means of adapting from M.2 to U.2, can be found in our Intel SSD 750 Review.