Introduction
VROC vs. RST, Optane vs. 960 PRO
Introduction
We've been hearing about Intel's VROC (NVMe RAID) technology for a few months now. ASUS started slipping clues in with their X299 motherboard releases starting back in May. The idea was very exciting, as prior NVMe RAID implementations on Z170 and Z270 platforms were bottlenecked by the chipset's PCIe 3.0 x4 DMI link to the CPU, and they also had to trade away SATA ports for M.2 PCIe lanes in order to accomplish the feat. X99 motherboards supported SATA RAID and even sported four additional ports, but they were left out of NVMe bootable RAID altogether. It would be foolish of Intel to launch a successor to their higher end workstation-class platform without a feature available in two (soon to be three) generations of their consumer platform.
To get a grip on what VROC is all about, lets set up some context with a few slides:
First, we have a slide laying out what the acronyms mean:
- VROC = Virtual RAID on CPU
- VMD = Volume Management Device
What's a VMD you say?
…so the VMD is extra logic present on Intel Skylake-SP CPUs, which enables the processor to group up to 16 lanes of storage (4×4) into a single PCIe storage domain. There are three VMD controllers per CPU.
VROC is the next logical step, and takes things a bit further. While boot support is restricted to within a single VMD, PCIe switches can be added downstream to create a bootable RAID possibly exceeding 4 SSDs. So long as the array need not be bootable, VROC enables spanning across multiple VMDs and even across CPUs!
Assembling the Missing Pieces
Unlike prior Intel storage technology launches, the VROC launch has been piecemeal at best and contradictory at worst. We initially heard that VROC would only support Intel SSDs, but Intel later published a FAQ that stated 'selected third-party SSDs' would also be supported. One thing they have remained steadfast on is the requirement for a hardware key to unlock RAID-1 and RAID-5 modes – a seemingly silly requirement given their consumer chipset supports bootable RAID-0,1,5 without any key requirement (and VROC only supports one additional SSD over Z170/Z270/Z370, which can boot from 3-drive arrays).
On the 'piecemeal' topic, we need three things for VROC to work:
- BIOS support for enabling VMD Domains for select groups of PCIe lanes.
- Hardware for connecting a group of NVMe SSDs to that group of PCIe lanes.
- A driver for OS mounting and managing of the array.
Let's run down this list and see what is currently available:
BIOS support?
Check. Hardware for connecting multiple drives to the configured set of lanes?
Check (960 PRO pic here). Note that the ASUS Hyper M.2 X16 Card will only work on motherboards supporting PCIe bifurcation, which allows the CPU to split PCIe lanes into subgroups without the need of a PLX chip. You can see two bifurcated modes in the above screenshot – one intended for VMD/VROC, while the other (data) selection enables bifurcation without enabling the VMD controller. This option presents the four SSDs to the OS without the need of any special driver.
With the above installed, and the slot configured for VROC in the BIOS, we are greeted by the expected disappointing result:
Now for that pesky driver. After a bit of digging around the dark corners of the internet:
Check! (well, that's what it looked like after I rapidly clicked my way through the array creation)
Don't even pretend like you won't read the rest of this review! (click here now!)
I downloaded the guide and I
I downloaded the guide and I think that on the review you might have missed a step to configure VROC. I see that you configured the hardware for connecting multiple drives to a configured set of lanes. On the guide they set specific VMD ports through an specific OCulink connection, whatever that is. They also configured the Volume Management Device as an OCulink connection. They did the same for every CPU the processor had. I’m assuming that the ASUS board has the ability to do this with a PCIe 3.0 connection. Correct if I’m wrong but I’m assuming that any RAID array created on the RSTe GUI will run under the PCH connection if the VMD ports aren’t linked to the PCIe 3.0 connection on the BIOS.
Anyone knows where I can find
Anyone knows where I can find the VROC key and the price? Intel says “contact your mainboard manifacturer” and Gigabyte (I have a GA-X299-UD4 with 2 x Samsung 960 PRO) says “contact your dealer” but I’m the dealer and I can’t find the key!
Thank you!
Hi, I have a couple questions
Hi, I have a couple questions about bandwidth if someone can answer them for me:
1. Would I experience a bottleneck with 4 x Samsung 960 Pros if I use this card in a x8 slot rather than a x16 slot? Will it make any noticeable difference?
2. How does this card compare to the dimm.2 risers on asus boards (Rampage VI Apex & Extreme)? The riser card provides 2 PCle x4 connections directly to the cpu. Does the Hyper m.2 x 16 card have additional overhead that would cause more latency than the riser cards?
As far as I know, but without
As far as I know, but without having actual empirical experience with 4 x Samsung 960 Pros, to exploit the raw bandwidth of an x16 slot the BIOS/UEFI must support what is called PCIe lane “bifurcation”.
In the ASUS UEFI, it shows up as x4/x4/x4/x4:
https://www.youtube.com/watch?v=9CoAyjzJWfw
In the ASRock UEFI, it shows up as 4×4:
http://supremelaw.org/systems/asrock/X399/
This allows the CPU to access a single x16 slot as four independent x4 PCIe slots.
As such, even if an x8 slot were able to be bifurcated, it would end up as 2×4, or x4/x4, and the other 2 NVMe SSDs would probably get ignored.
There are some versions of these add-in cards that have an on-board PLX chip, which may be able to address all 4 SSDs even if only x8 PCIe lanes are assigned to an x16 slot by the BIOS/UEFI.
(Also, by shifting the I/O processing to the CPU, this architecture should eliminate the need for dedicated RAID IOP’s on the add-in card.)
Also, a full x16 edge connector may not fit into an x8 mechanical slot.
Ideally, therefore, these “quad M.2” AICs are designed to install in a full x16 mechanical slot that is assigned the full x16 PCIe lanes with bifurcation support in the BIOS/UEFI subsystem.
You should ask this same question of Allyn, because he will surely have more insights to share with us here.
If anyone is interested,
If anyone is interested, ASRock replied to our query with simple instructions for doing a fresh install of Windows 10 to an ASRock Ultra Quad M.2 card installed in an AMD X399 motherboard. We uploaded that .pdf file to the Internet here:
http://supremelaw.org/systems/asrock/X399/
As you want something super
As you want something super new? Take a look at this site. Only here the choice of horny for every taste and completely free! They are hardcore slaves, they will and want implement anything you command !
http://gov.shortcm.li/kings1#F25