A little Optane for your HDD
Intel's Optane Memory caching solution, launched in April of 2017, was a straightforward feature. On supported hardware platforms, consisting of 7th and 8th generation Core processor-based computers, users could add a 16 or 32gb Optane M.2 module to their PC and enable acceleration for their slower boot device (generally a hard drive). Beyond that, there weren't any additional options; you could only enable and disable the caching solution.
However, users who were looking for more flexibility were out of luck. If you already had a fast boot device, such as an NVMe SSD, you had no use for these Optane Memory modules, even if you a slow hard drive in their system for mass storage uses that you wanted to speed up.
At GDC this year, Intel alongside the announcement of 64GB Optane Memory modules, announced that they are bringing support for secondary drive acceleration to the Optane Memory application.
Now that we've gotten our hands on this new 64GB module and the appropriate software, it's time to put it through its paces and see if it was worth the wait.
The full test setup is as follows:
|Test System Setup|
Intel Core i7-8700K
|Motherboard||Gigabyte H370 Aorus Gaming 3|
16GB Crucial DDR4-2666 (running at DDR4-2666)
Intel SSD Optane 800P
Intel Optane Memory 64GB and 1TB Western Digital Black
|Graphics Card||NVIDIA GeForce GTX 1080Ti 11GB|
|Graphics Drivers||NVIDIA 397.93|
|Power Supply||Corsair RM1000x|
|Operating System||Windows 10 Pro x64 RS4|
In coming up with test scenarios to properly evaluate drive caching on a secondary, mass storage device, we had a few criteria. First, we were looking for scenarios that require lots of storage, meaning that they wouldn't fit on a smaller SSD. In addition to requiring a lot of storage, the applications must also rely on fast storage.
Is it a usable feature?
EDIT: We've received some clarification from Intel on this feature:
"The feature is actually apart of RST. While this is a CPU-attached storage feature, it is not VROC. VROC is a CPU-attached PCIe Storage component of the enterprise version of the product, Intel RSTe. VROC requires the new HW feature Intel Volume Management Device (Intel VMD) which is not available on the Z370 Chipset.
The Intel Rapid Storage Technology for CPU-attached Intel PCIe Storage feature is supported with select Intel chipsets and requires system manufacturer integration. Please contact the system manufacturer for a list of their supported platforms."
While this doesn't change how the feature works, or our testing, we wanted to clarify this point and have removed all references to VROC on Z370 in this review.
While updating our CPU testbeds for some upcoming testing, we came across an odd listing on the UEFI updates page for our ASUS ROG STRIX Z370-E motherboard.
From the notes, it appeared that the release from late April of this year enables VROC for the Z370 platform. Taking a look at the rest of ASUS' Z370 lineup, it appears that all of its models received a similar UEFI update mentioning VROC. EDIT: As it turns out, while these patch notes call this feature "VROC", it is officially known as "Intel Rapid Storage Technology for CPU-attached Intel PCIe Storage " and slightly different than VROC on other Intel platforms.
While we are familiar with VROC as a CPU-attached RAID technology for NVMe devices on the Intel X299 and Xeon Scalable platforms, it has never been mentioned as an available option for the enthusiast grade Z-series chipsets. Could this be a preview of a feature that Intel has planned to come for the upcoming Z390 chipset?
Potential advantages of a CPU-attached RAID mode on the Z370 platform mostly revolve around throughput. While the chipset raid mode on the Z370 chipset will support three drives, the total throughput is limited to just under 4GB/s by the DMI 3.0 link between the processor and chipset.
Like we've seen AMD do on their X470 platform, CPU-attached RAID should scale as long as you have CPU-connected PCI-Express lanes available, and not being used by another device like a GPU or network card.
First, some limitations.
Primarily, it's difficult to connect multiple NVMe devices to the CPU rather than the chipset on most Z370 motherboards. Since the platform natively supports NVMe RAID through the Z370 chipset, all of the M.2 slots on our Strix Z370-E are wired to go through the chipset connection rather than directly to the CPU's PCIe lanes.
To combat this, we turned to the ASUS Hyper M.2 X16 card, which utilizes PCIe bifurcation to enable usage of 4 M.2 devices via one PCI-E X16 slot. Luckily, ASUS has built support for bifurcation, and this Hyper M.2 card into the UEFI for the Strix Z370-E.
Aiming to simplify the setup, we are using the integrated UHD 620 graphics of the i7-8700K, and running the Hyper M.2 card in the primary PCIe slot, usually occupied by a discrete GPU.
Subject: Storage | June 7, 2018 - 06:08 AM | Allyn Malventano
Tagged: toggle NAND, ssd, PCIe 3.0 x4, ONFI, NVMe, Marvell, controller, 88SS1100, 88SS1084
We've seen faster and faster SSDs over the past decade, and while the current common interface is PCIe 3.0 x4, SSD controllers still have a hard time saturating the available bandwidth. This is due to other factors like power consumption constraints of the M.2 form factor as well as the controllers not being sufficiently optimized to handle IO requests at a consistently low latency. This means there is plenty of room for improvement, and with that, we have two new NVme SSD controllers out of Marvell:
Above is the block diagram for the 88SS1100, an 8-Channel controller that promises higher performance over Marvell's previous parts. There is also a nearly identical 88SS1084, which drops to four physical channels but retains the same eight CE (chip enable) lines, meaning it can still talk to eight separate banks of flash, which should keep performance reasonable despite the halving of the physical channels available. Reducing channels to the flash helps save power and reduces the cost of the controller.
Marvell claims the new controller can reach 3.6GB/s throughput and 700,000 IOPS. Granted it would need to be mated to solid performing flash in order to reach those levels, that shouldn't be an issue as the new controllers increase compatibility with modern flash communication protocols (ONFi 4.0, Toggle 3.0, etc). Marvell's NANDEdge tech (their name for their NAND side interface) enters its fourth generation, promising compatibility with 96-layer and TLC / QLC flash.
Specs for the 8-Channel 88SS1100. 88SS1084 is identical except the BGA package drops in size to 12mm x 13.5mm and only requires 418 balls.
Rounding out the specs are the staples expected in modern SSD controllers, like OTP / Secure Drive / AES hardware crypto support, and NVMe 1.3 compliance for the host end of the interface.
While the two new parts are 'available or purchase now', it will take a few months before we see them appear in purchasable products. We'll be keeping an eye out for appearances in future SSD launches!
Subject: Storage | June 6, 2018 - 03:55 AM | Allyn Malventano
Tagged: ssd, Optane Memory, Optane, M.2 22110, M.2, Intel, 905P, 3D XPoint
At Computex 2018, Intel announced a new Optane 905P SSD:
...the Optane 905P 380GB, now in an M.2 form factor!
This looks to be a miniaturization of the 7-channel controller previously only available on the desktop add-in cards (note there are 7 packages). There is a catch though, as fitting 7 packages plus a relatively large controller means this is not M.2 2280, but M.2 22110. The M.2 22110 (110mm long) form factor may limit where you can install this product, as mobile platforms and some desktop motherboards only support up to an M.2 2280 (80mm) length. Power consumption may also be a concern for mobile applications, as this looks to be the full blown 7-channel controller present on the desktop AIC variants of the 905P and 900P.
We have no performance numbers just yet, but based on the above we should see figures in-line with the desktop Optane parts (and higher than the previous 'Optane Memory'/800P M.2 parts, which used a controller with fewer channels). Things may be slightly slower since this part would be limited to a ~7W power envelope - that is the maximum you can get out of an M.2 port without damaging the motherboard or overheating the smaller surface area of an M.2 form factor.
An interesting point to bring up is that while 3D XPoint does not need to be overprovisioned like NAND flash does, there is a need to have some spare area as well as space for the translation layer (used for wear leveling - still a requirement for 3D XPoint as it must be managed to some degree). In the past, we've noted that smaller capacities of a given line will see slightly less of a proportion of available space when comparing the raw media present to the available capacity. Let's see how this (theoretically) works out for the new 905P:
- 800P 58GB - 64GB RAW - 10%
- 800P 118GB - 128GB RAW - 8%
- 900P 280GB - 336GB RAW - 20%
- 905P 380GB - 448GB RAW - 18%
- 900P 480GB - 560GB RAW - 17%
- 905P 960GB - 1120GB RAW - 17%
I'm making an educated guess that the new 380GB part contains 4 die stacks within its packages. We've never seen 8 die stacks come out of Intel, and there is little reason to believe any would be used in this product based on the available capacity. Note that higher capacities run at ~17% excess media, but as the capacity reduces, the percentage excess increases. The 280GB 900P increases to 20% by that capacity, but the new 905P M.2 comes in at 18%. Not much of a loss there, meaning the cost/GB *should* come in-line with the pricing of the 480GB 900P, which should put the 905P 380GB right at a $450-$500 price point.
The new 905P M.2 22110 is due out later this year.
Subject: Storage | May 30, 2018 - 07:28 PM | Allyn Malventano
Tagged: ssd, QLC, Optane DC, Optane, Intel, DIMM, 3D XPoint, 20TB
Lots of good stuff coming out of Intel's press event earlier today. First up is Optane, now (finally and officially) in a DIMM form factor!:
We have seen and tested Optane in several forms, but all so far have been bottlenecked by the interface and controller architectures. The only real way to fully realize the performance gains of 3D XPoint (how it works here) is to move away from the slower interfaces that are holding it back. A DIMM form factor is just the next logical step here.
Intel shows the new 'Optane DC Persistent Memory' as yet another tier up the storage/memory stack. The new parts will be available in 128GB, 256GB, and 512GB capacities. We don't have confirmation on the raw capacity, but based on Intel's typical max stack height of 4 dies per package, 3D XPoint's raw die capacity of 16GB, and a suspected 10 packages per DIMM, that should come to 640GB raw capacity. Combined with a 60 DWPD rating (up from 30DWPD for P4800X), this shows Intel is loosening up their design margins considerably. This makes sense as 3D XPoint was a radically new and unproven media when first launched, and it has now built up a decent track record in the field.
Bridging The Gap chart - part of a sequence from our first P4800X review.
Recall that even with Intel's Optane DC SSD parts like the P4800X, there remained a ~100x latency gap between the DRAM and the storage. The move to DIMMs should help Intel push closer to the '1000x faster than NAND' claims made way back when 3D XPoint was launched. Even if DIMMs were able to extract all possible physical latency gains from XPoint, there will still be limitations imposed by today's software architectures, which still hold many legacy throwbacks from the times of HDDs. Intel generally tries to help this along by providing various caching solutions that allow Optane to directly augment the OS's memory. These new DIMMs, when coupled with supporting enterprise platforms capable of logically segmenting RAM and NV DIMM slots, should be able to be accessed either directly or as a memory expansion tier.
Circling back to raw performance, we'll have to let software evolve a bit further to see even better gains out of XPoint platforms. That's likely the reason Intel did not discuss any latency figures for the new products today. My guess is that latencies should push down into the 1-3us range, splitting the difference between current generation DRAM (~80-100ns) and PCIe-based Optane parts (~10us). While the DIMM form factor is certainly faster, there is still a management layer at play here, meaning some form of controller or a software layer to handle wear leveling. No raw XPoint sitting on the memory bus just yet.
Also out of the event came talks about QLC NAND flash. Recently announced by Intel / Micron, along with 96-layer 3D NAND development, QLC helps squeeze higher capacities out of given NAND flash dies. Endurance does take a hit, but so long as the higher density media is coupled to appropriate client/enterprise workloads, there should be no issue with premature media wear-out or data retention. Micron has already launched an enterprise QLC part, and while Intel been hush-hush on actual product launches, they did talk about both client and enterprise QLC parts (with the latter pushing into 20TB in a 2.5" form factor).
Introduction, Specifications and Packaging
ADATA has a habit of occasionally coming out of the woodwork and dropping a great performing SSD on the market at a highly competitive price. A few of their recent SATA SSD launches were promising, but some were very difficult to find in online stores. This has improved more recently, and current ADATA products now enjoy relatively wide availability. We were way overdue for an ADATA review, and the XPG SX8200 is a great way for us to get back into covering this company's offerings:
For those unaware, XPG is a computing-related sub-brand of ADATA, and if you have a hard time finding details for these drives online, it is because you must look at their dedicated xpg.com domain. Parent brand ADATA has since branched into LED lighting and other industrial applications, such as solid-state drive motor controllers and the like. Some PC products bear the ADATA name, such as USB drives and external hard drives.
Ok, enough rambling about other stuff. Let's take a look at this XPG SX8200!
Specs are mostly par for the course here, with a few notable exceptions. The SX8200 opts for a lower available capacity than you would typically see with a TLC SSD. That means a slight bump in OP, which helps nudge endurance higher due to that sacrifice. Another interesting point is that they have simply based their specs of 'up to 3200 MB/s read / 1700 MB/s write' from direct measurements of common benchmarking software. While the tests they used are 'short-run' benchmarks that will remain within the SLC cache of these SSDs, I do applaud ADATA for their openness here.
Straightforward packaging with a small bonus inside - in the form of a thermal adhesive-backed aluminum heat spreader. This is included as an option since some folks may have motherboards with integrated heat spreading M.2 socket covers or laptops with extremely tight clearances, and the added thickness may not play nicely in those situations.
Recently I came across an interesting product listing on Dell’s website for its new G3 15” gaming notebook. These are budget-friendly gaming systems with mainstream discrete GeForce graphics cards in them like the GTX 1050 and GTX 1050 Ti. Starting at just $699 they offer a compelling balance of performance and value, though we haven’t yet gotten hands on one for testing.
One tidbit that seemed off to me was this:
Several of these systems list 24GB of memory through a combination of 8GB of DDR4 and 16GB of Optane Memory for caching. A similar wording exists in the configuration page for these machines:
Clicking on the More Info link takes you to the “Help Me Choose” portion of the page that details what system memory does, how it helps the performance of your machine, and how Optane comes into the mix. There is important wording to point out that Dell provides (emphasis mine):
Some systems allow you to add Intel® Optane™ memory, which is a system acceleration solution for the 7th Gen and 8th Gen Intel® Core™ processor platforms. This solution comes in a module format and by placing this new memory media between the processor and a slower SATA-based storage devices ( HDD, SSHD or SATA SSD), you are able to store commonly used data and programs closer to the processor, allowing the systems to access this information more quickly and improve overall system performance.
Mixing DRAM with Intel® Optane™ delivers better performance and cost. For example, 4 GB DRAM + 16GB Intel® Optane™ memory delivers better performance and cost than just 8GB DRAM.
What is the difference between Intel® Optane™ memory and DRAM? Does it replace DRAM?
The Intel® Optane™ memory module does not replace DRAM. It can be, however, added to DRAM to increase systems performance.
If I use Intel® Optane™ memory with an HDD to accelerate my games, game launches and level loads become faster and close to that of an SSD experience, but what about the game play? Is the game play impacted?
Game play will not be that different between an SSD and an HDD based systems since the games in loaded into DRAM during play.
While my initial reaction of this as a clever way to trick consumers into thinking they are getting 24GB of memory in their PC when in reality it is only 8GB holds true, there are a lot of interesting angles to take.
First, yes, I believe it is a poor decision to incorporate Optane Memory into the specification of “memory” in these PCs. Optane Memory is an accelerant for system storage, and cannot replace DRAM (as the FAQ on Dell’s website states). If you have 8GB of memory, and your application workload fills that, having 16GB of memory would be a tremendous improvement in performance. Having 16GB of Optane caching on your system will only aid in moving and swapping data from main storage INTO that 8GB pool of physical memory.
Where Dell’s statements hold true though is in instances where memory capacity is not the bottleneck of performance, and your system has a standard spinning hard drive rather than an SSD installed. Optane Memory and its caching capabilities will indeed improve performance more than doubling the main system memory in instances where memory is not the limiter.
I do hope that Dell isn’t choosing to remove SSD options or defaults from these notebooks in order to maintain that performance claim; but based on my quick check, any notebook configuration that has the “24GB of memory” claim to it does NOT offer an SSD upgrade path.
Though it isn't called out one way or the other in the Dell specifications, my expectation is that they are NOT configuring these systems to use the Optane Memory as a part of the Windows page file, which MIGHT show some interesting benefits in regards to lower system memory capacity. Instead, these are likely configured with Optane Memory as a cache for the 1TB hard drive that is also a required piece of the configuration. If I'm incorrect, this config will definitely warrant some more testing and research.
Where the argument might shift is in the idea of performance per dollar improvements to overall system responsiveness. As the cost of DDR4 memory has risen, 16GB of Optane Memory (at around $25) is well below the cost of a single 8GB SO-DIMM for these notebooks (in the $80-90 range), giving OEMs a significant pricing advantage towards their bottom line. And yes, we have proven that Optane Memory works well and accelerates application load times and even level loads in some games.
But will it allow you to run more applications or games that might need or want more than 8GB of system memory? No.
Ideally, these configurations would include both 16GB of DDR4 system memory AND the 16GB of Optane Memory to get the best possible performance. But as system vendors and Intel itself look for ways to differentiate a product stack, while keeping prices lower and margins higher, this is one of the more aggressive tactics we have seen.
I’m curious what Dell’s input on this will be, if this is a direction they plan on continuing or one that they are simply trialing. Will other OEMs follow suit? Hopefully I’ll be able to get some interesting answers this week and during Computex early next month.
For now, it is something that potential buyers of these systems should pay attention to and make sure they are properly informed as to the hardware configuration capabilities and limits.
Subject: Storage | May 25, 2018 - 02:47 PM | Jeremy Hellstrom
Tagged: XPG SX8200, SM2262, NVMe, M.2, adta, 480GB
ADATA's XPG SX8200 uses the Silicon Motion SM2262 controller found in recent Intel and Mushkin M.2 SSDs, so we have an idea of its capabilities in conjunction with Micron's 64-layer 3D TLC NAND. In The Tech Reports real world testing this drive beat out Intel's 760p by a small margin in both reads and writes and it is slightly cheaper to pick up. It didn't come out as the fastest drive they've tested but it does show up near the top.
If you aren't quite sure if this drive is for you, just wait a wee bit as Al has it strapped down on his test bench right now *Allyn EDIT* our review is now live!.
"Adata's got a half-dozen NVMe M.2 drives available across its entire lineup, but its latest—the XPG SX8200—promises to dazzle with Micron's newest-gen 3D TLC and a Silicon Motion SM2262 controller. We break down the XPG SX8200 to find out if it's as good as the top dogs in the market."
Here are some more Storage reviews from around the web:
- Kingston A1000 480 GB @ TechPowerUp
- Crucial MX500 1TB M.2. @ Guru of 3D
- Crucial MX500 M.2 1 TB @ TechPowerUp
- AMD StoreMI Technology Review @ Neoseeker
- Silicon Power Bolt B80 240GB USB 3.1 Gen 2 Portable SSD Review @ NikKTech
- Netstor NA611TB3 Thunderbolt 3 External Drive Enclosure @ Kitguru
Subject: Storage | May 24, 2018 - 01:15 AM | Tim Verry
Tagged: toshiba, flash memory, fab, BiCS, 3d nand
Toshiba Memory Corporation (a subsidiary of Toshiba) is expanding its 3D flash memory production capabilities by beginning construction of a new state-of-the-art fab in Kitakami city which is in the Iwate prefecture in Japan. Toshiba Memory Corporation’s a new Toshiba Memory Iwate Corporation subsidiary began preparing for the new fab last September and construction will begin in July.
The new fab will be built with an earthquake absorbing structure and AI powered production lines with an emphasis on energy efficiency. TMIC plans to complete construction in 2019 and will hire 370 new graduates. Toshiba plans to use the new fab to boost its production capacity for its proprietary BiCS 3D flash memory to capture the massive growth market for enterprise and datacenter solid state drives. Further, Toshiba will extend its joint venture with Western Digital to include working together at the new fab.
Toshiba is quoted in the press release in stating:
“Going forward, TMC will expand its memory and SSD business and boost competitiveness by timely investments responding to market needs, and by development of BiCS FLASH™ and new generation memories.”
It is promising to see new fabs being opened and production capacities expanded by Toshiba and others (such as Micron) as it means that flash memory prices should stabilize (hopefully!), and the increased and newer production equipment will help enable the progress of new increasingly complex memory technologies.
Subject: Storage | May 21, 2018 - 04:31 PM | Allyn Malventano
Tagged: ssd, QLC, NVMe, nand, Intel, Floating Gate, flash, die, 1Tbit
In tandem with Micron's launch of their new enterprise QLC SSDs, there is a broader technology announcement coming out of Intel today. This release covers the fact that Intel and Micron have jointly developed shippable 64-Layer 3D QLC NAND.
IMFT's 3D NAND announcement came back in early 2015, and Intel/Micron Flash Technologies have been pushing their floating gate technology further and further. Not only do we have the QLC announcement today, but with it came talks of progress on 96-layer development as well. Combining QLC with 96-Layer would yield a single die capacity of 1.5 Tbit (192GB), up from the 1 Tbit (128GB) capacity of the 64-Layer QLC die that is now in production.
This new flash won't be meant for power users, but should be completely usable in a general use client SSD, provided there is a bit of SLC (or 3D XPoint???) cache on the front end. QLC does store 33% more data per the same die space, which should eventually translate to a lower $/GB once development costs have been recouped. Here's hoping for lower cost SSDs in the future!