Subject: Storage | August 11, 2016 - 12:06 PM | Allyn Malventano
Tagged: FMS, FMS 2016, XPoint, micron, QuantX, nand, ram
Earlier this week, Micron launched their QuantX branding for XPoint devices, as well as giving us some good detail on expected IOPS performance of solutions containing these new parts:
Thanks to the very low latency of XPoint, the QuantX solution sees very high IOPS performance at a very low queue depth, and the random performance very quickly scales to fully saturate PCIe 3.0 x4 with only four queued commands. Micron's own 9100 MAX SSD (reviewed here), requires QD=256 (64x increase) just to come close to this level of performance! At that same presentation, a PCIe 3.0 x8 QuantX device was able to double that throughput at QD=8, but what are these things going to look like?
The real answer is just like modern day SSDs, but for the time being, we have the prototype unit pictured above. This is essentially an FPGA development board that Micron is using to prototype potential controller designs. Dedicated ASICs based on the final designs may be faster, but those take a while to ramp up volume production.
So there it is, in the flesh, nicely packaged and installed on a complete SSD. Sure it's a prototype, but Intel has promised we will see XPoint before the end of the year, and I'm excited to see this NAND-to-DRAM performance-gap-filling tech come to the masses!
Subject: Storage | August 10, 2016 - 02:00 PM | Allyn Malventano
Tagged: 2.5, V-NAND, ssd, Samsung, nand, FMS 2016, FMS, flash, 64-Layer, 32TB, SAS, datacenter
..now this picture has been corrected for extreme parallax and was taken in far from ideal conditions, but you get the point. Samsung's keynote is coming up later today, and I have a hunch this will be a big part of what they present. We did know 64-Layer was coming, as it was mentioned in Samsung's last earnings announcement, but confirmation is nice.
*edit* now that the press conference has taken place, here are a few relevant slides:
With 48-Layer V-NAND announced last year (and still rolling out), it's good to see Samsung pushing hard into higher capacity dies. 64-Layer enables 512Gbits (64GB) per die, and 100MB/s per die maximum throughput means even lower capacity SSDs should offer impressive sequentials.
Samsung 48-Layer V-NAND. Pic courtesy of TechInsights.
We will know more shortly, but for now, dream of even higher capacity SSDs :)
*edit* and this just happened:
*additional edit* - here's a better picture taken after the keynote:
The 32TB model in their 2.5" form factor displaces last years 16TB model. The drive itself is essentially identical, but the flash packages now contain 64-layer dies, doubling the available capacity of the device.
Subject: Storage | August 9, 2016 - 05:59 PM | Allyn Malventano
Tagged: XPoint, Worm, storage, ssd, RocksDB, Optane, nand, flash, facebook
At their FMS 2016 Keynote, Facebook gave us some details on the various storage technologies that fuel their massive operation:
In the four corners above, they covered the full spectrum of storing bits. From NVMe to Lightning (huge racks of flash (JBOF)), to AVA (quad M.2 22110 NVMe SSDs), to the new kid on the block, WORM storage. WORM stands for Write Once Read Many, and as you might imagine, Facebook has lots of archival data that they would like to be able to read quickly, so this sort of storage fits the bill nicely. How do you pull off massive capacity in flash devices? QLC. Forget MLC or TLC, QLC stores four bits per cell, meaning there are 16 individual voltage states for each cell. This requires extremely precise writing techniques and reads must appropriately compensate for cell drift over time, and while this was a near impossibility with planar NAND, 3D NAND has more volume to store those electrons. This means one can trade the endurance gains of 3D NAND for higher bit density, ultimately enabling SSDs upwards of ~100TB in capacity. The catch is that they are rated at only ~150 write cycles. This is fine for archival storage requiring WORM workloads, and you still maintain NAND speeds when it comes to reading that data later on, meaning that decade old Facebook post will appear in your browser just as quickly as the one you posted ten minutes ago.
Next up was a look at some preliminary Intel Optane SSD results using RocksDB. Compared to a P3600, the prototype Optane part offers impressive gains in Facebook's real-world workload. Throughput jumped by 3x, and latency reduced to 1/10th of its previous value. These are impressive gains given this fairly heavy mixed workload.
More to follow from FMS 2016!
Subject: Storage | August 9, 2016 - 03:33 PM | Allyn Malventano
Tagged: XPoint, QuantX, nand, micron
Micron just completed their keynote address at Flash Memory Summit, and as part of the presentation, we saw our first look at some raw scaled Queue Depth IOPS performance figures from devices utilizing XPoint memory:
These are the performance figures from an U.2 device with a PCIe 3.0 x4 link. Note the outstanding ramp up to full saturation of the bus at a QD of only 4. Slower flash devices require much more parallelism and a deeper queue to achieve sufficient IOPS throughput to saturate that same bus. That 'slow' device on the bottom there, I'm pretty certain, is Micron's own 9100 MAX, which was the fastest thing we had tested to date, and it's being just walked all over by this new XPoint prototype!
Ok, so that's damn fast, but what if you had an add in card with PCIe 3.0 x8?
Ok, now that's just insane! While the queue had to climb to ~8 to reach these figures, that's 1.8 MILLION IOPS from a single HHHL add in card. That's greater than 7 GB/s worth of 4KB random performance!
In addition to the crazy throughput and IOPS figures, we also see latencies running at 1/10th that of flash-based NVMe devices.
..so it appears that while the cell-level performance of XPoint boasts 1000x improvements over flash, once you implement it into an actual solution that must operate within the bounds of current systems (NVMe and PCIe 3.0), we currently get only a 10x improvement over NAND flash. Given how fast NAND already is, 10x is no small improvement, and XPoint still opens the door for further improvement as the technology and implementations mature over time.
More to follow as FMS continues!
Subject: Storage | August 1, 2016 - 03:14 PM | Sebastian Peak
Tagged: M8PeG, ssd, solid state drive, preview, plextor, nand, M8Pe, M.2, CES 2016, M8PeY
Plextor announced their first M.2 SSD at CES 2016, and now the M8Pe series is officially set for a release this month. Computer Base (German language) had a chance to preview the new drive, and supplied a detailed look at the M.2 version (this is model M8PeG, and the version with a riser card is M8PeY).
The Plextor M8PeG SSD (Image credit: Computer Base)
Even the M.2 form-factor version of the SSD includes a heatsink, which Plextor warns creates incompatibility with notebooks as the M8PeG is 4.79 mm in height with the heatsink in place.
Specifications for the drives are as follows:
|Plextor M8PeG||Plextor M8PeY|
|Controller||Marvell 88SS1093 (8-Channel)|
|DRAM||512MB LPDDR3 (1024MB variant)|
|Capacity||128 GB, 256 GB, 512 GB|
|NAND||Toshiba 15nm Toggle 2.0 MLC|
|Form Factor||M.2 (80 mm)||PCIe card (HH, HL)|
|Interface||PCIe 3.0 x4|
So what did Computer Base have to report with their hands-on preview of the new drive? Here's their CrystalDiskMark result:
(Image credit: Computer Base)
Naturally we'll have to wait for a full-scale AllynReview™ to get a better idea of performance in all situations, but until then it's good to know we'll soon have another option to consider in the M.2 SSD market. As to pricing, we don't have anything just yet.
The M8Pe SSD lineup (Image credit: Computer Base)
Pre and Post Update Testing
Samsung launched their 840 Series SSDs back in May of 2013, which is over three years ago as of this writing. They were well-received as a budget unit but rapidly eclipsed by the follow-on release of the 840 EVO.
A quick check of our test 840 revealed inconsistent read speeds.
We broke news of Samsung’s TLC SSDs being effected by a time-based degrading of read speeds in September of 2014, and since then we have seen nearly every affected product patched by Samsung, with one glaring exception - the original 840 SSD. While the 840 EVO was a TLC SSD with a built-in SLC static data cache, the preceding 840 was a pure TLC drive. With the focus being on the newer / more popular drives, I had done only spot-check testing of our base 840 sample here at the lab, but once I heard there was finally a patch for this unit, I set out to do some pre-update testing so that I could gauge any improvements to read speed from this update.
As a refresher, ‘stale’ data on an 840 EVO would see reduced read speeds over a period of months after those files were written to the drive. This issue was properly addressed in a firmware issued back in April of 2015, but there were continued grumbles from owners of other affected drives, namely the base model 840. With the Advanced Performance Optimization patch being issued so long after others have been patched, I’m left wondering why there was such a long delay on this one? Differences in the base-840’s demonstration of this issue revealed themselves in my pre-patch testing:
Subject: Storage | February 14, 2016 - 02:51 PM | Allyn Malventano
Tagged: vnand, ssd, Samsung, nand, micron, Intel, imft, 768Gb, 512GB, 3d nand, 384Gb, 32 Layer, 256GB
You may have seen a wave of Micron 3D NAND news posts these past few days, and while many are repeating the 11-month old news with talks of 10TB/3.5TB on a 2.5"/M.2 form factor SSDs, I'm here to dive into the bigger implications of what the upcoming (and future) generation of Intel / Micron flash will mean for SSD performance and pricing.
Remember that with the way these capacity increases are going, the only way to get a high performance and high capacity SSD on-the-cheap in the future will be to actually get those higher capacity models. With such a large per-die capacity, smaller SSDs (like 128GB / 256GB) will suffer significantly slower write speeds. Taking this upcoming Micron flash as an example, a 128GB SSD will contain only four flash memory dies, and as I wrote about back in 2014, such an SSD would likely see HDD-level sequential write speeds of 160MB/sec. Other SSD manufacturers already recognize this issue and are taking steps to correct it. At Storage Visions 2016, Samsung briefed me on the upcoming SSD 750 Series that will use planar 16nm NAND to produce 120GB and 250GB capacities. The smaller die capacities of these models will enable respectable write performance and will also enable them to discontinue their 120GB 850 EVO as they transition that line to higher capacity 48-layer VNAND. Getting back to this Micron announcement, we have some new info that bears analysis, and that pertains to the now announced page and block size:
256Gb MLC: 16KB Page / 16MB Block / 1024 Pages per Block
384Gb TLC: 16KB Page / 24MB Block / 1536 Pages per Block
To understand what these numbers mean, using the MLC line above, imagine a 16MB CD-RW (Block) that can write 1024 individual 16KB 'sessions' (Page). Each 16KB can be added individually over time, and just like how files on a CD-RW could be modified by writing a new copy in the remaining space, flash can do so by writing a new Page and ignoring the out of date copy. Where the rub comes in is when that CD-RW (Block) is completely full. The process at this point is very similar actually, in that the Block must be completely emptied before the erase command (which wipes the entire Block) is issued. The data has to go somewhere, which typically means writing to empty blocks elsewhere on the SSD (and in worst case scenarios, those too may need clearing before that is possible), and this moving and erasing takes time for the die to accomplish. Just like how wiping a CD-RW took a much longer than writing a single file to it, erasing a Block takes typically 3-4x as much time as it does to program a page.
With that explained, of significance here are the growing page and block sizes in this higher capacity flash. Modern OS file systems have a minimum bulk access size of 4KB, and Windows versions since Vista align their partitions by rounding up to the next 2MB increment from the start of the disk. These changes are what enabled HDDs to transition to Advanced Format, which made data storage more efficient by bringing the increment up from the 512 Byte sector up to 4KB. While most storage devices still use 512B addressing, it is assumed that 4KB should be the minimum random access seen most of the time. Wrapping this all together, the Page size (minimum read or write) is 16KB for this new flash, and that is 4x the accepted 4KB minimum OS transfer size. This means that power users heavy on their page file, or running VMs, or any other random-write-heavy operations being performed over time will have a more amplified effect of wear of this flash. That additional shuffling of data that must take place for each 4KB write translates to lower host random write speeds when compared to lower capacity flash that has smaller Page sizes closer to that 4KB figure.
A rendition of 3D IMFT Floating Gate flash, with inset pulling back some of the tunnel oxide layer to show the location of the floating gate. Pic courtesy Schiltron.
Fortunately for Micron, their choice to carry Floating Gate technology into their 3D flash has netted them some impressive endurance benefits over competing Charge Trap Flash. One such benefit is a claimed 30,000 P/E (Program / Erase) cycle endurance rating. Planar NAND had dropped to the 3,000 range at its lowest shrinks, mainly because there was such a small channel which could only store so few electrons, amplifying the (negative) effects of electron leakage. Even back in the 50nm days, MLC ran at ~10,000 cycle endurance, so 30,000 is no small feat here. The key is that by using that same Floating Gate tech so good at controlling leakage for planar NAND on a new 3D channel that can store way more electrons enables excellent endurance that may actually exceed Samsung's Charge Trap Flash equipped 3D VNAND. This should effectively negate the endurance hit on the larger Page sizes discussed above, but the potential small random write performance hit still stands, with a possible remedy being to crank up the Over-Provisioning of SSDs (AKA throwing flash at the problem). Higher OP means less active pages per block and a reduction in the data shuffling forced by smaller writes.
A 25nm flash memory die. Note the support logic (CMOS) along the upper left edge.
One final thing helping out Micron here is that their Floating Gate design also enables a shift of 75% of the CMOS circuitry to a layer *underneath* the flash storage array. This logic is typically part of what you see 'off to the side' of a flash memory die. Layering CMOS logic in such a way is likely thanks to Intel's partnership and CPU development knowledge. Moving this support circuitry to the bottom layer of the die makes for less area per die dedicated to non-storage, more dies per wafer, and ultimately lower cost per chip/GB.
Samsung's Charge Trap Flash, shown in both planar and 3D VNAND forms.
One final thing before we go. If we know anything about how the Intel / Micron duo function, it is that once they get that freight train rolling, it leads to relatively rapid advances. In this case, the changeover to 3D has taken them a while to perfect, but once production gains steam, we can expect to see some *big* advances. Since Samsung launched their 3D VNAND their gains have been mostly iterative in nature (24, 32, and most recently 48). I'm not yet at liberty to say how the second generation of IMFT 3D NAND will achieve it, but I can say that it appears the next iteration after this 32-layer 256Gb (MLC) /384Gb (TLC) per die will *double* to 512Gb/768Gb (you are free to do the math on what that means for layer count). Remember back in the day where Intel launched new SSDs at a fraction of the cost/GB of the previous generation? That might just be happening again within the next year or two.
Subject: General Tech | July 30, 2015 - 02:45 PM | Ken Addison
Tagged: podcast, video, Intel, XPoint, nand, DRAM, windows 10, DirectX 12, freesync, g-sync, amd, nvidia, benq, uhd420, wasabi mango, X99, giveaway
PC Perspective Podcast #360 - 07/30/2015
Join us this week as we discuss Intel XPoint Memory, Windows 10 and DX12, FreeSync displays and more!
The URL for the podcast is: http://pcper.com/podcast - Share with your friends!
- iTunes - Subscribe to the podcast directly through the Store
- RSS - Subscribe through your regular RSS reader
- MP3 - Direct download link to the MP3 file
Hosts: Ryan Shrout, Jeremy Hellstrom, Josh Walrath, and Sebastian Peak
Program length: 1:28:34
Subject: Storage | March 26, 2015 - 02:12 PM | Sebastian Peak
Tagged: storage, ssd, planar, nand, micron, M.2, Intel, imft, floating-gate, 3d nand
Intel and Micron are jointly announcing new 3D NAND technology that will radically increase solid-storage capacity going forward. The companies have indicated that moving to this technology will allow for the type of rapid increases in capacity that are consistent with Moore’s Law.
The way Intel and Micron are approaching 3D NAND is very different from existing 3D technologies from Samsung and now Toshiba. The implementation of floating-gate technology and “unique design choices” has produced startling densities of 256 Gb MLC, and a whopping 384 Gb with TLC. The choice to base this new 3D NAND on floating-gate technology allows development with a well-known entity, and benefits from the knowledge base that Intel and Micron have working with this technology on planar NAND over their long partnership.
What does this mean for consumers? This new 3D NAND enables greater than 10TB capacity on a standard 2.5” SSD, and 3.5TB on M.2 form-factor drives. These capacities are possible with the industry’s highest density 3D NAND, as the >3.5TB M.2 capacity can be achieved with just 5 packages of 16 stacked dies with 384 Gb TLC.
A 3D NAND cross section from Allyn's Samsung 850 Pro review
While such high density might suggest reliance on ever-shrinking process technology (and the inherent loss of durability thus associated) Intel is likely using a larger process for this NAND. Though they would not comment on this, Intel could be using something roughly equivalent to 50nm flash with this new 3D NAND. In the past die shrinks have been used to increase capacity per die (and yields) such as IMFT's move to 20nm back in 2011, but with the ability to achieve greater capacity vertically using 3D cell technology a smaller process is not necessary to achieve greater density. Additionally, working with a larger process would allow for better endurance as, for example, 50nm MLC was on the order of 10,000 program/erase cycles. Samsung similarly moved to a larger process with with their initial 3D NAND, moving from their existing 20nm technology back to 30nm with 3D production.
This announcement is also interesting considering Toshiba has just entered this space as well having announced 48-layer 128 Gb density 3D NAND, and like Samsung, they are moving away from floating-gate and using their own charge-trap implementation they are calling BiCS (Bit Cost Scaling). However with this Intel/Micron announcement the emphasis is on the ability to offer a 3x increase in capacity using the venerable floating-gate technology from planar NAND, which gives Intel / Micron an attractive position in the market - depending on price/performance of course. And while these very large capacity drives seem destined to be expensive at first, the cost structure is likely to be similar to current NAND. All of this remains to be seen, but this is indeed promising news for the future of flash storage as it will now scale up to (and beyond) spinning media capacity - unless 3D tech is implemented in hard drive production, that is.
So when will Intel and Micron’s new technology enter the consumer market? It could be later this year as Intel and Micron have already begun sampling the new NAND to manufacturers. Manufacturing has started in Singapore, plus ground has also been broken at the IMFT fab in Utah to support production here in the United States.
It has become increasingly apparent that flash memory die shrinks have hit a bit of a brick wall in recent years. The issues faced by the standard 2D Planar NAND process were apparent very early on. This was no real secret - here's a slide seen at the 2009 Flash Memory Summit:
Despite this, most flash manufacturers pushed the envelope as far as they could within the limits of 2D process technology, balancing shrinks with reliability and performance. One of the largest flash manufacturers was Intel, having joined forces with Micron in a joint venture dubbed IMFT (Intel Micron Flash Technologies). Intel remained in lock-step with Micron all the way up to 20nm, but chose to hold back at the 16nm step, presumably in order to shift full focus towards alternative flash technologies. This was essentially confirmed late last week, with Intel's announcement of a shift to 3D NAND production.
Intel's press briefing seemed to focus more on cost efficiency than performance, and after reviewing the very few specs they released about this new flash, I believe we can do some theorizing as to the potential performance of this new flash memory. From the above illustration, you can see that Intel has chosen to go with the same sort of 3D technology used by Samsung - a 32 layer vertical stack of flash cells. This requires the use of an older / larger process technology, as it is too difficult to etch these holes at a 2x nm size. What keeps the die size reasonable is the fact that you get a 32x increase in bit density. Going off of a rough approximation from the above photo, imagine that 50nm die (8 Gbit), but with 32 vertical NAND layers. That would yield a 256 Gbit (32 GB) die within roughly the same footprint.
Representation of Samsung's 3D VNAND in 128Gbit and 86 Gbit variants.
20nm planar (2D) = yellow square, 16nm planar (2D) = blue square.
Image republished with permission from Schiltron Corporation.
It's likely a safe bet that IMFT flash will be going for a cost/GB far cheaper than the competing Samsung VNAND, and going with a relatively large 256 Gbit (vs. VNAND's 86 Gbit) per-die capacity is a smart move there, but let's not forget that there is a catch - write speed. Most NAND is very fast on reads, but limited on writes. Shifting from 2D to 3D NAND netted Samsung a 2x speed boost per die, and another effective 1.5x speed boost due to their choice to reduce per-die capacity from 128 Gbit to 86 Gbit. This effective speed boost came from the fact that a given VNAND SSD has 50% more dies to reach the same capacity as an SSD using 128 Gbit dies.
Now let's examine how Intel's choice of a 256 Gbit die impacts performance:
- Intel SSD 730 240GB = 16x128 Gbit 20nm dies
- 270 MB/sec writes and ~17 MB/sec/die
- Crucial MX100 128GB = 8x128Gbit 16nm dies
- 150 MB/sec writes and ~19 MB/sec/die
- Samsung 850 Pro 128GB = 12x86Gbit VNAND dies
- 470MB/sec writes and ~40 MB/sec/die
If we do some extrapolation based on the assumption that IMFT's move to 3D will net the same ~2x write speed improvement seen by Samsung, combined with their die capacity choice of 256Gbit, we get this:
- Future IMFT 128GB SSD = 4x256Gbit 3D dies
- 40 MB/sec/die x 4 dies = 160MB/sec
Even rounding up to 40 MB/sec/die, we can see that also doubling the die capacity effectively negates the performance improvement. While the IMFT flash equipped SSD will very likely be a lower cost product, it will (theoretically) see the same write speed limits seen in today's SSDs equipped with IMFT planar NAND. Now let's go one layer deeper on theoretical products and assume that Intel took the 18-channel NVMe controller from their P3700 Series and adopted it to a consumer PCIe SSD using this new 3D NAND. The larger die size limits the minimum capacity you can attain and still fully utilize their 18 channel controller, so with one die per channel, you end up with this product:
- Theoretical 18 channel IMFT PCIE 3D NAND SSD = 18x256Gbit 3D dies
- 40 MB/sec/die x 18 dies = 720 MB/sec
- 18x32GB (die capacity) = 576GB total capacity
Overprovisioning decisions aside, the above would be the lowest capacity product that could fully utilize the Intel PCIe controller. While the write performance is on the low side by PCIe SSD standards, the cost of such a product could easily be in the $0.50/GB range, or even less.
In summary, while we don't have any solid performance data, it appears that Intel's new 3D NAND is not likely to lead to a performance breakthrough in SSD speeds, but their choice on a more cost-effective per-die capacity for their new 3D NAND is likely to give them significant margins and the wiggle room to offer SSDs at a far lower cost/GB than we've seen in recent years. This may be the step that was needed to push SSD costs into a range that can truly compete with HDD technology.