Silicon at the Edge of Innovation
Attending an Open Compute Project Foundation (OCP) conference provides a unique look at the bleeding edge of data center innovation, and this week’s event in Dublin has provided insight across the data center landscape. The foundation of advancement starts with silicon, and I was keen to see the latest technologies that are driving system capabilities forward. Here’s our quick take on the most interesting updates from the show.
Was the Elephant in the Room?
While NVIDIA was present across the show floor in vendor booths, they have been limited in their engagement within OCP shows. The most notable impact of NVIDIA’s recent contributions was seen in OCP CEO George Tchaparian’s keynote, during which he announced an IDC forecast of $191B of global OCP infrastructure sales by 2029, up significantly from the $73.5B forecasted for 2028 shared last year. This is reflective, at least in part, of the massive impact that NVIDIA spec and technical document inclusion within the organization’s offerings represents to uplift of OCP infrastructure sales. This opportunity was echoed by the prominence of NVIDIA systems within integrated rack configurations on the show floor, and centrality of NVIDIA within the first day’s sessions. In my mind, this expansion to accelerated computing is foundational for the next generation of OCP’s existence as it takes the organization into the heart of AI compute and makes the org’s programs relevant to the next wave of AI centric data center operators like CoreWeave, Lambda, and more (more on this in an upcoming post about the neo-cloud). The OCP Foundation doubled down on the importance of AI centric configurations, rolling out a new AI marketplace this week to serve as a central information hub for adopting AI configurations.
Who’s Up Next?
Of course, this is a bleeding edge tech conference, so TechArena is always looking for who’s going to emerge to compete with NVIDIA in the AI computing game.
Enter Fractile, the AI accelerator company that claims that they can deliver 100X faster inferencing at 1/10th the cost of existing systems.
While their first-generation product silicon has yet to tape out, Fractile has targeted their designs to solve a fundamental problem of current AI processing on GPUs – time and cost wasted in movement of data between memory and processor, and the resultant intractable trade-offs for inference between lowering costs and increasing speed.
Instead, Fractile CEO Walter Goodwin explained, the company is pursuing in-memory computing that fuses key calculations into memory structures, eliminating the friction and bottlenecking of constant movement of data between processor and memory. This means faster inferencing, and it also means reduction of energy consumption at scale, both capabilities that have garnered attention from the likes of industry heavyweights Stan Boland and Pat Gelsinger.
Goodwin has assembled a seasoned team – hardware, for instance, is led by ex-NVIDIA VP Pete Hughes. Time will tell if Fractile is the player to give NVIDIA a real challenge in the market. At this point it’s clear that at least in approach, they are taking a road that gives them an opportunity to compete at hyperscale.
What about Data?
With compute performance moving at hyper-Moore’s Law, pressure is increasing on other areas of the platform to keep pace. As we wait for in-memory compute to become reality, we need to take a look at discrete memory innovation. While HBM has provided fast delivery of data for AI systems, there are power and capacity limitations that make this technology imperfect for all target applications. ZeroPoint Technology’s Nilesh Shah shared his views in an OCP session stating that LLM’s today are memory bound with HBM GPUs operating at <60% of utilization and memory inference averaging a 6:1 read to write ratio. What’s more, current lossy-based memory compression technologies degrade application accuracy. New innovation is needed, and Nilesh pointed the way with a one-two punch of breakthroughs. The first is MRAM, a new memory technology that delivers HBM like bandwidth with 30-50% less power utilization. According to MRAM provider Numem, the technology is ready for mainstream adoption. But, the full story of memory innovation potential is, when coupled with a new approach in cacheline compression technology that avoids traditional lossy intolerance to nanosecond latencies, it delivers a 1.5X improvement in memory performance. ZeroPoint has introduced AI MX 1.0 to help the industry take advantage of this new approach with on-chip decompression engines…and in the near future compressions also brought on-chip.
Are We Talking Efficiency?
I expected compute and memory advancement at OCP, but Empower Semiconductor surprised me with their new technology that improves efficiency of power delivery on the board. They’re the world’s leader in integrated voltage regulation, reducing the need for on-board capacitors by deploying FinFast-based regulators on the backside of the PCB board and greatly reducing the distance that power is traveling across the board to get to host processors. Their solutions are increasingly tapped by hyperscalers to drive down power utilization and claim a reduction of up to 20% vs. traditional capacitor technology. What’s coming next? Integration of their Crescendo solutions directly into the chip, made possible by the physical size reduction of their designs vs. competitive alternatives.
What’s the TechArena take?
The biggest story of silicon innovation at OCP was more about who was not in the room – minimal presence from NVIDIA, AMD and Intel, to name a few – as much as who was present in Dublin. While OCP configurations deliver system- and rack-level standards-based advancement, we think that silicon driven configurations were lacking, at least within this event. And as the organization grapples with broader impact beyond hyperscalers, bringing new silicon players like Fractile to the table to drive disruptive innovation to the marketplace will help unleash a new wave of innovation that the industry sorely needs.
Attending an Open Compute Project Foundation (OCP) conference provides a unique look at the bleeding edge of data center innovation, and this week’s event in Dublin has provided insight across the data center landscape. The foundation of advancement starts with silicon, and I was keen to see the latest technologies that are driving system capabilities forward. Here’s our quick take on the most interesting updates from the show.
Was the Elephant in the Room?
While NVIDIA was present across the show floor in vendor booths, they have been limited in their engagement within OCP shows. The most notable impact of NVIDIA’s recent contributions was seen in OCP CEO George Tchaparian’s keynote, during which he announced an IDC forecast of $191B of global OCP infrastructure sales by 2029, up significantly from the $73.5B forecasted for 2028 shared last year. This is reflective, at least in part, of the massive impact that NVIDIA spec and technical document inclusion within the organization’s offerings represents to uplift of OCP infrastructure sales. This opportunity was echoed by the prominence of NVIDIA systems within integrated rack configurations on the show floor, and centrality of NVIDIA within the first day’s sessions. In my mind, this expansion to accelerated computing is foundational for the next generation of OCP’s existence as it takes the organization into the heart of AI compute and makes the org’s programs relevant to the next wave of AI centric data center operators like CoreWeave, Lambda, and more (more on this in an upcoming post about the neo-cloud). The OCP Foundation doubled down on the importance of AI centric configurations, rolling out a new AI marketplace this week to serve as a central information hub for adopting AI configurations.
Who’s Up Next?
Of course, this is a bleeding edge tech conference, so TechArena is always looking for who’s going to emerge to compete with NVIDIA in the AI computing game.
Enter Fractile, the AI accelerator company that claims that they can deliver 100X faster inferencing at 1/10th the cost of existing systems.
While their first-generation product silicon has yet to tape out, Fractile has targeted their designs to solve a fundamental problem of current AI processing on GPUs – time and cost wasted in movement of data between memory and processor, and the resultant intractable trade-offs for inference between lowering costs and increasing speed.
Instead, Fractile CEO Walter Goodwin explained, the company is pursuing in-memory computing that fuses key calculations into memory structures, eliminating the friction and bottlenecking of constant movement of data between processor and memory. This means faster inferencing, and it also means reduction of energy consumption at scale, both capabilities that have garnered attention from the likes of industry heavyweights Stan Boland and Pat Gelsinger.
Goodwin has assembled a seasoned team – hardware, for instance, is led by ex-NVIDIA VP Pete Hughes. Time will tell if Fractile is the player to give NVIDIA a real challenge in the market. At this point it’s clear that at least in approach, they are taking a road that gives them an opportunity to compete at hyperscale.
What about Data?
With compute performance moving at hyper-Moore’s Law, pressure is increasing on other areas of the platform to keep pace. As we wait for in-memory compute to become reality, we need to take a look at discrete memory innovation. While HBM has provided fast delivery of data for AI systems, there are power and capacity limitations that make this technology imperfect for all target applications. ZeroPoint Technology’s Nilesh Shah shared his views in an OCP session stating that LLM’s today are memory bound with HBM GPUs operating at <60% of utilization and memory inference averaging a 6:1 read to write ratio. What’s more, current lossy-based memory compression technologies degrade application accuracy. New innovation is needed, and Nilesh pointed the way with a one-two punch of breakthroughs. The first is MRAM, a new memory technology that delivers HBM like bandwidth with 30-50% less power utilization. According to MRAM provider Numem, the technology is ready for mainstream adoption. But, the full story of memory innovation potential is, when coupled with a new approach in cacheline compression technology that avoids traditional lossy intolerance to nanosecond latencies, it delivers a 1.5X improvement in memory performance. ZeroPoint has introduced AI MX 1.0 to help the industry take advantage of this new approach with on-chip decompression engines…and in the near future compressions also brought on-chip.
Are We Talking Efficiency?
I expected compute and memory advancement at OCP, but Empower Semiconductor surprised me with their new technology that improves efficiency of power delivery on the board. They’re the world’s leader in integrated voltage regulation, reducing the need for on-board capacitors by deploying FinFast-based regulators on the backside of the PCB board and greatly reducing the distance that power is traveling across the board to get to host processors. Their solutions are increasingly tapped by hyperscalers to drive down power utilization and claim a reduction of up to 20% vs. traditional capacitor technology. What’s coming next? Integration of their Crescendo solutions directly into the chip, made possible by the physical size reduction of their designs vs. competitive alternatives.
What’s the TechArena take?
The biggest story of silicon innovation at OCP was more about who was not in the room – minimal presence from NVIDIA, AMD and Intel, to name a few – as much as who was present in Dublin. While OCP configurations deliver system- and rack-level standards-based advancement, we think that silicon driven configurations were lacking, at least within this event. And as the organization grapples with broader impact beyond hyperscalers, bringing new silicon players like Fractile to the table to drive disruptive innovation to the marketplace will help unleash a new wave of innovation that the industry sorely needs.
.png)
