Welcome to
the Forum

As AI hardware evolves, bringing memory closer to compute has become a critical design priority. Carl Schlachte, chairman, president and CEO of Ventiva, calls it the investment thesis of the decade. But it requires rethinking the physical space occupied by conventional cooling systems.

Carl returned to In the Arena with TechArena’s Allyson Klein at the 2026 AI Infra Summit, following a conversation last year, when he previewed a significant demonstration planned for CES 2026.

Ventiva develops thermal management technology for applications ranging from consumer electronics to data centers. Its solid-state airflow technology offers a highly modular cooling solution, creating new opportunities for component placement and system design.

Rethinking Laptop Cooling

Conventional laptop motherboards typically accommodate two fans measuring approximately 55 millimeters in diameter, with semicircular cutouts on either side of the board. This configuration consumes valuable space that could otherwise support memory, connectivity, and other components.

At CES 2026, Ventiva demonstrated an alternative architecture featuring a rectangular motherboard with cooling concentrated toward the back. The design delivered comparable thermal performance to a conventional fan-cooled laptop while operating silently, with no moving parts.  

The improvement extended beyond acoustics and thermal performance: eliminating the fan cutouts also freed motherboard space for additional components. For Ventiva, this represented a shift from solving a thermal management challenge to enabling new possibilities in system architecture.

Generating Airflow Without Moving Parts

Ventiva uses electrohydrodynamic (EHD) flow to generate airflow without mechanical fans. Its compact rectangular ionic cooling modules recover the physical board space used by conventional fans. This gives designers the ability to add more memory, storage, battery capacity, or other new components. For local AI, that flexibility helps address a critical design constraint: finding room for more memory closer to the processor.

Why Local AI Depends on Memory Proximity

The growing importance of inference and agentic AI is changing how system architects think about memory capacity, bandwidth, and physical placement. Carl argued that many major advances in AI infrastructure, including optical interconnects and new communication architectures, address the same underlying challenge: reducing the distance and communication overhead between memory and compute.

For local inference, that constraint becomes especially significant. According to Carl, memory supporting effective on-device AI needs to sit approximately 15 to 25 millimeters from the processor complex. Conventional laptop fans compete for that space, particularly as CPU and GPU packages become larger and more power-intensive.

But processor specifications alone don't tell the full story. A laptop may offer substantial AI compute capability, measured in trillions of operations per second (TOPS), while its practical inference performance also depends on memory capacity, bandwidth, and how efficiently it can access model data.  

Ventiva’s ultra-thin modular cooling solution creates room to position additional memory closer to the processor, potentially supporting larger models without increasing the device's external dimensions.

Carl sees this as an opportunity to run more AI workloads locally rather than relying exclusively on centralized infrastructure: “You can build a data center $3,000 at a time if you did it the right way.”

The idea is to equip individual machines with sufficient compute and memory to run appropriately sized language models. Rather than sending every inference request to the cloud, users could execute suitable workloads directly on their devices, reserving remote resources for more demanding tasks.

Ventiva doesn't replace established thermal technologies such as vapor chambers or heat pipes. Its solid-state airflow works alongside them, potentially improving heat removal while freeing space for other components.

The economics could be equally significant. Local inference can reduce recurring cloud computing costs by using hardware already deployed to end users. As more workloads become practical on client devices, organizations may be able to handle routine AI tasks without paying for remote compute on every request.

From Thermal Component to Platform Architecture

Carl described a broader change in how Ventiva engages with laptop manufacturers. Rather than discussing cooling as an isolated subsystem, the company is increasingly involved in conversations about motherboard layout and overall platform design.

A rectangular motherboard can simplify manufacturing by eliminating large cutouts associated with conventional fan assemblies. It may also allow more direct routing between I/O components and the processor, reducing the need for signal repeaters and potentially lowering system costs.

By changing where cooling hardware sits, manufacturers gain greater flexibility in how they allocate space, route connections, and integrate memory.

Carl also expects local AI capabilities to influence laptop purchasing and replacement cycles. As devices handle more inference workloads, buyers may place greater value on performance and longevity. This could challenge traditional three-year refresh cycles and the role of laptops as terminals for cloud-based applications.

The external form factor may remain familiar. Carl expects the clamshell design to persist, with more substantial changes occurring inside the chassis.

Addressing the Thermal Orphans in AI Servers

Ventiva’s solid-state cooling technology also has applications in data centers. As AI server power densities increase, CPUs, GPUs, and high-bandwidth memory increasingly rely on cold plates and direct-to-chip liquid cooling. This transition reduces the need for conventional fans around high-power processors, but creates challenges for components that still depend on airflow.

In traditional server architectures, air moves from the front of the chassis toward the rear, absorbing heat as it passes through the system. Reducing fan capacity can leave downstream components with both less airflow and higher inlet temperatures. Carl calls these components thermal orphans: boot-optimized storage, SSDs, and other devices toward the rear of the server that may not justify dedicated liquid cooling but still require effective thermal management.

Failures in these components can affect the availability of the entire system. Carl noted that even a 5% failure rate in rear-mounted storage can undermine overall server availability, regardless of the performance of the GPUs at the front.  

Ventiva’s approach delivers localized airflow directly to these underserved devices without requiring major changes to existing hardware designs. Form factor is particularly important, as manufacturers of rear-mounted cards are reluctant to redesign established mechanical configurations. A cooling solution that fits within existing dimensions and delivers a 5% to 10% improvement could offer a compelling advantage.

In the server, the goal is to supplement liquid cooling, not replace it. Targeted solid-state ionic cooling modules can operate alongside cold plates, addressing thermal requirements that direct-to-chip systems do not cover.

Carl described initial skepticism from thermal engineers accustomed to designing around mechanical fans, but once they saw airflow delivered into previously impractical locations, the discussion shifted toward additional applications.

Bringing Solid-State Cooling to the Data Center

Ventiva's data center business is focused on targeted improvements rather than replacing established cooling infrastructure. Carl identified rear-of-rack components as a practical entry point, since these devices can often be replaced independently of broader server or facility upgrade cycles. This creates an opportunity to improve thermal performance without waiting for the next platform generation.

A replacement card that operates more reliably or delivers better performance within the same mechanical envelope could justify a price premium, particularly when component failures threaten server availability. For Ventiva, this offers a potentially faster path to adoption than integration into new server architectures requiring multiyear design and qualification cycles.

The TechArena Take

Ventiva's technology demonstrates how thermal management decisions increasingly influence AI system architecture.

Using ultra-thin cooling modules in laptops can free motherboard space for memory closer to the processor, addressing a critical constraint on local inference. In data centers, targeted solid-state airflow can support components that remain dependent on air even as high-power processors transition to liquid cooling.

The broader opportunity is to give system designers greater freedom to allocate space according to computational requirements rather than the dimensions of conventional cooling hardware.

The implications for local AI are particularly significant. As more inference and agentic workloads become viable on client devices, AI infrastructure could become increasingly distributed across enterprise laptops and other endpoints.

Realizing that potential requires manufacturers to consider motherboard layouts, memory integration, and thermal architecture together. Ventiva's ongoing discussions with OEMs suggest that this design conversation is already underway.

Listen to the full In the Arena episode featuring Carl Schlachte.

OCP Global Summit week opened for me with brunch and a question that has percolated in my mind all year: What does it take to run AI agents in production? MiTAC Computing hosted its Executive Tech Forum on Monday morning , and I had the pleasure of moderating a panel with Intel, SK hynix, Solidigm and MiTAC under the title Powering the Agentic Era: Compute, Memory, Storage and Scale.

Each company took a few minutes to make the case for the layer it builds. Assembled end to end, the four presentations delivered a comprehensive view on how innovation is accelerating across the system and the data center. Only in working together can we solve this open systems challenge that is becoming more urgent. Agentic AI changes the demand on every part of a rack or pod level platform at once, and the companies that treat the challenge as a single design problem will deliver to true operator requirements. Here is more of what I took from the room.

From Training Once to Agents All the Time

SK hynix America’s Vis Valluri, senior director of industry intelligence, set the frame with a simple progression. Training happens once. Inference runs in real time. Agentic AI runs all the time. He made his claim citing data from OpenRouter and a16z showing that agents overtook humans as the main source of token traffic early this year, with token usage up 14 times since.

The mechanics of agentic workflows explain this surge precisely. A regular inference request makes one pass through prefill and decode and returns a response. Simple and linear! An agent works in a loop: prefill, decode, call a tool, observe the result, then fold that result back into the context for another round of prefill. The KV cache grows with every turn. Valluri put the consequence in one number. More than 85% of agent tokens come from cached prompts, which means agents spend most of their time rereading context they already hold.

That single behavior sets up everything the other speakers had to say and defined a challenge across compute, memory and storage working tightly in tandem.

Compute Sits at the Foundation

Gary Brown of Intel opened his segment with a message that compute acts as a foundation to memory, storage, broader infrastructure (don’t forget the network!). As the resident CPU builder, he made the case for compute with throughput data.

Brown showed Intel Xeon 6 against competing CPU instances on healthcare agent workloads covering reasoning, security and control operations, laying out performance benchmarks of the alternative platforms. And while Intel was seeking a performance leadership win, the bigger takeaway for me is that in the agentic era, core functions of workload management matter a lot... and these are the realm of CPUs.

Intel’s own recent announcements argue that as AI turns agentic, the bottleneck moves toward orchestration, concurrency and data movement. Brown’s portfolio slide spread agentic scale across what Intel calls the intelligence and execution layers, with GPUs handling prefill. An agent spends much of its life between model calls, running tools and checking results. Someone has to schedule all of that, and the processor is a natural orchestrator.

Memory Gets New Floors

Valluri took Intel’s view and built from it with a memory centric perspective on the same challenge. SK Hynix sees the hierarchy being rebuilt with new tiers that fill the gaps between SRAM, HBM, DRAM and SSD. He laid them out from fastest to largest: a G0.5 tier of 3D-stacked DRAM with near-SRAM speed, HBM at G1, a G1.5 tier of stacked flash called HBF offering roughly 1 TB at terabit-per-second scale, conventional DRAM, a G2.5 tier of CXL-pooled memory in the multi-terabyte range, and SSD beneath it all.

The demand driver he chose was personal agents, pointing to Meta’s personal agent running on Muse Spark. This agent remembers details you mentioned once, keeps working after you close the app and runs in a dedicated cloud VM for each user. Persistent context that grows per user means more memory at every tier. When I think about the scale of that arithmetic across a large user base, the new tiers look like necessities rather than options, and the importance of memory and storage innovation argue to take center stage as the existential challenge of agentic performance.

The Price of a Prompt

So where was storage in all of this? Enter Solidigm’s Ace Stryker. He made the bluntest case of the discussion, using simple math as his tool. He started with a 14-word question: what’s a cool bird I might see when I’m in San Jose next week? That prompt is 18 tokens. The model processes far more. System prompt and policies account for 10,000 tokens, retrieval for 15,000, tool schemas for 5,000 and conversation history for 12,000. The total reaches 42,018 tokens. My guess is, in reality, we could argue it being even higher.

And... that context has to live somewhere. For one user making one request, Stryker calculated about 13.1 GB of KV cache, scaled from NVIDIA’s figure of 40 GB at 128K tokens for Llama 3 70B at FP16. Multiply by concurrent users and sessions and the memory bill scales quickly.

Solidigm’s answer to drive efficiency into the equation is to transfer that model state on SSD and restore it rather than recompute it. In work with Metrum AI, retaining KV cache on a Solidigm D7-PS1010 cut time to first token from 31 seconds to 2.8 seconds in a single-user sports archive scenario. In a separate video analytics test, pairing KV cache with DiskANN retrieval lifted queries per second by 14%, from 2,071 to 2,371 at 1,024 query threads. Stryker flagged these as separate results rather than a combined speedup, and I will repeat that caution here. He closed on a line that deserves a spot on every infrastructure budget review: every token has a resource cost, and every resource has a dollar cost.

Rack-Scale Reality

MiTAC Computing ’s Raymond Huang took the conversation from cutting edge silicon innovation to system delivery at scale, under the banner, “Where AI Power Meets Rack-Level Scalability.” Silicon only matters once it ships as a working system, and his slides were about that last mile and connecting acute operator requirements in a couture fashion with bespoke systems leveraging industry standards.

MiTAC cited more than 30 MW of rack-scale manufacturing power in 2026, capacity for 5,000 racks per month, networking up to 800 Gb/s and power readiness from 480 VAC up to 800 VDC. The MiTAC California footprint anchors this: It's Newark site features an NPI direct liquid-cooling facility, while San Jose provides dedicated liquid-cooling manufacturing. A mega-scale, six-building campus in Fremont further strengthens the company’s ability to design, validate, and produce advanced computing solutions. From PCB and motherboard design to barebone, systems and complete AI clusters, MiTAC delivers vertically integrated, in-house engineering that accelerates time to market and ensures end-to-end quality. On the product side, Huang showed a 52U high-density liquid-cooled rack, including a direct liquid-cooled MI355X GPU configuration, and a 256-GPU cluster example with in-row coolant distribution and a separate aggregation switch rack.

MiTAC pairs the hardware with open firmware and management software called MiOBMC™, MiOPF™ and MiCoreView™, and with ecosystem partners including DDN, Rafay and Canonical Ubuntu. A partnership with Tonomia targets modular AI data centers. Its Diamond Cooling approach aims to keep token generation throttle-free at standard data center temperatures, which could trim power and cooling costs if it performs as described.

The delivery model closed the loop. US-based engineering and project management, local system and rack integration, and a build-to-inventory approach aim to shorten lead times for customers who cannot wait out a long supply chain.

The TechArena Take

From soup to nuts, one word runs through the discussion as a throughline. Agents carry context from step to step, and context is state. State has to live somewhere, and every element of infrastructure is now being asked to hold more of it, closer to the processor and for longer. SK Hynix described the tiers that hold it. Solidigm priced it in both gigs and dollars. Intel claimed the baton with the statement that the processor orchestrates it. MiTAC showed how to package it all into racks that arrive on schedule.

Run the memory math once and the stakes come into focus. A single request can carry 13.1 GB of cache, and a personal agent maintains its memory between sessions. The capacity question becomes a per-user number multiplied by a sea of agents and humans. Size up that investment, and the case for re-tiering and deeper performance tier SSD use comes into acute focus.

Capacity plans built around model size and GPU count will miss the largest cost driver in agentic deployments. Plan around the workflow and sustained memory: how large does it grow per session, how many sessions run at once, and where each piece of it should live. Operators who answer those questions across compute, memory, storage and rack design together leveraging all of the standards the OCP community provides will build agent infrastructure that scales at speed.

The open hardware community’s flagship week produced a 256-CPU rack built for AI agents, a new inference accelerator, an 800-volt AI factory already under construction and a cooling paper with a blunt message for all of them: the industry adopts a new method only after the old one fails, and it pays for the delay.

At the 2026 OCP Global Summit in San Jose, the open hardware community moved its focus to readiness. The specifications, supply chains and skills for the next infrastructure transition are being built now, before a chip or a power grid makes them mandatory.

The CPU Returns to the Main Stage

The clearest sign of that shift came from the processors that feed AI accelerators rather than the accelerators themselves. Arm’s Mohamed Awad told the Monday keynote audience that AI agents, which he described as software that automates a person’s intent, now run as dedicated virtual machines with their own CPUs, memory and security isolation.

“An agent is just a virtual user. A virtual user who never sleeps,” Awad said.

Awad split the market into three CPU jobs: the head node feeding the accelerator, the cloud-native CPU serving users and managing infrastructure, and the agentic CPU acting as the virtual user. Arm published a white paper on CPU performance for agentic AI, developed with Fujitsu, Meta and OpenAI, and asked the OCP community to treat it as a starting point.

NVIDIA’s Ian Buck answered with hardware. The company’s Vera CPU rack, an MGX design holding 256 liquid-cooled CPUs, is now shipping, and NVIDIA contributed its third-generation MGX 1.2 rack architecture to OCP, down to the compute tray, busbar and quick disconnects. Buck said NVIDIA designed Vera for single-threaded speed because a gigawatt-scale factory of GPUs should not wait on a CPU.

AMD’s Lisa Su made the same point from the other direction, introducing Helion, an OCP-based rack for agentic AI that customers can configure CPU-only or with GPUs, with partner systems planned for November.

Accelerators still had their moment. Microsoft unveiled Maia 300, an inference accelerator the company says delivers up to 7x the rack-scale token throughput of Maia 200, with multi-gigawatt deployments planned from 2027.

Scale-Up Ethernet and the Life of Copper

Broadcom’s Asad Khamisy gave the summit its most direct example of an industry extending an incumbent technology rather than replacing it. Five or more hyperscalers now run Ethernet scale-up networks in production, he said, and the OCP Ethernet for Scale-Up Networking effort, ESUN, has grown to more than 175 organizations since its launch at last year’s summit.

On cabling, Khamisy said new modulation can double bandwidth inside a rack while holding copper’s 2-meter reach, which he described as extending the life of copper in the rack. Beyond a thousand XPUs, he said, operators are moving to hybrid designs with copper inside the rack and optics across racks. Microsoft’s Reynold D’Sa pointed to the Optical Compute Interconnect MSA, the first optical specification for multi-rack scale-up systems.

Power Climbs to 800 Volts

Power delivery produced four new specifications in a single week. Buck said the first 800-volt DC AI factory, built by Visionbay on the OCP standard now in development, cuts copper use by about 45% and scales from today’s 100-kilowatt racks toward 1 megawatt. Microsoft contributed its Mt. Diablo 800-volt architecture, which separates power conversion from the compute rack, to OCP at versions 0.7 and 1.0, and D’Sa said the company is working with utilities on a common grid-interface specification released last month.

Google, Meta, Microsoft and NVIDIA published an OCP Grid Resilience Specification that targets restoring near-full load within two seconds of a grid disturbance. Meta described its Prometheus cluster in Ohio, spanning more than 30 structures, and said prefabricated fabric buildings can bring capacity online in three to five months. OCP CEO George Tchaparian said the summit drew roughly 15,000 attendees with a 3,000-person waitlist, and the foundation expects more than 700 member organizations by year-end.

The Paper That Names the Pattern

Each of those announcements reads differently against “Thermal Boundaries in AI Infrastructure,” the 104-page paper OCP published on Oct. 12. Its authors, from Promersion, Oracle, Meta and Avayla, name the dynamic that governs cooling transitions: a new method is adopted only when the current one can no longer do the job, because each new method brings a dependency the ecosystem has good reason to defer. The paper calls this the least-disruption law.

The cost of that deferral has a name, too. When air cooling hit its physics limit on the hottest accelerators, the paper says, “only a thin layer of downstream vendors was ready for it,” and the supply of cold plates, manifolds and coolant distribution units fell short. The authors call the resulting bill “preparation debt.”

Single-phase cold plate has a physics limit of its own, projected near 4,000 watts for a reticle-class die, and the paper notes that frontier package power doubles roughly every two years. The authors’ answer is not a successor technology but a toolbox: air, cold plate, immersion, two-phase methods and microfluidics kept ready, interoperable and matched to the heat, the site and the moment.

The keynotes supplied the toolbox in practice. Microsoft’s third-generation Mt. Atlas heat exchanger nearly doubles cooling capacity in the same footprint, and the company now runs multiple independent water loops at different temperatures for a mixed fleet of liquid-cooled and air-cooled racks. NVIDIA’s MGX 1.2 contribution standardizes flow rates and quick disconnects so that OCP members can build compatible parts, Buck said, and he said assembly of the Vera Rubin compute tray dropped from 90 minutes to one minute.

The TechArena Take

The thermal paper closes with a line that could serve as the summit’s thesis: “Air cooling is finishing. Liquid cooling is just beginning. No single method wins, and the only real choice is to be ready for all of them.”

We read this year’s OCP Global Summit as the week the community put the paper’s advice into practice. The 800-volt specification, the grid resilience specification, ESUN and the OCI MSA all arrived before the parts that depend on them have shipped at volume. That timing is the readiness the paper asks for.  

The paper also names the gap that the keynotes left open. Commissioning, the step between building a multi-technology thermal system and trusting it with production load, now binds schedules as tightly as physics binds the chip, and the paper says no single vendor owns it. Operators should expect the next round of OCP work to land there, at the seam between the server and the building, where the paper says the lasting value sits. The companies that prepare that seam now will deploy when the next wall arrives. The rest will scramble, and the paper has already shown what the scramble costs.

The boundary organizations once drew around their data now has to extend to the systems that use it. Sovereign AI gives organizations a path to make those systems truly their own.

Hospitals, banks, universities, and national labs have spent decades building systems to protect sensitive data, with patient records, transaction histories, and research files secured in tightly controlled environments.

AI is now putting time-tested safeguards under pressure as organizations seek access to increasingly capable models.

This often requires entrusting proprietary material to third-party infrastructure: every prompt or document retrieved by an LLM can shift confidential information beyond the environment designed to protect it.

Sovereign AI is an approach to AI adoption that gives organizations control over their entire AI stack, including models, compute, software, orchestration, and deployment. This doesn’t merely impact where information resides, but which technologies enterprises can adopt, how they operate, and whether resources remain available as demand grows.

Sovereign AI requires control across the stack

Sovereign AI often starts with a straightforward requirement: sensitive data must remain on premises or within a specific jurisdiction. But keeping data local doesn’t automatically ensure autonomy.

Sovereignty is control across the full AI stack. If you are not owning some piece of it, then part of your intelligence is outsourced, and it is not sovereign to you.

That distinction matters because every external dependency introduces another point where an organization can lose agency. A hospital may rely on a provider’s roadmap to support clinical applications, while financial institutions can face changes in pricing or availability, and research organizations may encounter export restrictions that limit access to critical technology.

These dependencies extend beyond data storage. A company can keep its proprietary datasets on premises while relying on external systems to process them, models to interpret them, or suppliers to provide the underlying compute. Organizations therefore need to consider what happens after they secure the data:

• Who controls the model?
• Can teams inspect and tune it?
• Will the necessary equipment remain available as deployments expand?
• Can software investments survive a change in supplier strategy or export policy?

The same concern applies to future capacity. AI deployments rarely remain fixed, so organizations need confidence that the equipment supporting them will remain available as demand grows.

Otherwise, supply constraints can limit expansion, disrupt budgets, and affect long-term planning. For example, a deployment can satisfy residency requirements at launch and still create future dependencies if additional equipment becomes difficult to procure or critical software relies on a particular roadmap.

Sovereignty ensures control over where information resides and the infrastructure used to process it.

Sovereign AI changes the compute equation

Bringing the complete AI stack inside an organization only works if the infrastructure can support its workloads efficiently. For many organizations, that means reconsidering architectures modeled on hyperscale environments and GPU-dense racks, which were designed for enormous training and inference workloads.

Sovereign AI often looks different, particularly as teams adopt agentic applications that combine AI inference with substantial general-purpose computing.

Agents shuffle through multiple operations as they complete a task. They orchestrate workflows, call tools, enforce security policies, retrieve information, query databases, and generate tokens. CPUs handle many of the control-heavy steps, while AI accelerators execute the matrix operations behind model inference. Those demands can alternate throughout a single workflow as an agent moves between reasoning, retrieval, tool use, and model execution.

For sovereign AI environments, the workload mix directly affects infrastructure decisions. A platform optimized primarily for token generation can leave expensive accelerators waiting while orchestration, database queries, security checks, or other CPU-heavy operations constrain throughput. Adding more GPUs does not necessarily address those bottlenecks, either.

Overprovisioning accelerators also raises the cost of maintaining AI infrastructure. A sovereign deployment needs enough acceleration for model operations alongside the CPU capacity and memory bandwidth required to support the rest of the workflow.

Sizing each resource around actual application requirements can therefore improve utilization while keeping more of the workload within a controlled environment, so organizations can design around applications they intend to run, rather than reproducing a hyperscale architecture inside their own data centers.

Balancing compute around sovereign AI workloads

Sovereign AI infrastructure should enable mixed workloads without overprovisioning one type of compute. Epic Semi’s Contrail AIX, for example, combines general-purpose processing and AI acceleration within one platform.

Teams should always look to support the full agentic workflow within the infrastructure they control. CPUs can handle orchestration, branching, security, database operations, and tool calls while AI cores execute model workloads. A common fabric and memory pool keep both types of compute running in the same environment. For instance, Contrail AIX combines 32 performance cores with 16 AI cores connected by a shared on-chip fabric, allowing general-purpose and model operations to access a unified memory environment.

This balance also provides more flexibility as agentic applications evolve. Enterprises are still learning how production workloads distribute demand across general-purpose processing and AI acceleration. Rather than reproducing a hyperscale configuration by default, teams can size sovereign AI infrastructure around the applications they intend to run. This approach connects control over the AI stack with the practical ability to operate efficiently at scale.

Open architecture reduces long-term dependencies

Maintaining sovereignty requires isolated workflows, enforced policies, securely managed systems, and the ability to expand capacity without unnecessarily introducing new external dependencies, especially as architecture and market conditions change. Revised product roadmaps, export policy, or component availability can undermine existing investments.

Open architectures give enterprises more freedom over the technology behind their AI systems, permitting multiple vendors to develop compatible components around a shared instruction set. Teams subsequently have access to alternatives if one supplier can no longer meet their needs.

One example is RISC-V, an open instruction set architecture (ISA) used to design processors. Or, as Sherwani describes it: “RISC-V is owned by the world. Anybody can adopt it, build on it, and contribute back to it.” RISC-V International governs the specification from Switzerland, creating a shared architectural foundation beyond any single processor supplier or national jurisdiction.

The choice of architecture can also affect the longevity of investments. Organizations may spend years developing applications, optimizing software, and integrating systems, and continued access to the foundation is critical. An open ISA provides more flexibility when there are shifts in supplier strategies, availability, or other external conditions.

Openness, however, still needs to translate into deployable infrastructure, including virtualization, memory expansion, standard interfaces, secure management, and other critical capabilities for AI systems. In practice, that means pairing architectural openness with critical production server capabilities, such as full hardware virtualization, IOMMU support, and standard interfaces.

Enterprises also need a realistic path to procuring equipment, especially in regulated and geographically distributed markets. These considerations determine whether architectural openness actually supports meaningful autonomy.

Proven infrastructure supports long-term sovereignty

Sovereign AI simultaneously supports and depends on an organization’s ability to maintain control as deployment expands. That makes infrastructure maturity and long-term availability important considerations: a proven platform can reduce risk, simplify integration, and give enterprises a clearer path to additional capacity without introducing new dependencies as requirements evolve.

Production AI, however, relies on more than silicon. Operating systems, storage, networking, memory, management, and cooling all have to work together to support reliable deployment, operation, and expansion at scale. For enterprises, the broader ecosystem can shorten the path from evaluating a processor to running AI workloads on infrastructure they control.

Supply continuity provides another benefit: sovereign AI deployments can expand significantly over time, making future access to equipment part of the initial architecture decision. With greater operational control, organizations can integrate the technology into existing environments, scale capacity alongside workloads, and preserve their infrastructure investments over time.

AI sovereignty enables controlled scalability

The AI stack is already absorbing unprecedented institutional knowledge, decision-making and computing activity. As AI becomes more deeply integrated into institutional operations and agents handle increasingly complex workflows, controlling the infrastructure beneath it will become critical.

Sovereignty gives organizations a way to retain that authority: where workloads run, which models they use, how systems evolve, and how much capacity they add. This dynamic fundamentally changes the role of AI infrastructure. Decisions about compute, software, models, and supply directly affect how much control organizations retain over their AI operations.

An architecture designed around internal requirements can give teams greater freedom to adopt new capabilities, adjust resources as workloads develop, and make technology choices according to their priorities. That flexibility becomes more valuable as AI moves from discrete applications into workflows that organizations rely on for essential operational functions.

The benefits compound over time, enabling enterprises to advance without forfeiting the control they worked to establish in the first place.

Contrail AIX translates these principles into infrastructure enterprises can deploy. Specifically built for AI environments, the platform combines compute, networking, storage, and the supporting software stack into an integrated and scalable system, enabling organizations to own their intelligence.

‍

VEIR closed an oversubscribed $110 million Series C round to move its superconducting power delivery systems for AI data centers from technology validation to commercial deployment and scaled manufacturing.

The Woburn, Mass., company will use the capital to expand manufacturing capacity, advance its product portfolio and build the technical and commercial capabilities needed to support adoption at scale. It expects its first commercial deployments in 2027.

Matter Venture Partners and Tyche Partners co-led the round, joined by LG Technology Ventures, Gates Frontier, Sabanci Climate Ventures and Hui Capital. Existing investors including Engine Ventures and Congruent Ventures also participated. VEIR has now raised $225 million in total.

VEIR’s systems pair superconducting cables with cryogenic cooling, controls, termination units and monitoring. These systems move significantly more power through a smaller physical footprint and reduce labor costs, material requirements and installation complexity. VEIR has demonstrated a 3-megawatt system and is working with customers and industry partners on AI-factory reference designs.

“This financing gives us the resources to move from demonstrating the potential of superconducting power delivery to manufacturing and deploying systems at commercial scale,” said Tim Heidel, CEO of VEIR, in a press release.

On Sept. 28, I judged the Energy category of Yotta 2026’s Innovate Arena in Las Vegas, where Tim Carr, VEIR’s senior director of business development, pitched a cable that carries the same power as a copper cable tray in a conductor one-tenth the size.

I was delighted to see VEIR on the Innovate Arena stage at Yotta and to see this round announced so soon after.

TechArena Take

Yotta 2026 put power generation at the center of the AI buildout conversation. VEIR is working on delivery, where copper requires more labor, material and space as campuses grow. This round gives VEIR the capital to prove its systems in commercial deployments. We’ll be watching for its first installations.

‍

AI infrastructure contracts with hyperscalers can come with terms that are hard for smaller AI startups to navigate. On a recent podcast, I sat down with Ilkhom Sidikov, who works on the neocloud io.net’s IO Intelligence platform, to talk about how decentralized GPU networks are competing with hyperscalers by offering flexible, affordable options to AI innovators.

From Web3 Roots to an OpenRouter Listing

Ilkhom joined io.net in early 2024, not long after the company began turning a decentralized GPU network into something usable for AI startups and developers, an effort Ilkhom said got its start “in the age of Web3 and AI.”

He first worked on network infrastructure, and since 2025, his main project has been IO Intelligence, the vertical where io.net builds applications for AI startups, including model training, model inference, and video and image generation.

One recent milestone was being listed on OpenRouter, the LLM gateway many developers know. Io.net’s GPU network now hosts open-source models handling 2 billion to 5 billion tokens a day for OpenRouter.

Where Neoclouds Offer More Flexibility

Ilkhom said hyperscalers typically lock customers into one- to three-year contracts calibrated to protect their own margins, agreements that can be “really hard” on engineers and small AI startups. Under such contracts, a customer can end up paying for GPUs, like older 4090s, that fall behind newer architectures before the term ends. Even the AI grants that hyperscalers sometimes offer, he added, come with a catch. They typically require running on that provider’s own infrastructure.

Neoclouds, Ilkhom said, offer more room to negotiate, and io.net itself supports pay-as-you-go pricing for both GPU rental and LLM inference. Io.net’s specific advantage is how it negotiates use periods. Customers can rent a cluster for as little as 10 or 12 hours, and once they’re done, io.net shifts those same GPUs into its LLM inference infrastructure rather than leaving them idle.

“We never lose money on the idle GPUs,” Ilkhom said, adding that an idle GPU costs the same to run as one fully in use. That constant reallocation is what lets io.net offer clusters at a fraction of hyperscaler pricing.

Earning Enterprise Trust

Enterprise teams bring healthy skepticism to decentralized compute, particularly around reliability and security. Ilkhom said io.net encountered fraudsters early on who would alter a GPU’s metadata to make a cheaper card, like a 4090, pass as a far pricier H100.

Io.net’s fix was building software that profiles every hosted GPU on an hourly basis, checking whether each card meets performance benchmarks for what it claims to be.

On the network side, io.net connects nodes through virtual private networks (VPNs) to secure data in transit. It recently added a confidential inference feature that lets enterprise clients verify that every token generated, from the GPU level through the gateway to the back-end application, happens inside a secured environment.

Engineering for Scale and Profit

Running inference profitably at scale comes down to a handful of techniques, Ilkhom said: quantizing models down to formats like FP8 or INT4, using key-value (KV) caching to speed up throughput, and batching inference requests. Skip those optimizations, he said, and providers selling API tokens can end up burning cash, since deploying a raw model with default settings can make it hard to even break even.

Agents Raise New Demands

The rise of AI agents presents new demands on compute and data. Context windows have grown enormous, Ilkhom said, citing Anthropic’s own claim that nothing stops the company from building a model with a million-token context size. Combined with the development of MCP servers and skills that let agents pull real-time data instead of relying on static databases or vector search, agents now behave less like single inference calls and more like systems that maintain memory and reach out for information as they need it.

The TechArena Take

My conversation with Ilkhom conversation showed how decentralized providers like io.net are winning through an intensive focus on optimizing their resources. Io.net does this by keeping GPUs constantly busy rather than allowing them to sit idle between contracts.

Every technique he described, from shifting idle clusters into LLM inference to quantizing models, squeezes more usable compute out of the same hardware hyperscalers rely on too. Hourly hardware benchmarking helps build the trust enterprise customers need before relying on that hardware at all.

Ilkhom’s 5-year outlook does not predict that decentralized compute will displace the hyperscalers. After all, regulated and well-capitalized buyers have reasons beyond price to stay where they are. What his outlook does suggest is a market potentially splitting by workload, with contract-scale commitments on one side and short-duration, price-driven capacity on the other. For technology leaders, this opens up new possibilities to start matching each workload to the economics that fit it.

To learn more, visit io.net.

‍

Niklas Panten and his team at etalytics keep track of how much energy their software saves per customer and site across global deployments. Watching the numbers climb unifies the team around the company’s purpose, Panten said.

Founded in 2020 as a spinoff of Niklas’ PhD research at the Technical University of Darmstadt, etalytics builds AI software that orchestrates the cooling, heating and power systems in data centers and factories so they use less energy.

The ETA Factory

Panten, co-founder and CEO of etalytics, built his PhD on a mix of mechanical engineering, computer science and energy science. He helped develop the ETA Factory, a full-scale research factory TU Darmstadt opened in 2016 to test energy efficiency on working production lines.

His research group spent more than 10 years analyzing and optimizing energy systems in manufacturing plants. Its partners included Bosch, Volkswagen and Mercedes, along with chemical and pharmaceutical companies. The researchers used AI to run cooling, heating, ventilation, on-site generation and batteries more efficiently, work Panten said began before the AI boom.

“In our research, we found almost 50% in savings in those systems,” he said.

Two research projects at the factory, PHI-Factory and SynErgie, ran from 2014 to 2018 and produced the first ideas and prototypes of etaONE, the platform etalytics sells today, according to the company. Thomas Weber, who co-founded etalytics with Panten and is now its chief strategy officer, worked on his PhD in the same lab. The founding team, which includes CTO Björn König, came together in 2019.

“(This work) came from personal passion; I was a controls engineer and I started on power converters,” he said. “I found out that we can… make (power converters) much more efficient. But even if you have the most efficient power converter, if you have the component that has a power converter in there, and it’s not operating efficiently, you’re still losing energy.”

The same problem repeated one level up. “And then looking at the component, you can optimize the component, but if it’s not optimized or orchestrated in the system, then you’re still losing energy,” he said.

That finding is the basis of etalytics: to run at its most efficient point, a whole system has to be orchestrated. In a data center, Panten applies it to cooling “from the rooftop to the server room.”

From Factories to Data Centers

In 2020, Equinix asked the team to take its approach to data centers.

“They basically asked us, ‘Hey guys, we’ve seen your potential here, can you transfer this to data centers?’” Panten said.

At Equinix’s FR6 data center in Frankfurt, etaONE cut the energy used for cooling by 900 MWh a year and avoided 240 tonnes of CO₂ emissions a year, with a payback of less than 12 months, according to etalytics.

The company kept its industrial work. At Merck’s headquarters in Darmstadt, etaONE cut the electricity used for cooling by an average of 21%, in a project coordinated by TU Darmstadt’s ETA research group. At the Stellantis paintshop in Rüsselsheim, Germany, it cut ventilation energy use by more than 60%, in a project with the university.

“Cooling is a cross-sectional technology,” Panten said. “So, the same cooling equipment that we deploy for data centers, you will find in automotive, chemical, pharma, logistics, especially cold logistics, food processing.”

Proof at Telehouse

The company’s etaONE platform sits on top of a data center’s building management system (BMS). It streams the cooling plant’s telemetry into a physics-based digital twin of the equipment, calculates new set points every two to three minutes and writes them back to the BMS.

“We don’t rip and replace anything, we just add on top,” Panten said.

At Telehouse’s Frankfurt campus, digital twins of the cooling towers showed their performance declining. etalytics traced the decline to fouling, a buildup of deposits on the towers’ heat exchangers that reduces how well they shed heat. A heat wave was approaching, and Telehouse moved its maintenance up. Panten said the predictive maintenance product grew out of checking etalytics’ own models against the plant.

“As soon as we saw deviation, we knew it’s not our models that deviate, it’s the physics that change,” he said.

Over an assessment from April 14 to Aug. 5, 2026, etaONE adjusted the cooling tower outlet temperature and the recooling pump flow. One unit used 10.4% less electricity, and etalytics projects 497 MWh in annual savings across five units. The project won the Infrastructure Sustainability Award at the Platform Global Awards 2026.

Panten said etalytics chose physics-based models over reinforcement learning so operators know why the system makes each change.

Megawatts for Compute

Panten said energy efficiency is crucial in an industry constrained by power.

“Everyone is looking for more megawatts, so when you make megawatts available from cooling, you can put them in compute,” he said.

He pointed to research from the International Energy Agency, which projects that well-documented AI use cases could save more than 13 exajoules of energy by 2035, equivalent to 3% of global final energy consumption, if barriers to wider adoption are overcome.

Looking Ahead

etalytics opened an office in the San Francisco Bay Area in May 2026, and Panten is now based in Menlo Park, California.

Microsoft’s venture fund, M12, led an investment in etalytics in 2025. This year was the company’s first time attending Yotta, and Panten said it will have a booth next year. In October, the company plans to launch etaONE IQ, an assistant for operators’ daily work, at an event with Microsoft and NVIDIA during San Francisco Tech Week. Panten said etalytics’ AI will also move onto the coolant distribution units that serve AI factories.

TechArena Take

We see cooling orchestration as one of the fastest ways for operators to free megawatts for compute without waiting on the grid. The physics-based approach matters as much as the savings, because operators of mission-critical sites need to understand every change an AI makes before they trust it.

Customers are signing five-year contracts for AI silicon that has barely started shipping, while the models running on that hardware turn over in weeks, leaders from Crusoe, Microsoft and VAST Data said during a panel at AI Infra Summit 2026.

I had the delightful opportunity to moderate “Under the Hood of AI Infrastructure: What Top Leaders Do Differently,” with Erwan Menard, Crusoe’s senior vice president of product management; Donald Thompson, a distinguished engineer at Microsoft; and Alon Horev, co-founder and chief technology officer at VAST Data.

Across the hour, the three described an industry committing capital years ahead of knowing what it will run. Hardware gets committed in multiyear blocks and new power takes years to bring online, while the models and workloads running on them can change in weeks.

Erwan described clocks that do not agree: “I think in days. The data center guys used to think in years. Now, at places like Crusoe, they think in quarters. The energy world thinks in multiple years.”

One of Alon’s customers moves faster still, retiring the model its users are talking to every 30 minutes and publishing a newer one so that results stay fresh.

Alon called AI the “reset event of infrastructure globally,” and the technology hasn’t stopped resetting. Four themes from our discussion show where: Training and inference have folded into one continuous loop, the pace of innovation has turned into a design constraint, the megawatt has become the unit to plan around, and the data center is becoming a single computer spread across a building.

The Model That Changes Every 30 Minutes

Inference is probably bigger than training across the worldwide GPU fleet, Erwan said, and at the AI-native companies Crusoe works with, it “becomes the lion’s share” once a product reaches volume.

Open models drive a lot of inference traffic too, he said. Crusoe helps customers modify a model with their own data, put it into production and retire it on their own schedule.

“The big motivation we hear from customers is control,” he said.

Donald carried the idea further. Digital workers are trained by people inside a specific work context, and their trajectories have to improve over time, which rules out pure training and pure inference.

“You have to be in a constant learning mode,” he said.

He described sleep-time compute, fine-tuning during quiet hours, and reinforcement learning in eval environments built to replay what an agent has just gone through.

Alon’s 30-minute customer is running that pattern in production. They kept pressing his team to speed up model loading, and he asked why they needed it hundreds or even thousands of times a day.

“Because we train nonstop,” they said.

Innovation Outruns the Build Cycle

Erwan told a story about an inference engine. Someone at the show that morning claimed bragging rights for building the first one three years ago, and Erwan doubted it was still in production.

“The pace at which these things are evolving is very annoying,” he said.

Crusoe shipped its first inference service a year ago and still tunes it weekly against fresh open source, new papers and new practices. Managed Kubernetes taught him the same lesson. A general-purpose version takes thousands of engineers and years; an AI-specific one has to be excellent at a short list of things, and Crusoe’s now powers 70% of its compute fleet.

Erwan described the new pace: Research reaches production in weeks, instead of two years, and his researchers talk directly to customers, who have seen Jensen Huang’s Pareto curve and are weighing throughput against latency.

“You need to walk with them and innovate with them on the fly,” he said.

Alon agreed that standards will likely hold you back at this pace, though he sees new ones forming organically, with SNIA and various companies working together. What changed, he said, is that buyers are less afraid of proprietary technology. Before AI, buyers wanted every layer of the stack to be generic, which made it hard to innovate or change big parts of it. “And AI changed that mindset at least for now.”

Getting More From Each Megawatt

Most AI infrastructure gets bought in multiyear reservations now, Erwan said. He described five-year commitments for Vera Rubin, silicon that is only now starting to arrive. Buying that far ahead means committing to power before anyone knows what the workload will draw, and in reserved clusters already running, consumption sometimes lands well below what’s available. That gap opens room for a provider who understands both the machine’s power needs and the workload.

Closing the gap eventually means packing workloads onto shared hardware, in multitenant environments that absorb the peaks and valleys of inference traffic without stranding costly GPUs, he said. He predicted that by 2028, everyone will be optimizing for availability of megawatts.

“The energy wall is coming, and it’s going to take a lot of time to remove it,” he said.

Donald said Microsoft is working at the application and platform level to drive 24/7 inference, which is what pushes demand for megawatts.

Alon came at the same problem from the data side. He described the job of infrastructure builders as making “every megawatt, every terabyte on SSD and every CPU” serve training, fine-tuning and inference alike. He later asked whether reducing the doubled hardware some organizations carry on every stored byte could cut power along with it.

Packing workloads better only goes so far. Erwan expects power to stay short, since adjusting energy supply to demand takes years. Crusoe’s other answer runs through geography: hydro in Norway, geothermal in Iceland, and a small modular reactor paired with a modular data center that he hopes to put in service by mid-2027.

The Data Center as a Single Computer

Alon called inference “a clustered problem,” with prefill and decode disaggregated so that “some GPUs do this; other GPUs do that.” Every open session ties up a block of GPU memory for as long as it lasts. That block is the KV cache. In some models, contexts of a million tokens push it to “tens or even hundreds of gigabytes,” crowding out other users, so it gets moved to a storage system or an SSD on another host. Routing then has to send that user back to whichever machine is holding their cache, so no single GPU completes the job alone.

“Everything is integrated into one logical computer that’s doing inference, … and KV cache is becoming a core component of it,” Alon said.

Eighteen to 24 months ago, Crusoe sized its data centers at four to five terabytes of storage per GPU, Erwan said. Today it is north of 10, and a given inference proof of concept takes roughly 20% fewer GPUs than a year ago.

He also wants the agent’s memory and context treated as its own layer of the stack. There is too much of it to hold in expensive memory, so some has to move to cheaper storage, and he sees “true infrastructural challenges in delivering that context.”

We should look at a data center now, Alon said at the end of the hour, as “a single computer just distributed across so many different machines and components.”

The TechArena Take

Hardware and power get committed for years. The models running on them change in weeks. That is the mismatch, and the panel treated it as a purchasing problem. Most of their answers were things to buy: longer silicon reservations, sites in Norway and Iceland, a reactor, more megawatts by 2028. One answer was operational, packing more work onto hardware already installed and already paid for, and it got less time.

That ordering is backward for anyone who has already signed. A five-year contract cannot be renegotiated into a different clock. What can still change is how the building runs: what gets packed where, what waits for a cheaper hour, what the machines actually draw against what the spec promised.

Donald raised this on stage, asking how intelligence gets applied to running the data center itself instead of leaving a person at a dashboard. Nobody answered. That is the question TechArena would put to vendors this year: What does the cluster do on its own when the model changes in six weeks? The answer may decide who is still competitive in 2028, when Erwan expects megawatt availability to be what everyone optimizes.

For decades, the industry has named walls: the memory wall, the compute wall, the network wall, the cooling wall, the data wall. The list keeps growing. This week, more than 6,000 people converged on Las Vegas for Yotta 2026 and unpacked the wall beneath all of them: power. Without power, none of it functions.

Across three days, the builders, suppliers and founders on Yotta’s stages and on the expo hall floor made the case: The industry can deploy only as much AI as it can power, so every gain in compute, cooling, storage, networking and I/O now counts in tokens per second per watt.

The Power Gap

Varun Sivaram, CEO and founder of Emerald AI, told the Crusoe Main Stage audience on Tuesday morning how much of the AI factory capacity planned in the United States will lack grid power to run it.

“If I put some numbers to it, between now and 2028, the United States seeks to connect something like 70 gigawatts of AI factories,” he said. Less than half of that will successfully plug into the grid, he added. “There's a massive supply-demand gap just on the power side.”  

Sarfraz Taj, vice president of business development at Terrestrial Energy, opened his Innovate Arena pitch on Monday with the total capacity of data center projects now planned in the United States.

“So, Vegas is all about numbers. Let me start with some numbers: 300 gigawatts. That’s the capacity of data center projects currently planned in the U.S.,” Taj said.

The first response to the gap is already used up, he explained. Some hyperscalers and developers went directly to power producers, restarted shuttered nuclear plants and pushed existing plants to higher output.

“Against a 300 gigawatt plant capacity for data centers, that low-hanging fruit is now all but virtually gone, so not a viable way to get additional electricity,” he said.

Chris Lander, vice president of XFRA at Span.io, described how long the shortage of grid connections now adds to a data center build. He listed interconnect queue delays, transmission line build-outs, lead times for substations and transformers, and public opposition in local communities.

“A year ago, it would take around three years to build a data center for AI,” he said. “We’re seeing that three years already extend to five years.”

Solar Power at Night, From Orbit

The farthest source of new power pitched at Yotta sits in orbit. Abdullah Al-Shakarchi, co-CEO of Overview Energy, presented it in the Innovate Arena on Monday.

“We are in the business of selling invisible light,” Al-Shakarchi said.

Overview Energy places satellites in geosynchronous orbit, where they receive sunlight around the clock. The satellites convert sunlight to electricity and use it to power near-infrared lasers. The lasers send an invisible beam to solar farms on the ground, so those farms produce power at night.

Al-Shakarchi said a few hundred satellites could power a single 300- to 400-megawatt training data center. The satellites can also redirect their beams. A satellite serving Las Vegas could send its energy to Morocco within minutes, he said.

“This is the world’s only movable power plant,” he added.

His company’s approach turns solar from a source that produces power 35% of the time into one that produces 70% to 90% of the time, using solar farms that already exist. Meta, one of the company’s first customers, announced a gigawatt agreement with Overview Energy in April to power its data centers from space. Al-Shakarchi said the company plans to reach orbit next year, power a U.S. utility-scale solar project in 2028 and serve its first customers in 2029.

Trevor Smith, CEO of Atomic-6, pitched a different use of orbit: data centers that operate in space. Atomic-6 builds solar arrays, radiators and debris shielding for those satellites. Smith said orbital data centers will stay small, in nodes of 100 to 200 kilowatts that connect to one another.

“You’re not going to see Costco-sized data centers flying around in space,” he said. “It just doesn’t make sense.”  

The Grid Already Built

Vladimir Troy, vice president of AI infrastructure at NVIDIA, said the nearest source of new power is grid capacity the country has already built. Emerald AI’s Sivaram interviewed Troy on Tuesday’s main stage. Troy said utilities build the grid to meet peak demand, and demand stays below that peak most of the time.

“Like 99% of the time, (it’s) just sitting there and being underutilized,” Troy said.

An AI factory that reduces its power draw during peak hours can use that capacity, he added. Sivaram said the U.S. grid has 100 gigawatts of available capacity today for AI factories that can operate that way. Emerald AI builds the software that lets AI factories adjust their power use. NVIDIA, Google and Emerald AI launched the AI Energy Management Alliance on Sept. 17 to advance that approach.

New Power on Site

Sites that cannot wait for a grid connection need power generated and delivered on the campus itself. Founders and suppliers at Yotta pitched both.

Emerson Reiter, chief commercial officer and co-founder of Teragen Energy, pitched fuel cells. Teragen builds one-megawatt units that fit in a 20-foot shipping container.

“Primarily, fuel cells turn molecules to megawatts without combustion,” Reiter said.

Teragen’s cells can ramp from zero to full power in minutes rather than hours, he added. The company plans commercial deployments in 2028.

Vince Canino, president and CEO of Capstone Energy+, told TechArena that the company's gas turbines can serve a data center from groundbreaking to full operation. Each turbine has a single moving part that rides on a cushion of air, so it runs without oil, coolants or water, he said.

"Because we can move our equipment around very easily, we can start out as construction power," Canino said. "As construction is done, we can move the equipment to bridge power, and then we can put it in its final permanent position as primary power to the data center."

Constantin Eis, CEO of CMBlu Energy, pitched a water-based organic flow battery that cannot catch fire. He compared the U.S. market to Germany, his home market, which is closing power plants.

“Actually, I think the capital is there, the land is there, customers are there. The only problem which is missing is the energy,” he said.

Power generated on site still has to reach the racks. Tim Carr, senior director of business development at Veir, pitched a superconducting cable that carries the same power as a copper cable tray in a conductor one-tenth the size.

“We believe that power delivery, not just power generation, is the next necessary evolution,” he said.

Melton Chang, executive vice president of power systems at Schneider Electric, told TechArena that the company is replacing the wiring and custom hardware in medium-voltage switchgear with software. He said the change shortens time to power and moves most commissioning from the job site to the factory.

“Software-defined is not just about one product,” he said. “Software-defined is really about the system.”

More Tokens From Each Watt

Once power reaches a site, the next question is how much of it becomes tokens. Peter Panfil, vice president of power architecture and application at Vertiv, said a 100-megawatt site running at industry-norm efficiency puts about 60 megawatts to productive work at the GPU. He also cited a GPU utilization rate of about 70% that he heard three times at the conference.

“That’s a pretty damn expensive sports car to only be able to run at 70% of its capacity,” Panfil said during the Tokens Per Watt panel on Wednesday, referring to GPU capacity left unused.

Sean Corcoran, vice president of sales for PowerOne at Airsys, discussed a metric called PCE: the IT load a site provisions divided by the building load it provisions.

“It’s all about finding that waste of power and efficiently using it to compute more and more,” he said.

Power spent on cooling produces no tokens. Matthew Orcutt, vice president of strategy and systems product management at Trane Technologies, told TechArena that NVIDIA’s support for 45°C facility water lets a data center cool its GPUs with dry coolers, without chillers or compressors, for most hours of the year. LiquidStack, a Trane Technologies company, launched a coolant distribution unit at Yotta that supports that temperature. Orcutt said the amount of water a data center uses is a separate design decision.

Storage Joins the Math

Storage design also affects how much work each watt produces. On TechArena’s Tuesday panel, Angel Camacho, director of product marketing at Western Digital, said the industry first answered shortages by buying more storage and more power.

“Until suddenly, we hit a wall. It becomes too expensive,” he said.

Marc Austin, CEO and co-founder of Hedgehog, described a customer demonstration at Computex. One GPU ran an inference workload in 90 seconds. The same GPU, paired with a second accelerator, ran it in 30 seconds.

"Do the kind of simple math on that and it’s six times the tokens per watt, which is huge,” he said.

Betsy Chernoff, principal AI product marketing manager at WEKA, said storage has joined that calculation only recently.

“When you get over to the inference side of the house, you can have a fully utilized GPU, and still have it incredibly inefficient,” she said.

The Side Yard

Some companies took the opposite approach and moved the compute to places where power already exists. Span, the smart electrical panel company Lander represented, places liquid-cooled inference nodes in homeowners’ side yards, next to the electrical panel. The nodes run on capacity the home does not use, and the homeowner receives subsidized energy and internet in return.

“Instead of waiting for power and traditional data center infrastructure, organizations are increasingly looking at how they can take the compute to the power,” said Angie Keeler, co-founder and CEO of Zella DC.

Submer Group launched Corenix, which builds and tests complete data center modules in a factory before shipping them to a site.

The Neighbors

Plans to build on the ground require approval from the community around the site. Niklas Panten, co-founder and CEO of etalytics, whose software reduces the power that data center cooling plants use, said he understands the objection.

“To be honest, I don’t want (a data center) in my backyard either," he said.

Allyson Klein, founder and CEO of TechArena, moderated Wednesday’s Tokens per Watt panel and gave its final thought.

“I think that we’re going to see the focus on manageability, and how we create a control plane for operators that not just takes into account the compute infrastructure, but also storage and networking, as well as potential discussions about control of your power and cooling equipment from an integrated pane of glass,” she said. “I think that those things can provide us an ability to drive up every metric that was talked about on stage today and will bring this industry closer together. What I’m excited about is actually being here to see it all happen… This industry is incredible at innovating itself out of a problem.”

TechArena Take

We came away from Yotta 2026 seeing the power wall as a design problem as much as a supply problem. The speakers with the most concrete answers treated power, cooling, compute, storage and networking as one system, and they measured that system by the tokens it delivers per watt. Overview Energy took the other approach: it adds new hours of output to solar farms that already exist. We expect both approaches to grow.

We see tokens per second per watt as the measure to watch. Operators who design their sites around it will add AI capacity in the next year without waiting for a place in the interconnect queue. The operators who win community approval will be the ones who can show their neighbors a stronger grid and lower rates.

‍

Why enterprise infrastructure must optimize the entire workflow, not only model inference

A recent study of five representative agentic AI workloads found that the dominant bottleneck shifted from retrieval to code execution, scientific tools, summarization, and model inference. In several tests, work outside the model consumed more than 80% of end-to-end latency.

That matters because an AI agent does much more than running a model. It plans, retrieves data, calls tools, updates its state and repeats the process until the task is complete. Each stage stresses a different part of the system.

For banks, hospitals, universities and national labs that buy and run their own infrastructure to keep data on premises, optimizing only the accelerator may only yield very limited impact on the entire workflow, or even worse, leave your most expensive and power-hungry accelerator waiting.

The bottleneck moves

The study, “Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective,” published in November 2025, instrumented the five workloads end-to-end across two systemsi. The results do not converge. In each workload, a different stage consumed the largest share of end-to-end latency:

Selected results from two systems and multiple benchmarks. The dominant stage changes with workload inputs and hardware configuration.

• Only in the last case is the bottleneck the matrix math the industry has been building for.

For enterprise infrastructure teams, that ratio changes the buying decision. The bottleneck moves with the work, and a production deployment runs several kinds of work at once. No single resource can be scaled up to fit the whole fleet. Take the retrieval-augmented pipeline from the study, where nearest-neighbor search accounted for more than 80% of the time and runs on CPU cores. Buying more accelerators speeds up the model inference in the remaining share. The search step, which decides how long the whole job takes, does not get faster at all.

One agent needs several kinds of compute

Look one layer down and the same pattern holds. At the model level, prompt prefill benefits from high parallel compute throughput, while token-by-token decode depends more heavily on memory bandwidth and capacity. Outside the model, data processing, tool calls, orchestration and security run efficiently on general-purpose cores. An agent moves between these phases continuously.

Balanced systems are becoming a mainstream design priority. Intel CEO Lip-Bu Tan told the Computex 2026 audience that agentic AI is “returning the CPU to a position of prominence,”ii and AMD made the same case for CPUs in the AI data center earlier in the year.iii The single-silicon era of AI infrastructure is closing.

Why we built Contrail AIX

Organizations and nations that desire to gain greater control over how AI is deployed, customized and governed keep their data on premises, deploy open-weight models and build infrastructure for the complete agent workflow deployable inside their own walls. That is what Epic Semi designed Contrail AIX to be.

What we built to run AI agents

Contrail Compute, the RISC-V CPU and AI superchip inside our Contrail AIX platform, puts 32 high-performance RISC-V cores and 16 RISC-V AI acceleration cores on one die, sharing one memory and storage space across a common on-chip fabric.

This architecture keeps retrieval, orchestration, model execution, and tool calls close to the same agent state, reducing unnecessary data movement between separate devices. Each Contrail Compute chip can serve as a native agentic AI execution unit, and these units are small, complete, and fungible. Small means granular enough to scale independently. Complete means useful work can run locally. Fungible means any unit can be scheduled, replicated, drained, or replaced through a common operating model.

Agent-aware orchestration compiles the workflow into a physical plan, places each fragment on the right resources, preserves data locality, and continuously resizes the fleet. Units pool when demand rises, fail over automatically, and disappear when no longer needed.

We built it on RISC-V so a customer’s software investment outlives any single vendor’s roadmap and any single government’s export policy. Contrail AIX carries a 5A992C export classification, which reaches substantially all markets. It supports the RISC-V hypervisor extension with full hardware virtualization and IOMMU, RVA23-compatible architecture, trusted secure boot and full-chip security management. Ubuntu came up shortly after first silicon, and llama.cpp and open-weight models are running today. Compute is the first product on a multi-generation roadmap, from a team that has ramped millions of microprocessors.

Agentic benchmarking is underway. We will publish performance results after optimization and validation so customers can evaluate defensible end-to-end measurements rather than synthetic claims.

If you operate AI on premises and want control of your data, bring us your workflow bottlenecks. We’ll help you measure them end-to-end on Contrail AIX.
‍

[1] Ritik Raj, Souvik Kundu, Ishita Vohra, Hong Wang and Tushar Krishna, "Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective," arXiv:2511.00739, submitted Nov. 1, 2025, revised April 16, 2026, https://arxiv.org/abs/2511.00739.

[1] Intel Newsroom, "Computex 2026: An Intelligent World Built on Silicon," June 2026, https://newsroom.intel.com/artificial-intelligence/computex-2026-an-intelligent-world-built-on-silicon. Quote as reported in Divyanshi Sharma, "Computex 2026: Intel announces Xeon 6+ processors, says AI will make CPUs important again," Digit, June 2, 2026,

[1] AMD, "Agentic AI Brings New Attention to CPUs in the AI Data Center," AMD corporate blog, March 13, 2026, https://amd.com/en/blogs/2026/agentic-ai-brings-new-attention-to-cpus-in-the-ai-data.html.

‍

VAST Data today introduced VAST DataEnclave, a confidential AI runtime that lets model builders place proprietary models inside a customer's own data center and run them against regulated data. The capability, built on NVIDIA Confidential Computing, previews today and ships in Q1 2027 through VAST and OEM partners including Cisco and Supermicro.

DataEnclave keeps data and model weights encrypted while they are in use, and releases decryption keys only after it verifies the environment those assets are about to enter. That gives an enterprise a way to apply a frontier model to data it cannot move, and gives a model builder a way to deploy weights into infrastructure it does not own.

The Gap at Execution

Encryption already covers model weights on storage and across the network. The gap opens at execution. To serve a prompt, weights decrypt into GPU memory, where the infrastructure's administrators, a compromised hypervisor, other tenants sharing that memory and firmware exploits can all reach them.

Enterprises and model builders each give something up to that exposure. Banks, hospitals and government agencies hold data they cannot ship to a hosted AI service, so the best models never see it. Model builders treat base weights as their most valuable asset and keep them on a short list of trusted platforms, which puts air-gapped and sovereign facilities out of reach. VAST, citing Synergy Research, puts about 50% of the world's data somewhere frontier models cannot go today.

Verify First, Decrypt Second

DataEnclave adds a secure container runtime and an attestation service inside the VAST DataEngine.

Hardware-isolated execution runs workloads in confidential virtual machines and containers, using trusted execution environments from Intel, AMD and NVIDIA Vera to encrypt guest memory, GPU memory and NVLink traffic. Active data and models stay isolated from infrastructure administrators and from other tenants on the same hardware.

Verify-before-decrypt attestation checks the trusted execution environment, including NVIDIA GPU attestation, before any decryption key is released. VAST states the rule plainly in its briefing: "No attestation. No access."

Independent key control lets each party hold its own keys in its own trust domain through bring-your-own KMS integrations. The model builder protects base weights. The enterprise protects its data and the fine-tuned weights that increasingly carry its own IP.

That key control is what VAST says separates DataEnclave from the confidential VMs cloud providers already sell. The weight owner verifies the platform itself, keeps its own keys and gets the same arrangement whether the deployment sits on-premises, in a cloud or behind an air gap.

VAST logs attestation events, key releases and enclave lifecycle actions to a tamper-proof audit trail in the VAST DataBase. Deployments run connected or fully air-gapped, on the open CNCF Trustee stack or on Fortanix confidential AI infrastructure. The same runtime gives agents isolated sandboxes through VAST AgentEngine.

More Than 20 Partners at Launch

Cohere, CrowdStrike, Deepgram, Factory, Fundamental, NVIDIA and TwelveLabs are among the model builders signed on. Cisco and Supermicro will deliver integrated confidential AI systems. BUZZ HPC, Nscale and Sharon AI plan to offer attested sovereign environments, and Fortanix supplies the attestation and key management layer.

Fundamental sells NEXUS, a large tabular model it says runs inside a bank's or health system's own infrastructure.

"Confidential computing turns trust from a promise into a proof, and that's the difference between AI that regulated industries can pilot and AI they can actually put into production," said Jeremy Fraenkel, CEO of Fundamental.

Sovereign clouds gain a second benefit. Hardware enforces the isolation, so a regional provider can host proprietary models and regulated data without dedicating a whole machine to each tenant.

Models as a Managed Resource

DataEnclave also advances a larger VAST position. Models belong under the operating system rather than on top of it, managed as a logical resource alongside data and paired to tasks by purpose, cost and security constraints.

"Models are becoming a resource the operating system has to manage, the same way it manages data," said Renen Hallak, founder and CEO of VAST Data. "That means knowing which model fits which task, what it can see, who can use it and under what rules, and doing all of that inside the same security and operational boundaries an enterprise applies to everything else."

TechArena Take

The most interesting part of this launch sits one level above the enclave. VAST is treating models the way it treats data, as a resource the operating system manages, and DataEnclave turns that idea into an enforceable policy about where a model runs and who holds the keys.  

Operators running more than a handful of models across their environments should look closely at what it means to govern models with the same controls they already apply to data. The detail worth the most attention here is key control over fine-tuned weights. Those weights carry what an organization has learned about its own business, and many teams still treat them as a byproduct of training. Giving enterprises the same protection the frontier labs hold for their base weights sets a precedent we expect the rest of the market to follow.

The partner list tells its own story. Seven model builders agreed to send their models into environments they do not control, which points to where enterprise AI is heading. Models travel to the data. For financial services, healthcare and government, that direction of travel opens use cases that have been waiting on it.

We will be watching two things through 2027: the first deployments in regulated sectors, and the performance envelope operators get with attestation and enclave execution in the path.

‍