Welcome to
the Forum

The boundary organizations once drew around their data now has to extend to the systems that use it. Sovereign AI gives organizations a path to make those systems truly their own.

Hospitals, banks, universities, and national labs have spent decades building systems to protect sensitive data, with patient records, transaction histories, and research files secured in tightly controlled environments.

AI is now putting time-tested safeguards under pressure as organizations seek access to increasingly capable models.

This often requires entrusting proprietary material to third-party infrastructure: every prompt or document retrieved by an LLM can shift confidential information beyond the environment designed to protect it.

Sovereign AI is an approach to AI adoption that gives organizations control over their entire AI stack, including models, compute, software, orchestration, and deployment. This doesn’t merely impact where information resides, but which technologies enterprises can adopt, how they operate, and whether resources remain available as demand grows.

Sovereign AI requires control across the stack

Sovereign AI often starts with a straightforward requirement: sensitive data must remain on premises or within a specific jurisdiction. But keeping data local doesn’t automatically ensure autonomy.

Sovereignty is control across the full AI stack. If you are not owning some piece of it, then part of your intelligence is outsourced, and it is not sovereign to you.

That distinction matters because every external dependency introduces another point where an organization can lose agency. A hospital may rely on a provider’s roadmap to support clinical applications, while financial institutions can face changes in pricing or availability, and research organizations may encounter export restrictions that limit access to critical technology.

These dependencies extend beyond data storage. A company can keep its proprietary datasets on premises while relying on external systems to process them, models to interpret them, or suppliers to provide the underlying compute. Organizations therefore need to consider what happens after they secure the data:

• Who controls the model?
• Can teams inspect and tune it?
• Will the necessary equipment remain available as deployments expand?
• Can software investments survive a change in supplier strategy or export policy?

The same concern applies to future capacity. AI deployments rarely remain fixed, so organizations need confidence that the equipment supporting them will remain available as demand grows.

Otherwise, supply constraints can limit expansion, disrupt budgets, and affect long-term planning. For example, a deployment can satisfy residency requirements at launch and still create future dependencies if additional equipment becomes difficult to procure or critical software relies on a particular roadmap.

Sovereignty ensures control over where information resides and the infrastructure used to process it.

Sovereign AI changes the compute equation

Bringing the complete AI stack inside an organization only works if the infrastructure can support its workloads efficiently. For many organizations, that means reconsidering architectures modeled on hyperscale environments and GPU-dense racks, which were designed for enormous training and inference workloads.

Sovereign AI often looks different, particularly as teams adopt agentic applications that combine AI inference with substantial general-purpose computing.

Agents shuffle through multiple operations as they complete a task. They orchestrate workflows, call tools, enforce security policies, retrieve information, query databases, and generate tokens. CPUs handle many of the control-heavy steps, while AI accelerators execute the matrix operations behind model inference. Those demands can alternate throughout a single workflow as an agent moves between reasoning, retrieval, tool use, and model execution.

For sovereign AI environments, the workload mix directly affects infrastructure decisions. A platform optimized primarily for token generation can leave expensive accelerators waiting while orchestration, database queries, security checks, or other CPU-heavy operations constrain throughput. Adding more GPUs does not necessarily address those bottlenecks, either.

Overprovisioning accelerators also raises the cost of maintaining AI infrastructure. A sovereign deployment needs enough acceleration for model operations alongside the CPU capacity and memory bandwidth required to support the rest of the workflow.

Sizing each resource around actual application requirements can therefore improve utilization while keeping more of the workload within a controlled environment, so organizations can design around applications they intend to run, rather than reproducing a hyperscale architecture inside their own data centers.

Balancing compute around sovereign AI workloads

Sovereign AI infrastructure should enable mixed workloads without overprovisioning one type of compute. Epic Semi’s Contrail AIX, for example, combines general-purpose processing and AI acceleration within one platform.

Teams should always look to support the full agentic workflow within the infrastructure they control. CPUs can handle orchestration, branching, security, database operations, and tool calls while AI cores execute model workloads. A common fabric and memory pool keep both types of compute running in the same environment. For instance, Contrail AIX combines 32 performance cores with 16 AI cores connected by a shared on-chip fabric, allowing general-purpose and model operations to access a unified memory environment.

This balance also provides more flexibility as agentic applications evolve. Enterprises are still learning how production workloads distribute demand across general-purpose processing and AI acceleration. Rather than reproducing a hyperscale configuration by default, teams can size sovereign AI infrastructure around the applications they intend to run. This approach connects control over the AI stack with the practical ability to operate efficiently at scale.

Open architecture reduces long-term dependencies

Maintaining sovereignty requires isolated workflows, enforced policies, securely managed systems, and the ability to expand capacity without unnecessarily introducing new external dependencies, especially as architecture and market conditions change. Revised product roadmaps, export policy, or component availability can undermine existing investments.

Open architectures give enterprises more freedom over the technology behind their AI systems, permitting multiple vendors to develop compatible components around a shared instruction set. Teams subsequently have access to alternatives if one supplier can no longer meet their needs.

One example is RISC-V, an open instruction set architecture (ISA) used to design processors. Or, as Sherwani describes it: “RISC-V is owned by the world. Anybody can adopt it, build on it, and contribute back to it.” RISC-V International governs the specification from Switzerland, creating a shared architectural foundation beyond any single processor supplier or national jurisdiction.

The choice of architecture can also affect the longevity of investments. Organizations may spend years developing applications, optimizing software, and integrating systems, and continued access to the foundation is critical. An open ISA provides more flexibility when there are shifts in supplier strategies, availability, or other external conditions.

Openness, however, still needs to translate into deployable infrastructure, including virtualization, memory expansion, standard interfaces, secure management, and other critical capabilities for AI systems. In practice, that means pairing architectural openness with critical production server capabilities, such as full hardware virtualization, IOMMU support, and standard interfaces.

Enterprises also need a realistic path to procuring equipment, especially in regulated and geographically distributed markets. These considerations determine whether architectural openness actually supports meaningful autonomy.

Proven infrastructure supports long-term sovereignty

Sovereign AI simultaneously supports and depends on an organization’s ability to maintain control as deployment expands. That makes infrastructure maturity and long-term availability important considerations: a proven platform can reduce risk, simplify integration, and give enterprises a clearer path to additional capacity without introducing new dependencies as requirements evolve.

Production AI, however, relies on more than silicon. Operating systems, storage, networking, memory, management, and cooling all have to work together to support reliable deployment, operation, and expansion at scale. For enterprises, the broader ecosystem can shorten the path from evaluating a processor to running AI workloads on infrastructure they control.

Supply continuity provides another benefit: sovereign AI deployments can expand significantly over time, making future access to equipment part of the initial architecture decision. With greater operational control, organizations can integrate the technology into existing environments, scale capacity alongside workloads, and preserve their infrastructure investments over time.

AI sovereignty enables controlled scalability

The AI stack is already absorbing unprecedented institutional knowledge, decision-making and computing activity. As AI becomes more deeply integrated into institutional operations and agents handle increasingly complex workflows, controlling the infrastructure beneath it will become critical.

Sovereignty gives organizations a way to retain that authority: where workloads run, which models they use, how systems evolve, and how much capacity they add. This dynamic fundamentally changes the role of AI infrastructure. Decisions about compute, software, models, and supply directly affect how much control organizations retain over their AI operations.

An architecture designed around internal requirements can give teams greater freedom to adopt new capabilities, adjust resources as workloads develop, and make technology choices according to their priorities. That flexibility becomes more valuable as AI moves from discrete applications into workflows that organizations rely on for essential operational functions.

The benefits compound over time, enabling enterprises to advance without forfeiting the control they worked to establish in the first place.

Contrail AIX translates these principles into infrastructure enterprises can deploy. Specifically built for AI environments, the platform combines compute, networking, storage, and the supporting software stack into an integrated and scalable system, enabling organizations to own their intelligence.

‍

VEIR closed an oversubscribed $110 million Series C round to move its superconducting power delivery systems for AI data centers from technology validation to commercial deployment and scaled manufacturing.

The Woburn, Mass., company will use the capital to expand manufacturing capacity, advance its product portfolio and build the technical and commercial capabilities needed to support adoption at scale. It expects its first commercial deployments in 2027.

Matter Venture Partners and Tyche Partners co-led the round, joined by LG Technology Ventures, Gates Frontier, Sabanci Climate Ventures and Hui Capital. Existing investors including Engine Ventures and Congruent Ventures also participated. VEIR has now raised $225 million in total.

VEIR’s systems pair superconducting cables with cryogenic cooling, controls, termination units and monitoring. These systems move significantly more power through a smaller physical footprint and reduce labor costs, material requirements and installation complexity. VEIR has demonstrated a 3-megawatt system and is working with customers and industry partners on AI-factory reference designs.

“This financing gives us the resources to move from demonstrating the potential of superconducting power delivery to manufacturing and deploying systems at commercial scale,” said Tim Heidel, CEO of VEIR, in a press release.

On Sept. 28, I judged the Energy category of Yotta 2026’s Innovate Arena in Las Vegas, where Tim Carr, VEIR’s senior director of business development, pitched a cable that carries the same power as a copper cable tray in a conductor one-tenth the size.

I was delighted to see VEIR on the Innovate Arena stage at Yotta and to see this round announced so soon after.

TechArena Take

Yotta 2026 put power generation at the center of the AI buildout conversation. VEIR is working on delivery, where copper requires more labor, material and space as campuses grow. This round gives VEIR the capital to prove its systems in commercial deployments. We’ll be watching for its first installations.

‍

AI infrastructure contracts with hyperscalers can come with terms that are hard for smaller AI startups to navigate. On a recent podcast, I sat down with Ilkhom Sidikov, who works on the neocloud io.net’s IO Intelligence platform, to talk about how decentralized GPU networks are competing with hyperscalers by offering flexible, affordable options to AI innovators.

From Web3 Roots to an OpenRouter Listing

Ilkhom joined io.net in early 2024, not long after the company began turning a decentralized GPU network into something usable for AI startups and developers, an effort Ilkhom said got its start “in the age of Web3 and AI.”

He first worked on network infrastructure, and since 2025, his main project has been IO Intelligence, the vertical where io.net builds applications for AI startups, including model training, model inference, and video and image generation.

One recent milestone was being listed on OpenRouter, the LLM gateway many developers know. Io.net’s GPU network now hosts open-source models handling 2 billion to 5 billion tokens a day for OpenRouter.

Where Neoclouds Offer More Flexibility

Ilkhom said hyperscalers typically lock customers into one- to three-year contracts calibrated to protect their own margins, agreements that can be “really hard” on engineers and small AI startups. Under such contracts, a customer can end up paying for GPUs, like older 4090s, that fall behind newer architectures before the term ends. Even the AI grants that hyperscalers sometimes offer, he added, come with a catch. They typically require running on that provider’s own infrastructure.

Neoclouds, Ilkhom said, offer more room to negotiate, and io.net itself supports pay-as-you-go pricing for both GPU rental and LLM inference. Io.net’s specific advantage is how it negotiates use periods. Customers can rent a cluster for as little as 10 or 12 hours, and once they’re done, io.net shifts those same GPUs into its LLM inference infrastructure rather than leaving them idle.

“We never lose money on the idle GPUs,” Ilkhom said, adding that an idle GPU costs the same to run as one fully in use. That constant reallocation is what lets io.net offer clusters at a fraction of hyperscaler pricing.

Earning Enterprise Trust

Enterprise teams bring healthy skepticism to decentralized compute, particularly around reliability and security. Ilkhom said io.net encountered fraudsters early on who would alter a GPU’s metadata to make a cheaper card, like a 4090, pass as a far pricier H100.

Io.net’s fix was building software that profiles every hosted GPU on an hourly basis, checking whether each card meets performance benchmarks for what it claims to be.

On the network side, io.net connects nodes through virtual private networks (VPNs) to secure data in transit. It recently added a confidential inference feature that lets enterprise clients verify that every token generated, from the GPU level through the gateway to the back-end application, happens inside a secured environment.

Engineering for Scale and Profit

Running inference profitably at scale comes down to a handful of techniques, Ilkhom said: quantizing models down to formats like FP8 or INT4, using key-value (KV) caching to speed up throughput, and batching inference requests. Skip those optimizations, he said, and providers selling API tokens can end up burning cash, since deploying a raw model with default settings can make it hard to even break even.

Agents Raise New Demands

The rise of AI agents presents new demands on compute and data. Context windows have grown enormous, Ilkhom said, citing Anthropic’s own claim that nothing stops the company from building a model with a million-token context size. Combined with the development of MCP servers and skills that let agents pull real-time data instead of relying on static databases or vector search, agents now behave less like single inference calls and more like systems that maintain memory and reach out for information as they need it.

The TechArena Take

My conversation with Ilkhom conversation showed how decentralized providers like io.net are winning through an intensive focus on optimizing their resources. Io.net does this by keeping GPUs constantly busy rather than allowing them to sit idle between contracts.

Every technique he described, from shifting idle clusters into LLM inference to quantizing models, squeezes more usable compute out of the same hardware hyperscalers rely on too. Hourly hardware benchmarking helps build the trust enterprise customers need before relying on that hardware at all.

Ilkhom’s 5-year outlook does not predict that decentralized compute will displace the hyperscalers. After all, regulated and well-capitalized buyers have reasons beyond price to stay where they are. What his outlook does suggest is a market potentially splitting by workload, with contract-scale commitments on one side and short-duration, price-driven capacity on the other. For technology leaders, this opens up new possibilities to start matching each workload to the economics that fit it.

To learn more, visit io.net.

‍

Niklas Panten and his team at etalytics keep track of how much energy their software saves per customer and site across global deployments. Watching the numbers climb unifies the team around the company’s purpose, Panten said.

Founded in 2020 as a spinoff of Niklas’ PhD research at the Technical University of Darmstadt, etalytics builds AI software that orchestrates the cooling, heating and power systems in data centers and factories so they use less energy.

The ETA Factory

Panten, co-founder and CEO of etalytics, built his PhD on a mix of mechanical engineering, computer science and energy science. He helped develop the ETA Factory, a full-scale research factory TU Darmstadt opened in 2016 to test energy efficiency on working production lines.

His research group spent more than 10 years analyzing and optimizing energy systems in manufacturing plants. Its partners included Bosch, Volkswagen and Mercedes, along with chemical and pharmaceutical companies. The researchers used AI to run cooling, heating, ventilation, on-site generation and batteries more efficiently, work Panten said began before the AI boom.

“In our research, we found almost 50% in savings in those systems,” he said.

Two research projects at the factory, PHI-Factory and SynErgie, ran from 2014 to 2018 and produced the first ideas and prototypes of etaONE, the platform etalytics sells today, according to the company. Thomas Weber, who co-founded etalytics with Panten and is now its chief strategy officer, worked on his PhD in the same lab. The founding team, which includes CTO Björn König, came together in 2019.

“(This work) came from personal passion; I was a controls engineer and I started on power converters,” he said. “I found out that we can… make (power converters) much more efficient. But even if you have the most efficient power converter, if you have the component that has a power converter in there, and it’s not operating efficiently, you’re still losing energy.”

The same problem repeated one level up. “And then looking at the component, you can optimize the component, but if it’s not optimized or orchestrated in the system, then you’re still losing energy,” he said.

That finding is the basis of etalytics: to run at its most efficient point, a whole system has to be orchestrated. In a data center, Panten applies it to cooling “from the rooftop to the server room.”

From Factories to Data Centers

In 2020, Equinix asked the team to take its approach to data centers.

“They basically asked us, ‘Hey guys, we’ve seen your potential here, can you transfer this to data centers?’” Panten said.

At Equinix’s FR6 data center in Frankfurt, etaONE cut the energy used for cooling by 900 MWh a year and avoided 240 tonnes of CO₂ emissions a year, with a payback of less than 12 months, according to etalytics.

The company kept its industrial work. At Merck’s headquarters in Darmstadt, etaONE cut the electricity used for cooling by an average of 21%, in a project coordinated by TU Darmstadt’s ETA research group. At the Stellantis paintshop in Rüsselsheim, Germany, it cut ventilation energy use by more than 60%, in a project with the university.

“Cooling is a cross-sectional technology,” Panten said. “So, the same cooling equipment that we deploy for data centers, you will find in automotive, chemical, pharma, logistics, especially cold logistics, food processing.”

Proof at Telehouse

The company’s etaONE platform sits on top of a data center’s building management system (BMS). It streams the cooling plant’s telemetry into a physics-based digital twin of the equipment, calculates new set points every two to three minutes and writes them back to the BMS.

“We don’t rip and replace anything, we just add on top,” Panten said.

At Telehouse’s Frankfurt campus, digital twins of the cooling towers showed their performance declining. etalytics traced the decline to fouling, a buildup of deposits on the towers’ heat exchangers that reduces how well they shed heat. A heat wave was approaching, and Telehouse moved its maintenance up. Panten said the predictive maintenance product grew out of checking etalytics’ own models against the plant.

“As soon as we saw deviation, we knew it’s not our models that deviate, it’s the physics that change,” he said.

Over an assessment from April 14 to Aug. 5, 2026, etaONE adjusted the cooling tower outlet temperature and the recooling pump flow. One unit used 10.4% less electricity, and etalytics projects 497 MWh in annual savings across five units. The project won the Infrastructure Sustainability Award at the Platform Global Awards 2026.

Panten said etalytics chose physics-based models over reinforcement learning so operators know why the system makes each change.

Megawatts for Compute

Panten said energy efficiency is crucial in an industry constrained by power.

“Everyone is looking for more megawatts, so when you make megawatts available from cooling, you can put them in compute,” he said.

He pointed to research from the International Energy Agency, which projects that well-documented AI use cases could save more than 13 exajoules of energy by 2035, equivalent to 3% of global final energy consumption, if barriers to wider adoption are overcome.

Looking Ahead

etalytics opened an office in the San Francisco Bay Area in May 2026, and Panten is now based in Menlo Park, California.

Microsoft’s venture fund, M12, led an investment in etalytics in 2025. This year was the company’s first time attending Yotta, and Panten said it will have a booth next year. In October, the company plans to launch etaONE IQ, an assistant for operators’ daily work, at an event with Microsoft and NVIDIA during San Francisco Tech Week. Panten said etalytics’ AI will also move onto the coolant distribution units that serve AI factories.

TechArena Take

We see cooling orchestration as one of the fastest ways for operators to free megawatts for compute without waiting on the grid. The physics-based approach matters as much as the savings, because operators of mission-critical sites need to understand every change an AI makes before they trust it.

Customers are signing five-year contracts for AI silicon that has barely started shipping, while the models running on that hardware turn over in weeks, leaders from Crusoe, Microsoft and VAST Data said during a panel at AI Infra Summit 2026.

I had the delightful opportunity to moderate “Under the Hood of AI Infrastructure: What Top Leaders Do Differently,” with Erwan Menard, Crusoe’s senior vice president of product management; Donald Thompson, a distinguished engineer at Microsoft; and Alon Horev, co-founder and chief technology officer at VAST Data.

Across the hour, the three described an industry committing capital years ahead of knowing what it will run. Hardware gets committed in multiyear blocks and new power takes years to bring online, while the models and workloads running on them can change in weeks.

Erwan described clocks that do not agree: “I think in days. The data center guys used to think in years. Now, at places like Crusoe, they think in quarters. The energy world thinks in multiple years.”

One of Alon’s customers moves faster still, retiring the model its users are talking to every 30 minutes and publishing a newer one so that results stay fresh.

Alon called AI the “reset event of infrastructure globally,” and the technology hasn’t stopped resetting. Four themes from our discussion show where: Training and inference have folded into one continuous loop, the pace of innovation has turned into a design constraint, the megawatt has become the unit to plan around, and the data center is becoming a single computer spread across a building.

The Model That Changes Every 30 Minutes

Inference is probably bigger than training across the worldwide GPU fleet, Erwan said, and at the AI-native companies Crusoe works with, it “becomes the lion’s share” once a product reaches volume.

Open models drive a lot of inference traffic too, he said. Crusoe helps customers modify a model with their own data, put it into production and retire it on their own schedule.

“The big motivation we hear from customers is control,” he said.

Donald carried the idea further. Digital workers are trained by people inside a specific work context, and their trajectories have to improve over time, which rules out pure training and pure inference.

“You have to be in a constant learning mode,” he said.

He described sleep-time compute, fine-tuning during quiet hours, and reinforcement learning in eval environments built to replay what an agent has just gone through.

Alon’s 30-minute customer is running that pattern in production. They kept pressing his team to speed up model loading, and he asked why they needed it hundreds or even thousands of times a day.

“Because we train nonstop,” they said.

Innovation Outruns the Build Cycle

Erwan told a story about an inference engine. Someone at the show that morning claimed bragging rights for building the first one three years ago, and Erwan doubted it was still in production.

“The pace at which these things are evolving is very annoying,” he said.

Crusoe shipped its first inference service a year ago and still tunes it weekly against fresh open source, new papers and new practices. Managed Kubernetes taught him the same lesson. A general-purpose version takes thousands of engineers and years; an AI-specific one has to be excellent at a short list of things, and Crusoe’s now powers 70% of its compute fleet.

Erwan described the new pace: Research reaches production in weeks, instead of two years, and his researchers talk directly to customers, who have seen Jensen Huang’s Pareto curve and are weighing throughput against latency.

“You need to walk with them and innovate with them on the fly,” he said.

Alon agreed that standards will likely hold you back at this pace, though he sees new ones forming organically, with SNIA and various companies working together. What changed, he said, is that buyers are less afraid of proprietary technology. Before AI, buyers wanted every layer of the stack to be generic, which made it hard to innovate or change big parts of it. “And AI changed that mindset at least for now.”

Getting More From Each Megawatt

Most AI infrastructure gets bought in multiyear reservations now, Erwan said. He described five-year commitments for Vera Rubin, silicon that is only now starting to arrive. Buying that far ahead means committing to power before anyone knows what the workload will draw, and in reserved clusters already running, consumption sometimes lands well below what’s available. That gap opens room for a provider who understands both the machine’s power needs and the workload.

Closing the gap eventually means packing workloads onto shared hardware, in multitenant environments that absorb the peaks and valleys of inference traffic without stranding costly GPUs, he said. He predicted that by 2028, everyone will be optimizing for availability of megawatts.

“The energy wall is coming, and it’s going to take a lot of time to remove it,” he said.

Donald said Microsoft is working at the application and platform level to drive 24/7 inference, which is what pushes demand for megawatts.

Alon came at the same problem from the data side. He described the job of infrastructure builders as making “every megawatt, every terabyte on SSD and every CPU” serve training, fine-tuning and inference alike. He later asked whether reducing the doubled hardware some organizations carry on every stored byte could cut power along with it.

Packing workloads better only goes so far. Erwan expects power to stay short, since adjusting energy supply to demand takes years. Crusoe’s other answer runs through geography: hydro in Norway, geothermal in Iceland, and a small modular reactor paired with a modular data center that he hopes to put in service by mid-2027.

The Data Center as a Single Computer

Alon called inference “a clustered problem,” with prefill and decode disaggregated so that “some GPUs do this; other GPUs do that.” Every open session ties up a block of GPU memory for as long as it lasts. That block is the KV cache. In some models, contexts of a million tokens push it to “tens or even hundreds of gigabytes,” crowding out other users, so it gets moved to a storage system or an SSD on another host. Routing then has to send that user back to whichever machine is holding their cache, so no single GPU completes the job alone.

“Everything is integrated into one logical computer that’s doing inference, … and KV cache is becoming a core component of it,” Alon said.

Eighteen to 24 months ago, Crusoe sized its data centers at four to five terabytes of storage per GPU, Erwan said. Today it is north of 10, and a given inference proof of concept takes roughly 20% fewer GPUs than a year ago.

He also wants the agent’s memory and context treated as its own layer of the stack. There is too much of it to hold in expensive memory, so some has to move to cheaper storage, and he sees “true infrastructural challenges in delivering that context.”

We should look at a data center now, Alon said at the end of the hour, as “a single computer just distributed across so many different machines and components.”

The TechArena Take

Hardware and power get committed for years. The models running on them change in weeks. That is the mismatch, and the panel treated it as a purchasing problem. Most of their answers were things to buy: longer silicon reservations, sites in Norway and Iceland, a reactor, more megawatts by 2028. One answer was operational, packing more work onto hardware already installed and already paid for, and it got less time.

That ordering is backward for anyone who has already signed. A five-year contract cannot be renegotiated into a different clock. What can still change is how the building runs: what gets packed where, what waits for a cheaper hour, what the machines actually draw against what the spec promised.

Donald raised this on stage, asking how intelligence gets applied to running the data center itself instead of leaving a person at a dashboard. Nobody answered. That is the question TechArena would put to vendors this year: What does the cluster do on its own when the model changes in six weeks? The answer may decide who is still competitive in 2028, when Erwan expects megawatt availability to be what everyone optimizes.

For decades, the industry has named walls: the memory wall, the compute wall, the network wall, the cooling wall, the data wall. The list keeps growing. This week, more than 6,000 people converged on Las Vegas for Yotta 2026 and unpacked the wall beneath all of them: power. Without power, none of it functions.

Across three days, the builders, suppliers and founders on Yotta’s stages and on the expo hall floor made the case: The industry can deploy only as much AI as it can power, so every gain in compute, cooling, storage, networking and I/O now counts in tokens per second per watt.

The Power Gap

Varun Sivaram, CEO and founder of Emerald AI, told the Crusoe Main Stage audience on Tuesday morning how much of the AI factory capacity planned in the United States will lack grid power to run it.

“If I put some numbers to it, between now and 2028, the United States seeks to connect something like 70 gigawatts of AI factories,” he said. Less than half of that will successfully plug into the grid, he added. “There's a massive supply-demand gap just on the power side.”  

Sarfraz Taj, vice president of business development at Terrestrial Energy, opened his Innovate Arena pitch on Monday with the total capacity of data center projects now planned in the United States.

“So, Vegas is all about numbers. Let me start with some numbers: 300 gigawatts. That’s the capacity of data center projects currently planned in the U.S.,” Taj said.

The first response to the gap is already used up, he explained. Some hyperscalers and developers went directly to power producers, restarted shuttered nuclear plants and pushed existing plants to higher output.

“Against a 300 gigawatt plant capacity for data centers, that low-hanging fruit is now all but virtually gone, so not a viable way to get additional electricity,” he said.

Chris Lander, vice president of XFRA at Span.io, described how long the shortage of grid connections now adds to a data center build. He listed interconnect queue delays, transmission line build-outs, lead times for substations and transformers, and public opposition in local communities.

“A year ago, it would take around three years to build a data center for AI,” he said. “We’re seeing that three years already extend to five years.”

Solar Power at Night, From Orbit

The farthest source of new power pitched at Yotta sits in orbit. Abdullah Al-Shakarchi, co-CEO of Overview Energy, presented it in the Innovate Arena on Monday.

“We are in the business of selling invisible light,” Al-Shakarchi said.

Overview Energy places satellites in geosynchronous orbit, where they receive sunlight around the clock. The satellites convert sunlight to electricity and use it to power near-infrared lasers. The lasers send an invisible beam to solar farms on the ground, so those farms produce power at night.

Al-Shakarchi said a few hundred satellites could power a single 300- to 400-megawatt training data center. The satellites can also redirect their beams. A satellite serving Las Vegas could send its energy to Morocco within minutes, he said.

“This is the world’s only movable power plant,” he added.

His company’s approach turns solar from a source that produces power 35% of the time into one that produces 70% to 90% of the time, using solar farms that already exist. Meta, one of the company’s first customers, announced a gigawatt agreement with Overview Energy in April to power its data centers from space. Al-Shakarchi said the company plans to reach orbit next year, power a U.S. utility-scale solar project in 2028 and serve its first customers in 2029.

Trevor Smith, CEO of Atomic-6, pitched a different use of orbit: data centers that operate in space. Atomic-6 builds solar arrays, radiators and debris shielding for those satellites. Smith said orbital data centers will stay small, in nodes of 100 to 200 kilowatts that connect to one another.

“You’re not going to see Costco-sized data centers flying around in space,” he said. “It just doesn’t make sense.”  

The Grid Already Built

Vladimir Troy, vice president of AI infrastructure at NVIDIA, said the nearest source of new power is grid capacity the country has already built. Emerald AI’s Sivaram interviewed Troy on Tuesday’s main stage. Troy said utilities build the grid to meet peak demand, and demand stays below that peak most of the time.

“Like 99% of the time, (it’s) just sitting there and being underutilized,” Troy said.

An AI factory that reduces its power draw during peak hours can use that capacity, he added. Sivaram said the U.S. grid has 100 gigawatts of available capacity today for AI factories that can operate that way. Emerald AI builds the software that lets AI factories adjust their power use. NVIDIA, Google and Emerald AI launched the AI Energy Management Alliance on Sept. 17 to advance that approach.

New Power on Site

Sites that cannot wait for a grid connection need power generated and delivered on the campus itself. Founders and suppliers at Yotta pitched both.

Emerson Reiter, chief commercial officer and co-founder of Teragen Energy, pitched fuel cells. Teragen builds one-megawatt units that fit in a 20-foot shipping container.

“Primarily, fuel cells turn molecules to megawatts without combustion,” Reiter said.

Teragen’s cells can ramp from zero to full power in minutes rather than hours, he added. The company plans commercial deployments in 2028.

Vince Canino, president and CEO of Capstone Energy+, told TechArena that the company's gas turbines can serve a data center from groundbreaking to full operation. Each turbine has a single moving part that rides on a cushion of air, so it runs without oil, coolants or water, he said.

"Because we can move our equipment around very easily, we can start out as construction power," Canino said. "As construction is done, we can move the equipment to bridge power, and then we can put it in its final permanent position as primary power to the data center."

Constantin Eis, CEO of CMBlu Energy, pitched a water-based organic flow battery that cannot catch fire. He compared the U.S. market to Germany, his home market, which is closing power plants.

“Actually, I think the capital is there, the land is there, customers are there. The only problem which is missing is the energy,” he said.

Power generated on site still has to reach the racks. Tim Carr, senior director of business development at Veir, pitched a superconducting cable that carries the same power as a copper cable tray in a conductor one-tenth the size.

“We believe that power delivery, not just power generation, is the next necessary evolution,” he said.

Melton Chang, executive vice president of power systems at Schneider Electric, told TechArena that the company is replacing the wiring and custom hardware in medium-voltage switchgear with software. He said the change shortens time to power and moves most commissioning from the job site to the factory.

“Software-defined is not just about one product,” he said. “Software-defined is really about the system.”

More Tokens From Each Watt

Once power reaches a site, the next question is how much of it becomes tokens. Peter Panfil, vice president of power architecture and application at Vertiv, said a 100-megawatt site running at industry-norm efficiency puts about 60 megawatts to productive work at the GPU. He also cited a GPU utilization rate of about 70% that he heard three times at the conference.

“That’s a pretty damn expensive sports car to only be able to run at 70% of its capacity,” Panfil said during the Tokens Per Watt panel on Wednesday, referring to GPU capacity left unused.

Sean Corcoran, vice president of sales for PowerOne at Airsys, discussed a metric called PCE: the IT load a site provisions divided by the building load it provisions.

“It’s all about finding that waste of power and efficiently using it to compute more and more,” he said.

Power spent on cooling produces no tokens. Matthew Orcutt, vice president of strategy and systems product management at Trane Technologies, told TechArena that NVIDIA’s support for 45°C facility water lets a data center cool its GPUs with dry coolers, without chillers or compressors, for most hours of the year. LiquidStack, a Trane Technologies company, launched a coolant distribution unit at Yotta that supports that temperature. Orcutt said the amount of water a data center uses is a separate design decision.

Storage Joins the Math

Storage design also affects how much work each watt produces. On TechArena’s Tuesday panel, Angel Camacho, director of product marketing at Western Digital, said the industry first answered shortages by buying more storage and more power.

“Until suddenly, we hit a wall. It becomes too expensive,” he said.

Marc Austin, CEO and co-founder of Hedgehog, described a customer demonstration at Computex. One GPU ran an inference workload in 90 seconds. The same GPU, paired with a second accelerator, ran it in 30 seconds.

"Do the kind of simple math on that and it’s six times the tokens per watt, which is huge,” he said.

Betsy Chernoff, principal AI product marketing manager at WEKA, said storage has joined that calculation only recently.

“When you get over to the inference side of the house, you can have a fully utilized GPU, and still have it incredibly inefficient,” she said.

The Side Yard

Some companies took the opposite approach and moved the compute to places where power already exists. Span, the smart electrical panel company Lander represented, places liquid-cooled inference nodes in homeowners’ side yards, next to the electrical panel. The nodes run on capacity the home does not use, and the homeowner receives subsidized energy and internet in return.

“Instead of waiting for power and traditional data center infrastructure, organizations are increasingly looking at how they can take the compute to the power,” said Angie Keeler, co-founder and CEO of Zella DC.

Submer Group launched Corenix, which builds and tests complete data center modules in a factory before shipping them to a site.

The Neighbors

Plans to build on the ground require approval from the community around the site. Niklas Panten, co-founder and CEO of etalytics, whose software reduces the power that data center cooling plants use, said he understands the objection.

“To be honest, I don’t want (a data center) in my backyard either," he said.

Allyson Klein, founder and CEO of TechArena, moderated Wednesday’s Tokens per Watt panel and gave its final thought.

“I think that we’re going to see the focus on manageability, and how we create a control plane for operators that not just takes into account the compute infrastructure, but also storage and networking, as well as potential discussions about control of your power and cooling equipment from an integrated pane of glass,” she said. “I think that those things can provide us an ability to drive up every metric that was talked about on stage today and will bring this industry closer together. What I’m excited about is actually being here to see it all happen… This industry is incredible at innovating itself out of a problem.”

TechArena Take

We came away from Yotta 2026 seeing the power wall as a design problem as much as a supply problem. The speakers with the most concrete answers treated power, cooling, compute, storage and networking as one system, and they measured that system by the tokens it delivers per watt. Overview Energy took the other approach: it adds new hours of output to solar farms that already exist. We expect both approaches to grow.

We see tokens per second per watt as the measure to watch. Operators who design their sites around it will add AI capacity in the next year without waiting for a place in the interconnect queue. The operators who win community approval will be the ones who can show their neighbors a stronger grid and lower rates.

‍

Why enterprise infrastructure must optimize the entire workflow, not only model inference

A recent study of five representative agentic AI workloads found that the dominant bottleneck shifted from retrieval to code execution, scientific tools, summarization, and model inference. In several tests, work outside the model consumed more than 80% of end-to-end latency.

That matters because an AI agent does much more than running a model. It plans, retrieves data, calls tools, updates its state and repeats the process until the task is complete. Each stage stresses a different part of the system.

For banks, hospitals, universities and national labs that buy and run their own infrastructure to keep data on premises, optimizing only the accelerator may only yield very limited impact on the entire workflow, or even worse, leave your most expensive and power-hungry accelerator waiting.

The bottleneck moves

The study, “Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective,” published in November 2025, instrumented the five workloads end-to-end across two systemsi. The results do not converge. In each workload, a different stage consumed the largest share of end-to-end latency:

Selected results from two systems and multiple benchmarks. The dominant stage changes with workload inputs and hardware configuration.

• Only in the last case is the bottleneck the matrix math the industry has been building for.

For enterprise infrastructure teams, that ratio changes the buying decision. The bottleneck moves with the work, and a production deployment runs several kinds of work at once. No single resource can be scaled up to fit the whole fleet. Take the retrieval-augmented pipeline from the study, where nearest-neighbor search accounted for more than 80% of the time and runs on CPU cores. Buying more accelerators speeds up the model inference in the remaining share. The search step, which decides how long the whole job takes, does not get faster at all.

One agent needs several kinds of compute

Look one layer down and the same pattern holds. At the model level, prompt prefill benefits from high parallel compute throughput, while token-by-token decode depends more heavily on memory bandwidth and capacity. Outside the model, data processing, tool calls, orchestration and security run efficiently on general-purpose cores. An agent moves between these phases continuously.

Balanced systems are becoming a mainstream design priority. Intel CEO Lip-Bu Tan told the Computex 2026 audience that agentic AI is “returning the CPU to a position of prominence,”ii and AMD made the same case for CPUs in the AI data center earlier in the year.iii The single-silicon era of AI infrastructure is closing.

Why we built Contrail AIX

Organizations and nations that desire to gain greater control over how AI is deployed, customized and governed keep their data on premises, deploy open-weight models and build infrastructure for the complete agent workflow deployable inside their own walls. That is what Epic Semi designed Contrail AIX to be.

What we built to run AI agents

Contrail Compute, the RISC-V CPU and AI superchip inside our Contrail AIX platform, puts 32 high-performance RISC-V cores and 16 RISC-V AI acceleration cores on one die, sharing one memory and storage space across a common on-chip fabric.

This architecture keeps retrieval, orchestration, model execution, and tool calls close to the same agent state, reducing unnecessary data movement between separate devices. Each Contrail Compute chip can serve as a native agentic AI execution unit, and these units are small, complete, and fungible. Small means granular enough to scale independently. Complete means useful work can run locally. Fungible means any unit can be scheduled, replicated, drained, or replaced through a common operating model.

Agent-aware orchestration compiles the workflow into a physical plan, places each fragment on the right resources, preserves data locality, and continuously resizes the fleet. Units pool when demand rises, fail over automatically, and disappear when no longer needed.

We built it on RISC-V so a customer’s software investment outlives any single vendor’s roadmap and any single government’s export policy. Contrail AIX carries a 5A992C export classification, which reaches substantially all markets. It supports the RISC-V hypervisor extension with full hardware virtualization and IOMMU, RVA23-compatible architecture, trusted secure boot and full-chip security management. Ubuntu came up shortly after first silicon, and llama.cpp and open-weight models are running today. Compute is the first product on a multi-generation roadmap, from a team that has ramped millions of microprocessors.

Agentic benchmarking is underway. We will publish performance results after optimization and validation so customers can evaluate defensible end-to-end measurements rather than synthetic claims.

If you operate AI on premises and want control of your data, bring us your workflow bottlenecks. We’ll help you measure them end-to-end on Contrail AIX.
‍

[1] Ritik Raj, Souvik Kundu, Ishita Vohra, Hong Wang and Tushar Krishna, "Towards Understanding, Analyzing, and Optimizing Agentic AI Execution: A CPU-Centric Perspective," arXiv:2511.00739, submitted Nov. 1, 2025, revised April 16, 2026, https://arxiv.org/abs/2511.00739.

[1] Intel Newsroom, "Computex 2026: An Intelligent World Built on Silicon," June 2026, https://newsroom.intel.com/artificial-intelligence/computex-2026-an-intelligent-world-built-on-silicon. Quote as reported in Divyanshi Sharma, "Computex 2026: Intel announces Xeon 6+ processors, says AI will make CPUs important again," Digit, June 2, 2026,

[1] AMD, "Agentic AI Brings New Attention to CPUs in the AI Data Center," AMD corporate blog, March 13, 2026, https://amd.com/en/blogs/2026/agentic-ai-brings-new-attention-to-cpus-in-the-ai-data.html.

‍

VAST Data today introduced VAST DataEnclave, a confidential AI runtime that lets model builders place proprietary models inside a customer's own data center and run them against regulated data. The capability, built on NVIDIA Confidential Computing, previews today and ships in Q1 2027 through VAST and OEM partners including Cisco and Supermicro.

DataEnclave keeps data and model weights encrypted while they are in use, and releases decryption keys only after it verifies the environment those assets are about to enter. That gives an enterprise a way to apply a frontier model to data it cannot move, and gives a model builder a way to deploy weights into infrastructure it does not own.

The Gap at Execution

Encryption already covers model weights on storage and across the network. The gap opens at execution. To serve a prompt, weights decrypt into GPU memory, where the infrastructure's administrators, a compromised hypervisor, other tenants sharing that memory and firmware exploits can all reach them.

Enterprises and model builders each give something up to that exposure. Banks, hospitals and government agencies hold data they cannot ship to a hosted AI service, so the best models never see it. Model builders treat base weights as their most valuable asset and keep them on a short list of trusted platforms, which puts air-gapped and sovereign facilities out of reach. VAST, citing Synergy Research, puts about 50% of the world's data somewhere frontier models cannot go today.

Verify First, Decrypt Second

DataEnclave adds a secure container runtime and an attestation service inside the VAST DataEngine.

Hardware-isolated execution runs workloads in confidential virtual machines and containers, using trusted execution environments from Intel, AMD and NVIDIA Vera to encrypt guest memory, GPU memory and NVLink traffic. Active data and models stay isolated from infrastructure administrators and from other tenants on the same hardware.

Verify-before-decrypt attestation checks the trusted execution environment, including NVIDIA GPU attestation, before any decryption key is released. VAST states the rule plainly in its briefing: "No attestation. No access."

Independent key control lets each party hold its own keys in its own trust domain through bring-your-own KMS integrations. The model builder protects base weights. The enterprise protects its data and the fine-tuned weights that increasingly carry its own IP.

That key control is what VAST says separates DataEnclave from the confidential VMs cloud providers already sell. The weight owner verifies the platform itself, keeps its own keys and gets the same arrangement whether the deployment sits on-premises, in a cloud or behind an air gap.

VAST logs attestation events, key releases and enclave lifecycle actions to a tamper-proof audit trail in the VAST DataBase. Deployments run connected or fully air-gapped, on the open CNCF Trustee stack or on Fortanix confidential AI infrastructure. The same runtime gives agents isolated sandboxes through VAST AgentEngine.

More Than 20 Partners at Launch

Cohere, CrowdStrike, Deepgram, Factory, Fundamental, NVIDIA and TwelveLabs are among the model builders signed on. Cisco and Supermicro will deliver integrated confidential AI systems. BUZZ HPC, Nscale and Sharon AI plan to offer attested sovereign environments, and Fortanix supplies the attestation and key management layer.

Fundamental sells NEXUS, a large tabular model it says runs inside a bank's or health system's own infrastructure.

"Confidential computing turns trust from a promise into a proof, and that's the difference between AI that regulated industries can pilot and AI they can actually put into production," said Jeremy Fraenkel, CEO of Fundamental.

Sovereign clouds gain a second benefit. Hardware enforces the isolation, so a regional provider can host proprietary models and regulated data without dedicating a whole machine to each tenant.

Models as a Managed Resource

DataEnclave also advances a larger VAST position. Models belong under the operating system rather than on top of it, managed as a logical resource alongside data and paired to tasks by purpose, cost and security constraints.

"Models are becoming a resource the operating system has to manage, the same way it manages data," said Renen Hallak, founder and CEO of VAST Data. "That means knowing which model fits which task, what it can see, who can use it and under what rules, and doing all of that inside the same security and operational boundaries an enterprise applies to everything else."

TechArena Take

The most interesting part of this launch sits one level above the enclave. VAST is treating models the way it treats data, as a resource the operating system manages, and DataEnclave turns that idea into an enforceable policy about where a model runs and who holds the keys.  

Operators running more than a handful of models across their environments should look closely at what it means to govern models with the same controls they already apply to data. The detail worth the most attention here is key control over fine-tuned weights. Those weights carry what an organization has learned about its own business, and many teams still treat them as a byproduct of training. Giving enterprises the same protection the frontier labs hold for their base weights sets a precedent we expect the rest of the market to follow.

The partner list tells its own story. Seven model builders agreed to send their models into environments they do not control, which points to where enterprise AI is heading. Models travel to the data. For financial services, healthcare and government, that direction of travel opens use cases that have been waiting on it.

We will be watching two things through 2027: the first deployments in regulated sectors, and the performance envelope operators get with attestation and enclave execution in the path.

‍

AI infrastructure leaders from around the globe gathered this week for the 2026 AI Infra Summit in Santa Clara against a backdrop of AI model slowdown discussions. Speakers from Crusoe to OpenAI separated the two issues: the slowdown call concerns frontier model training, and the infrastructure they are building runs inference.

Across three days at the summit, operators and suppliers pointed to the continued growth of inference demand, stating that they can build only as fast as they can secure power. They laid out how to get more tokens from each megawatt through rack-level power management, campuses that ramp with the grid, and cooling that uses no water.

The Slowdown Question

Ed Nelson, co-founder of the summit, put the question to Lip-Bu Tan, CEO of Intel, on the main stage: does the slowdown call mean the infrastructure buildout has to pause, or does it shift the focus toward inference at scale? Tan responded by saying he’s seen multiple cycles of technology growth, but this one is bigger.

“…Some correction is kind of natural, and I don't overreact to it,” Tan said. “But something that you have to pay attention (to): We live in the Silicon Valley. It’s a bubble. And we all get so excited about AI. When you go to a different part of the country…you find that there's a very strong anti-AI sentiment.”

Tan said leaders in the industry have to address the negative side directly while continuing to adopt the technology, prioritizing security and ensuring that data and privacy are not compromised.

The Workload Changed

Ian Buck, VP of hyperscale and HPC computing at NVIDIA, put numbers on how much the inference workload has changed. The 2023 chat workload that became the industry benchmark ran about 1,000 input tokens, a 4,000-token cache and about three turns, with a person reading each answer before asking the next question. The agentic benchmark Buck cited averages 142,000 input tokens. He described agents running 65 turns to reach an answer. Commercial services already run thousands of sub-agents, he said. He called the new workload 100 times more demanding.

“Agents have now taken humans out of the loop,” Buck said, “and the compute will be consumed as fast as the agent can make the turn.”

Demand is not slowing

Dave Patterson, distinguished engineer and Google Fellow, described hardware staying in service long past its amortization schedule inside Google. TPU v2 and v3, chips from eight and ten years ago, still run inference. Google amortizes accelerators over six years and CPUs over eight, and Patterson said the hardware will last longer than that.

“There’s just such a desperate need for cycles that people are absolutely keeping them beyond the amortization,” Patterson said.

Bloomberg’s Dina Bass asked Sachin Katti, VP of compute strategy and GPT-Infra at OpenAI, whether the safety and alignment discussion affects OpenAI’s compute plans. He said it increases the need for compute. Making very capable models safe requires safety models and alignment models. Those have to be trained and served as well.

“If anything, I think it actually double underscores the need for more compute,” Katti said.

During a TechArena panel, an audience member asked when the industry gets its next moment to pause and reset expectations. Erwan Menard, SVP of product management at Crusoe, compared the current supply imbalance to the flooding of hard disk drive factories in Thailand after a monsoon in 2011; at the time, Thailand manufactured ~40% of the world's hard disk drives. He called it the closest precedent in his more than 25 years in infrastructure.

“What we’re experiencing now is way bigger than that at every layer of it, which tells us that demand is essentially insatiable,” he said.

Two years ago, Menard said, the question at a session like this would have been what a provider planned to do with the Hopper GPUs on its balance sheet. NVIDIA was shipping a new generation every year, and there was no time to monetize the old silicon before the amortization schedule ran out. Two generations later, the GPU-hour price of a Hopper cluster has risen since the beginning of the year. Menard called that another validation of demand.

Donald Thompson, distinguished engineer at Microsoft, took the audience question after Menard.

“A lot of it is irrational,” Thompson said. “And I think it’ll take an exogenous event to create a pause. I don’t think that the market itself will pause. Even if it’s in our best interest to pause, I don’t think we will self-regulate in that way.”

Asked what kind of event, he said geopolitical.

Inference is Production

John Roese, global CTO and chief AI officer at Dell, described what it took to get AI into production inside his own company. Dell had 900 AI projects when he took the role. The company discarded them and restarted with four strategic projects covering sales, services, supply chain and engineering. Roese put the result at $30 billion in revenue growth over two years alongside a decline of about 7% in absolute cost.

“To get an enterprise into production is non-trivial,” he said. “Many of the things you have to deal with have nothing to do with the technology.”

Power is the Constraint

NVIDIA’s Ian Buck said the company does not think in terms of absolute token performance.

“The metric that matters is your total token throughput per megawatt,” Buck said.

Buck described software that manages the power of each rack in real time and coordinates with the data center management system. On one model he cited, GPU power ranged from 600 watts to 1,900 watts. Operators provision racks for that peak. Buck said the gap between provisioned power and actual draw is capacity operators are not using. Managing to the actual draw lets them install more racks inside the same provisioned power. He put the gain at up to 40% more compute in the same megawatts.

The Energy Wall

Menard described the constraint from the operator side. Crusoe sources energy, builds data centers, manages GPUs and serves tokens. He said the company optimizes for the value it extracts from each megawatt.

“If you look at 2028, what you’re optimizing is availability of megawatts,” he said. “So, you need to be ready for that because the energy wall is coming and it’s gonna take a lot of time to remove it.”

Crusoe may offer its customers infrequent service degradations to avoid stressing the grid, Menard said, because the company shares that energy with the broader community.

Katti described a similar approach to the grid at the 8-gigawatt Ohio campus OpenAI is building with NVIDIA. Jobs can ramp down when grid demand spikes and ramp up when it drops.

“This can help stabilize the grid in Ohio,” Katti said.

Buck said a scheduler that knows where each job runs and how much power it draws can balance delivery across rows of the Ohio site. He put the gain at 30% to 40% more GPUs inside the same allocation. The site’s power system was designed for that control, from the compute trays up to the transformer.  

More From What Is Already Built

Katti also described a second source of efficiency: a model tuning its own inference. OpenAI took a checkpoint of its Astra model, already optimized for Blackwell, and ran it on Rubin with minor changes. Katti said it ran three times faster out of the box. The team then had the model optimize its own inference serving on Rubin. Over 72 hours, Katti said, the model produced another 2x. He said that work would have taken weeks or months in the past.

Two announcements applied the premise of more tokens per megawatt to installed infrastructure. Axiado previewed a platform efficiency controller that runs AI agents on management silicon. The agents scale processor voltage and frequency and drive cooling from measured conditions rather than worst-case assumptions. Hammerhead AI announced a distribution agreement with TD SYNNEX for a 1-megawatt SKU of its power orchestration software. The offering targets operators facing utility interconnection queues that run several years in major U.S. markets.

Nick Harris, founder and CEO of Lightmatter, described a different route to more capacity without new power. Optical interconnect spans a kilometer, so operators can spread current-generation GPUs across existing facilities rather than concentrating them in new high-density racks.

“It’s reusing data centers that are already there, because optics doesn’t care,” Harris said. “A kilometer reach is not a problem.”

The Water Question

Robert Hormuth, corporate vice president of architecture and strategy at AMD, addressed the public reaction to data center siting, power usage and water usage. He said the industry needs to do a better job governing all three.

“I don’t know if I want one in my backyard either,” Hormuth said.

Airsys, a global provider of specialized thermal management and engineered cooling solutions, won TechArena’s inaugural Ad Astra: AI Infrastructure Competition for LiquidRack ahead of the summit. LiquidRack integrates cooling into the rack rather than relying on facility infrastructure. The system uses patented spray cooling, which Airsys says transfers heat up to three times more efficiently than conventional methods. For data centers running LiquidRack, Airsys reports a PUE below 1.02, no water consumption and 80% less dielectric fluid than immersion cooling. Jeff Moore, VP of strategic partnerships at Aegis Cooling, an Airsys company, joined the Day 1 panel on power and cooling at scale.

The TechArena Take

The slowdown conversation and the infrastructure conversation ran on separate tracks in Santa Clara, and operators should keep them separate. Frontier labs pacing capability gains does not reduce the compute required to serve models already in production. The safety work those labs described adds to it.

What changed at this summit is the unit of account. Tokens per megawatt showed up in NVIDIA’s keynote, in Crusoe’s operating model and in the claims vendors brought to the floor. Operators plan around the power they can get. Rack-level power management, grid-aware scheduling, models tuning their own inference and cooling that uses no water all add capacity without adding megawatts.

Crusoe's Erwan Menard gave the timeline worth watching. Software moves in days, he said, and the energy world moves in multiple years. Operators planning for 2028 are not optimizing for chips. They are optimizing for the megawatts they can secure.

The best per-accelerator server result on MLPerf’s DeepSeek R1 reasoning test improved 5.7x in a year, according to MLPerf Inference v6.1 results released Wednesday by MLCommons, the benchmark setter for the AI industry.

The round set participation records, with 30 organizations submitting 486 data center and edge performance results. Five organizations and one individual contributor submitted for the first time. MLCommons listed five new processors, accelerators and platforms in the results, among them NVIDIA’s Rubin GPU and the Vera Rubin NVL72 rack-scale platform in preview. MLCommons added two tests, one for end-to-end retrieval-augmented generation (RAG) and one for agentic inference at the edge.

Round-Over-Round Gains

The DeepSeek R1 result sits at the top of a 2.7x to 5.7x range the consortium reported for per-accelerator gains over the past year. The best per-accelerator server result on the visual language model test improved 2.99x against v6.0 six months ago. Llama 2 70B, in its sixth round, showed a median per-accelerator gain of 5.58x since v4.0.

MLCommons attributed the range to software and new platforms, and it named next-generation accelerators and lower-precision formats as the two key drivers behind the Llama 2 70B gain. Miro Hodak, MLPerf Inference working group co-chair, said in a press briefing that early rounds ran FP8, while almost all v6.1 results used FP4.

Submitters also shifted which models they ran. GPT-OSS-120B drew 112 submissions, the most of any workload in the suite, unseating Llama 2 70B from the top spot. Version 6.1 also opens speculative decoding to GPT-OSS in the interactive scenario, where it joins DeepSeek R1.

The benchmark treats power measurement as optional, and many submissions skip it, said David Kanter, founder of MLCommons and head of MLPerf, when asked during the briefing Q&A whether submitters recorded wall power during runs. Shruti Koparkar, an NVIDIA product marketing senior manager, added that her company submitted no measured power this round, and that normalizing against a rack’s rated power falls short “because you are not actually drawing all the power on every workload.”

Systems Behind the Numbers

Systems got bigger and more varied. Multi-node submissions reached 16, up from 13 in v6.0. Crusoe submitted a 512-accelerator run on AMD Instinct MI355X GPUs, the largest system ever entered in the benchmark. Cisco unified NVIDIA H200 and AMD Instinct MI350X GPUs into one inference pool, which Hodak said was the benchmark’s first submission to combine accelerators from the two vendors. Cisco reported 99.9% aggregation efficiency on Llama 2 70B. MangoBoost reported that its LLMBoost software knit 32 GPUs at four sites into a single serving pool across an intercontinental network, achieving 97% scaling efficiency in offline mode.

Hodak tied distributed inference to production practice. “Distributing models across multiple nodes is a trend in the industry, especially if you want to minimize time to first token,” he said. Cloud companies running more than one vendor’s silicon need a way to ship workload between them, he said, and geographic distribution answers a need for resilience.

The new silicon MLCommons listed runs from the desktop to the data center rack. AMD’s Ryzen AI Max+ 395 is a unified-memory APU for local AI, and Intel’s Arc Pro B70 a 32GB workstation GPU for large-model inference. AMD’s Instinct MI350P brings a PCIe option for air-cooled servers. NVIDIA entered its Rubin GPU and Vera Rubin NVL72 platform in preview, the category for hardware due to ship by the next round.

The Benchmark’s New Tests

Both new tests measure multi-step workloads rather than a single pass:

End-to-End RAG measures a full question-answering pipeline rather than a single model call. Four models handle embedding, reranking, document grading and answer generation, and a sufficiency checker sends the query back around the loop up to five times. It reports documents per second for building the vector database and tasks per second for answering, where one question runs a dozen or more model calls.

Edge Agentic Inference replays 20 recorded agent trajectories drawn from real GitHub issues in SWE-bench Verified, 1,007 turns in total, against Qwen3.6-27B at a 32K context window, and reports mean single-stream latency per turn. Accuracy is gated on a separate run against the Berkeley Function-Calling Leaderboard v4.

Ramesh Chukka, who co-chairs the End-to-End RAG task force, said the effort began more than a year ago with member feedback asking for a benchmark that “measures the performance and accuracy from end to end, not just for the inference.”

Atlas Inference, a first-time submitter, ran the edge test on an NVIDIA DGX Spark and on AMD Strix Halo. “For the past few years, serious agentic work meant a datacenter round trip,” the company wrote in its submitter statement. “That assumption is what this submission is meant to retire.” The company reported that it completed the full 1,007 turns on the DGX Spark in under 64 minutes.

Lambda ran the same agentic test on data center silicon, submitting in the edge category, open division, on eight NVIDIA B200 GPUs. Chuan Li, chief scientific officer at Lambda, said the company swapped the benchmark's 27 billion-parameter reference model for Kimi K2.6, a trillion-parameter model from Moonshot AI. Li said Lambda also processed about 9% more output tokens per second on GPT-OSS-120B than it did six months ago, on the same NVIDIA GB300 hardware.

Orrick Industries, another first-time submitter, reported 274,927 tokens per second on Llama 3.1 8B from six NVIDIA B200 GPUs in the open division, where results are not required to be mathematically equivalent to the reference model.

Edge Agentic Inference already runs through MLPerf’s API-centric harness, and the RAG test is set to follow. Sixteen of the 30 submitting organizations used the harness this round, up from a single open-division submitter in v6.0. That harness is the foundation of MLPerf Endpoints, a suite that plots each system’s tradeoff between total throughput and speed per user. Endpoints opens rolling submissions in October 2026 and is slated to replace MLPerf Inference for the data center in 2027. The TechArena Take

Version 6.1 brings the suite closer to how AI gets deployed. Since it began in 2019, MLPerf Inference has measured single-shot workloads, which still cover a large share of production work: image recognition, transcription and single-pass answers. RAG pipelines and agentic coding sessions bring two more shapes into the suite. Buyers sizing infrastructure for those workloads now have peer-reviewed numbers to back up vendor assertions.

The range of systems across the round is the other story. One submission ran 512 accelerators. Another ran on a single SoC, with client and server both on the device. Cisco pooled GPUs from two vendors, and MangoBoost’s software held four sites together across an intercontinental network. A benchmark that puts all of that through a robust peer review process serves an industry buying at every one of those scales.

MLCommons is rebuilding its inference benchmark while participation sits at an all-time high. MLPerf Endpoints arrives in 2027, and Kanter said MLCommons is revisiting normalization for future inference benchmarks, calling it “absolutely critical.”

Executive Summary

As AI infrastructure continues to grow in complexity, the way it’s measured has simplified to the cost and quality of the tokens it produces. Over the summer, TechArena’s editorial team sat down with more than 20 companies building different components of the AI infrastructure stack to talk about the challenges they face and how to make the cost of a token predictable. A pattern surfaced in our discussions: Each layer that solves its own bottleneck hands a harder problem to the layer above, from compute to memory to the network to the interconnect, until the constraint turns physical and the integrated rack becomes the unit of design, with power, cooling, and firmware as partners to the silicon. Operators who run their infrastructure as one integrated machine can forecast their own cost per token with a confidence the stitched-together alternative cannot match. Every seam between vendors and layers hides variance someone has to absorb. Integration removes the uncertainty operators create for themselves, and silicon roadmaps, model efficiency, and demand will keep moving on schedules nobody controls, which is exactly why no operator can afford to stack self-made uncertainty on top. In this era of AI infrastructure, integration has become the quiet requirement underlying all the others.

Opening: The Unit of Value

Every currency needs a mint, and every mint has one job: strike the same coin, at the same weight, at a cost the treasury can predict. The AI economy is building its mints now. Its coin is the token, struck each time a model answers a question, writes a line of code, or completes a step in an agent’s chain of work. And the mints striking it hold thousands of parts from dozens of vendors, similar to a rocket or a jetliner, with a key difference: This machine has to hit its numbers every hour, for years, at a cost promised in advance.

The AI buildout is well underway. By Dario Amodei’s math, the AI industry will stand up 10 to 15 gigawatts of computing capacity this year alone at a cost near $10 billion per gigawatt. Meta is planning a single site, Hyperion, with plans to scale to 5 gigawatts over several years, on a footprint that would cover a significant part of Manhattan. Sundar Pichai said in Alphabet’s Q3 2025 earnings call that Google was “processing over 1.3 quadrillion monthly tokens, more than 20x growth in a year.” Commitments of this size get made years before the first token ships, so every buyer in the market is underwriting a forecast. An operator that cannot forecast its own cost per token hands that uncertainty to whoever finances the buildout, and capital charges for uncertainty.

Neocloud providers already operate by this rule: CoreWeave reported its average customer contract stretching from four years to five as its backlog grew, and long commitments at known economics are what make a gigawatt bankable. Gigawatts and dollars measure the size of the machine. Its nature is harder to hold in one view. An AI data center brings together silicon, memory, storage, network fabric, optics, power trains, and cooling loops, thousands of parts from dozens of vendors that all have to work in concert. The mint’s discipline of same coin, same weight, same cost holds only when every one of those parts does its job. So, every AI budget conversation eventually lands on the same two numbers: what a token costs to produce and whether its quality holds. The question buyers now put to every vendor, every operator, and every architecture has become: Who can make the price and quality of a token predictable over time?

The industry sells itself in superlatives: the biggest cluster, the fastest chip, the largest model. Next to those, predictability sounds like a modest goal. It is the harder one to deliver.

Lynn Comp, TechArena advisor and Intel’s head of global sales and go-to-market, and vice president of its AI Center of Excellence, has watched enterprises cross from experimentation into production, and she hears the question in its rawest form.

“No CFO is happy when they are told ‘I don’t know how much budget I will need for AIOps at scale, nor can I guarantee I can hit that budget’ by their sysadmins and IT architects,” she said.

A proof of concept can tolerate surprise. A production line cannot.

The people who buy the most compute frame the goal the same way. Satya Nadella told investors on Microsoft’s Q1 FY2026 earnings call that the company’s fleet objective is “maximizing tokens per dollar per watt,” a metric that compresses the whole infrastructure stack into one ratio. Sam Altman put the demand side on the record in early 2025: “The cost to use a given level of AI falls about 10x every 12 months, and lower prices lead to much more use.” Those two statements together explain the paradox this report unpacks. Token prices keep falling, and budgets still surprise their owners, since demand, context lengths, and workload mix all grow faster than any one component improves. Falling unit prices make predictability harder to deliver, and the operators who can promise it hold the scarcest product in the market.

Diane Bryant has watched the industry rebuild itself around this kind of prize before. Her career runs from Intel, where she served as CIO and led the data center group, through Google Cloud and board seats including Broadcom. She describes the current buildout as the unwinding of the standard data center she helped create. The industry spent 20 years converging on 19-inch racks, common heights, Ethernet, PCIe, DDR, and one processor architecture. By 2016, she noted, x86 held 99.5% of server CPUs. Homogeneity kept operations simple and markets competitive, and enterprises demanded it, accepting the efficiency any general-purpose design gives up on a specific workload.

AI broke the bargain. Data center demand now concentrates in a handful of companies rather than thousands of enterprises, and those companies own their software stacks, so no processor architecture holds them.  

“With scale, these companies can afford to customize their infrastructure, tuned to run a particular workload,” she said. Her math on why is stark. Working from Anthropic’s public statements on revenue and capacity, she calculates that the company generates “$30B in revenue per 1GW of data center power.” An operator who tunes infrastructure to extract more compute from that gigawatt lifts revenue and cuts operating cost at once.  

“The ROIC is massive,” she said. Custom silicon, custom racks, custom networks, and custom orchestration all follow from that arithmetic.

Bryant rejects the alarm that usually attaches to this story.  

“It used to be Intel that had the power-on infrastructure. It’s shifted to the hyperscalers and AI model players. It’s naive to think this is new,” she said.  

Power over infrastructure has moved before. What matters for everyone downstream is what the new holders of it now require. The operators one tier below the giants feel those requirements without hyperscale budgets to absorb mistakes. David Driggers, CEO and founder of  Cirrascale Cloud Services, lives on the difference between a well-matched deployment and a wasteful one.  

“Inference is forever, as we like to put it, and those costs compound,” he said.  

Training is an event with an end date. Serving tokens is a permanent operating condition, and every inefficiency baked into the infrastructure repeats with every request, forever.

So the stakes are set. Tokens are the unit of value, predictability is the prize, and the customers with the most buying power are rebuilding everything in pursuit of both. What follows is a walk up the stack in the order the industry built it, one silo at a time. Every layer that solves its own bottleneck hands a harder one to the layer above. The constraint starts at the chip.

Part I. The Silo World: Components in Order

1. Compute: The Driver of AI Infrastructure Performance

A token’s cost starts in silicon, and the numbers on accelerator spec sheets no longer predict it. Delivered compute now depends on process node, advanced packaging, memory bandwidth, and interconnect reach, and the gap between peak performance on a slide and sustained throughput in a rack has become the first place token economics go wrong. The interviews for this report converge on a replacement metric from three separate directions: useful work per watt.

Eddie Ramirez, vice president of marketing for Arm’s infrastructure business, starts from the constraint that reframes everything else.  

“AI infrastructure is increasingly constrained by fixed power, cooling, and rack density rather than demand for compute,” he said.  

Once the power envelope is fixed, efficiency stops being a virtue and becomes the whole game.

“Efficiency therefore isn’t simply about lowering TCO, but it is about how much useful AI work can be delivered within a fixed power envelope,” Ramirez added. On that accounting, a chip that wastes watts costs its owner twice, once in the power bill and again in the tokens those watts never produced.

The merchant silicon leaders have adopted the same arithmetic. At GTC 2026, NVIDIA CEO Jensen Huang put it in revenue terms: “AI factory revenues are equal to tokens per watt, so with power constraints every unused watt is revenue lost.” He named the output in the same breath, declaring that “tokens are the new commodity,” and committed the company to a new architecture every year to keep chasing the ratio. When the company selling the most accelerators in the world describes its own product in tokens per watt and revs it annually, the metric has finished its migration from engineering detail to industry scoreboard.  

Qualcomm’s entry into data center inference tests a specific bet: that two decades of thermally constrained phone design taught the company the right instincts for a power-constrained rack. “Designing for efficiency first enables us to optimize the entire rack around cost per token and power efficiency rather than chasing benchmark peaks,” said Tony Pialis, executive vice president and general manager of data center. The phrase to notice is “the entire rack.” Even the silicon vendors have stopped talking about chips in isolation, a shift this report returns to at its fulcrum.

Manuel Botija, vice president of product at Axelera AI, applies the harshest version of the discipline. His chips go where the resources run out.  

“Data center architectures are built for abundant power, abundant budget, and abundant time. None of that is available at the edge,” he said.  

Axelera's answer is digital in-memory computing. Inference runs mostly on matrix multiplication, and the chip performs that math directly inside the memory cell, so the data barely has to move at all.

“A unified architecture that handles both on the same silicon keeps cost per token stable as workloads evolve from vision to generative AI, which is the discipline that will define the platforms operators can build a business on two years from now,” he said. His premise, that moving data now costs more than computing with it, is the deeper truth the next section takes up.

Buyers, meanwhile, have learned that no single accelerator wins everywhere.  

“No single chip serves a 1B-parameter model and a 400B-parameter model economically, so our customers get the accelerator that fits their workload instead of bending their workload to fit whatever one vendor is selling,” Driggers said. “In training, you can run a thousand GPUs and barely notice a network hiccup. In inference, that same hiccup is a failed request in front of a customer.”

Matching workload to silicon is itself a lever on cost per token, and it only works when the rest of the stack can feed whatever chip wins the assignment.

One more gap defines this layer: the distance between the performance a buyer purchases and the performance a rack delivers. Peak throughput on a launch slide assumes a fed accelerator, and every section that follows in this report describes a way accelerators go hungry. Supply adds its own tax, with packaging capacity, HBM allocation, and lead times shaping the performance actually available to buy in any given quarter.

That is where the trouble begins. Push the compute ceiling higher and the chip starves. The constraint moves off the die and into memory.

2. Systemic Performance: Storage and Memory

The memory wall stopped being a research topic and became a line item.  

“Model sizes are growing far faster than memory bandwidth and capacity, creating what the industry increasingly describes as the memory wall,” Pialis said. He carries the point into the power budget. Memory architecture matters, he said, because “data movement increasingly consumes more energy than computation itself.”

Inference made the wall taller. Randy Kreiser, field CTO at Graid Technology, draws the distinction that defines the new constraint.  

“Training is limited by how fast you can process data; inference is limited by how much conversational state you can keep available,” he said. Every active request carries a KV cache, the model’s working memory of a conversation, and that cache grows with context length and concurrent users rather than model size. Agentic workloads compound it, since a long-running agent accumulates state across every step it takes. “The constraint is not the GPU; it is the memory available to hold and reuse state,” Kreiser said. HBM cannot rescue the economics on its own. It arrives soldered to an accelerator, and buying memory by buying compute you do not need is the waste Lynn Comp pointed to in the Opening — the one CFOs refuse to fund.

Storage vendors read the same shift as a change in their job description.  

“The shift is from ‘store it cheaply and reliably’ to ‘keep the GPU from ever waiting,’” said Jeniece Wnorowski, director of content strategy and industry expert programs at Solidigm. “Every idle GPU second caused by a stalled data loader, slow checkpointing operation, storage-network congestion, or memory bottleneck directly increases training costs and degrades economics.”  

The company’s high-density NVMe now plays a role no storage roadmap predicted a decade ago.

“SSDs increasingly act as a memory-expansion tier, storing model weights or KV cache data that can be swapped in and out of GPU memory on demand,” she said. Capacity still matters. It just stopped being the headline.

The economics of failure here are unforgiving, said Andy Pernsteiner, field CTO at VAST Data.

“Every cycle spent waiting on I/O is a cycle that’s gone for good. You can’t bank idle GPU time and use it later,” he said. “For model builders, GPUs are the most expensive resource in their pipelines. For nearly everyone else, GPUs are the scarcest resource they have, outside of skilled staff.”

Agentic workloads deepen the dependency. An agent’s retrieval step rarely ends at a vector search; it usually needs the structured or unstructured source data the vector points back to. That puts the data platform in the serving path of every request.

Turning storage into a memory tier only works if the tier is fast enough, and Kreiser’s benchmark numbers show how narrow the margin is. In a controlled vLLM and LMCache test on a 235 billion parameter mixture-of-experts model across four NVIDIA H200 GPUs, conventional Linux software RAID made things worse, stretching time to first token from 29.4 seconds with no offload to 36.6. Graid’s GPU-accelerated approach, which runs RAID’s parallel parity math on a sliver of an installed GPU, cut it to 9.0 seconds, 3.27 times faster than no offload at all.
‍

“The business case is simple: Fetch must beat recompute. If it does not, the offload tier works against you,” Kreiser said. “Adding protection to an insufficiently fast storage path can make inference worse, not better.”

Graid organizes its agentic storage portfolio by deployment scale, server to rack to platform, aligns the platform tier with NVIDIA’s STX reference architecture, and plans native execution on BlueField-4 DPUs in the second half of 2026. He expects buyers to grade storage on new metrics.  

“Over the next two years, storage will increasingly be evaluated in inference outcomes rather than raw capacity: cost per million tokens served, cache-hit rate, and time to first token,” Kreiser said. “Terabytes remain necessary, but they stop being the headline metric.”

The pattern of the silo walk is now visible. Memory and storage achieved their assignment, keeping the accelerator fed, and the solution multiplied the traffic between nodes. A fed node is fast. A thousand fed nodes have to talk to each other, and the constraint jumps to the fabric.  

3. Systemic Performance: Network

AI clusters fail as teams, and the network is where the teamwork happens. Training and large-scale inference run collective operations that force thousands of accelerators to exchange results and wait for the slowest participant, so one congested link or one mistuned switch taxes every GPU in the job. Tail latency, an afterthought in traditional networking, becomes the governing statistic. Operators respond the expensive way, overprovisioning bandwidth they will rarely use to insure against the percentile that ruins the job. The relevant number is job completion time, and it belongs to the fabric.

The technology contest at this layer has settled into a familiar shape. InfiniBand held the early AI clusters on the strength of its latency discipline, and purpose-built scale-up fabrics hold the frontier. Ethernet absorbs capability from both, and its ecosystem and economics keep widening its share of new deployments. Buyers push the market toward open standards for the same reason they resist single-vendor racks: A fabric commitment outlasts several generations of the chips it connects.

A fabric commitment outlasts several generations of the chips it connects.

Brandon Draeger, chief marketing officer at Cornelis Networks, makes the case for purpose-built over general-purpose.

“The network is not simply moving bits; it is determining how efficiently the entire machine works,” he said.

Draeger puts a number on what a congested fabric costs: A sustained 1% utilization improvement across 10,000 GPUs, at an assumed cost of $4 per GPU-hour, works out to roughly $3.5 million in annualized compute capacity.

“At scale, a small fabric inefficiency becomes a very large infrastructure bill,” he said.  

Marc Austin, CEO and co-founder of Hedgehog, built a company on the observation that most operators cannot spend what hyperscalers spend to get job completion time right.  

“The network defends it or destroys it,” he said. Hedgehog packages the hyperscaler playbook, open networking and cloud-style abstractions, into software an ordinary operator can run.

“It turns the network from a monthslong engineering project into a product you deploy,” he said. “Any one misconfigured switch quietly taxes every job on the cluster.”

Aanchal Sharma, senior director of product management at Astera Labs, prices the same failure in the report’s currency.  

“Put the fastest chip in the world behind a slow, high-latency fabric, and you’ve built an expensive space heater, with GPUs sitting idle waiting on data instead of generating tokens,” she said. Astera’s connectivity silicon spans the three directions the industry now scales in, up within the rack, out across the cluster, and across sites, and Sharma’s point holds at every one of them.  

“Faster chips buy more compute. They don’t buy less waiting,” she said.  

“Tokens per watt and tokens per dollar get decided in the fabric long before anyone reads a chip’s spec sheet,” she added.

The fabric is also where the industry’s standards politics now play out. Bryant catalogs the fragmentation: Scale-up connectivity has split into competing options, with NVLink and UALink on one side and the Ethernet-based SUE and ESUN on the other.

Sharma makes the case for keeping those interfaces open.  

“An open fabric means an operator never has to bet the entire rack on one company’s roadmap,” she said.

The network carries one more assignment in the integrated machine, and it is the one the arc of this report depends on: composability. Infrastructure that gets pooled and reassembled around a workload, rather than fixed at the moment of purchase, needs a fabric that can redraw the machine’s boundaries in software. Austin sees a solved problem waiting to be borrowed.  

“Composability is fundamentally a network abstraction problem, and hyperscalers already showed the answer: open networking plus VPC abstractions,” he said.  

Operators learned in the cloud era to treat compute as fungible. AI infrastructure brings the same lesson one layer down, with the fabric as the instrument.

Every fix at this layer raises the load on the physical medium underneath it. Faster fabrics push more bits through copper that must carry them farther, at a higher density, and on a shrinking power budget. Copper is running out of room, and the constraint drops into the interconnect itself.

4. The Interconnect: Optical and the Limit of Copper

As data rates climb, electrical signaling loses reach, bandwidth density, and power efficiency at the same time, and the industry’s answer is to move light closer to the silicon until it arrives inside the package. Co-packaged optics is the structural version of that answer, and it changes what the interconnect is: a component of the chip rather than a cable between boxes.

Vishal Chandrasekar, director of product management at Ayar Labs, states the goal in system terms.

“They are trying to connect thousands of accelerators so they can operate as a single unified system, with the bandwidth and latency needed to support increasingly large AI models,” he said of the operators driving demand. Copper forces a choice between bandwidth and distance at the moment AI needs both.

“CPO removes that tradeoff by using light to extend high-bandwidth, low-latency connectivity across tens of meters,” he said, citing up to 10 times higher bandwidth, 10 times lower latency, and three to five times better power efficiency than copper and pluggable alternatives. “A successful demonstration matters, but customers also need confidence in reliability, supply, packaging, fiber attachment, thermal performance, and production yield.”

Under the optics sits a materials problem, and Robert Blum, senior vice president of sales and marketing at Lightwave Logic, works at the layer where it gets solved. The materials that carried the industry to 200 gigabits per lane, improved III-V compounds and silicon photonics, are reaching their ceiling.  

“New materials are required for the next modulator generation where 400 Gbps speeds are needed,” Blum said. His company’s electro-optic polymers compete for that generation.

“EO polymers have really improved in performance and reliability, thanks in part to the lessons learned from the OLED industry, and are now ready for deployment,” he said.

Foundries favor the polymers, Blum said, since they integrate into standard silicon photonics processes more easily than lithium niobate.

Tying this back to the token, moving a bit costs energy, and the interconnect moves more bits than any other layer. Optics attacks the joules per bit directly, and every picojoule saved in transit returns to the power budget as compute.

Co-packaging also rewrites the manufacturing contract between the optics and the silicon they serve, and Blum’s description of the change is the integration thesis in miniature. Optical assemblies for pluggable transceivers tolerate standard solder reflow and wire bonding.  

“For CPO, you need to integrate much more tightly with the switch ASIC, CPU, or GPU,” Blum said, which pulls in hybrid bonding, higher processing temperatures, and more complex assemblies. “All this requires much closer collaboration with system integrators.”

A component that once shipped in a box now gets engineered alongside the chip it feeds.

“Optics tends to be more complicated than copper,” he said, with high-fiber-count detachable connectors still a bottleneck and optical engine form factors not yet standardized.

What remains is the least glamorous part of any technology transition.  

“The real race is about ramping production capacity and getting the 400G ecosystem in place,” Blum said. Lightwave Logic has five Fortune Global 500 customers engaged in prototype testing and targets high-volume production in 2027, with the remaining gates spread across foundry process maturity, back-end qualification, and an ecosystem of DSPs, SerDes, connectors, and laser sources that all have to arrive together.

That phrase, arrive together, is the hinge of this report. Optics solves the last movement problem inside and between racks, and in doing so, it removes the excuse every silo had for engineering alone. Once the parts can move data as one machine, someone has to build them as one machine. The problem stops being electrical. It becomes physical.

Part II. The Turn

5. Rack-Scale: The Unit of Compute Becomes the Building Block

Every voice in Part I described a different layer and ended in the same place.

“AI infrastructure is increasingly a systems problem rather than a chip problem,” Qualcomm EVP Pialis said.

The unit of compute is no longer the chip or the server. It is the rack, designed, sold, and bought as a single product, with the row not far behind.

“AI infrastructure performs best when power, cooling, controls, and software are designed as one integrated technology system instead of assembled piece by piece,” said Steven Carlini, chief advocate for AI and data centers at Schneider Electric.

Operators using the catalog model, picking servers from one vendor, power from a second, cooling from a third, and management software from a fourth, could once assume the interfaces between those purchases were forgiving. At AI densities, no interface is forgiving. A megawatt rack punishes every assumption its designers did not share.

Adam Morton, CTO of data center infrastructure at Flex, makes the same case from the manufacturing side. “Designed as a system from the outset, [integrated AI racks] are generally more efficient, cost-effective, and scalable than racks that come together as a collection of components,” he said.  

“Customized data centers have been the industry standard for decades. That era is coming to an end,” Morton said. “‘Snowflake’ projects with bespoke designs, supply chains, permitting paths, and commissioning plans don’t scale, and certainly not at an accelerated pace. … The future AI factory will be designed once and repeated many times.”

Diane Bryant tracks the same turn from the buyer’s side.

“Racks are all now custom built to the dimensions that optimize power and cooling for the custom xPUs,” she said. The 19-inch rack survived every previous platform shift in computing. It did not survive this one. When the companies with the deepest pockets abandon the most durable standard in the data center to win thermal and power headroom, the message to the rest of the market is unambiguous.

The merchant vendors heard it. AMD CEO Lisa Su highlighted the company’s Helios platform during an earnings call with investors, describing it as an integrated rack-scale solution featuring Instinct MI400-series GPUs, Venice EPYC CPUs, and Pensando NICs. Built on Meta’s double-wide Open Rack Wide standard, Helios was designed and “optimized for the performance, power, cooling, and serviceability required for the next generation of AI infrastructure,” Su said. She reported “a lot of interest in the full rack-scale solution.”

NVIDIA sells its flagship as a rack. The neoclouds pour the same logic into concrete: Chase Lochmiller, CEO of Crusoe, calls the data center “the new unit of compute.” Each phrasing moves the boundary of the product outward, from the chip to the rack to the building.

Moving the boundary moves the accountability with it, and the industry has not finished deciding who holds it. A rack that spans one vendor’s silicon, a second vendor’s power train, a third’s cooling loop, and an integrator’s assembly has to answer an old question at a new scale: When the machine underdelivers, whose machine is it? Our interviews suggest the market is answering with engineering rather than contracts. Thermal, power, and management vendors describe co-design relationships that start at the silicon roadmap, years before a purchase order. The alternative is discovering incompatibility at commissioning, when every idle day burns the most expensive depreciation schedule in the industry.

Component vendors now design for that boundary or design themselves out of the market. Randy Kreiser of Graid sees it from the storage layer.

“Once the rack is the unit of deployment, storage cannot be treated as a component to integrate afterward,” he said. Graid organizes its roadmap by deployment scale rather than by SKU for exactly that reason, and some version of that reorganization appears in nearly every interview in this report.

Rack-scale architecture also delivers the promise the cloud made and AI briefly broke: composability. A machine designed as one system can be pooled, partitioned, and reassembled around a workload in software rather than fixed at the moment of purchase. The fabric abstractions that Hedgehog’s Marc Austin described in the network section are the mechanism. The rack designed as a product is the precondition.

“The technology is converging faster than most organizations are,” said Lakecia Gunter, TechArena advisor and former global CTO and corporate board director. “Companies cannot operate integrated AI infrastructure through disconnected technology, facilities, finance, security, and sustainability teams. The operating model must evolve with the architecture.”  

Treating the rack as one product also knocks down a wall inside the operator’s own organization. The teams who run servers and the teams who run power and cooling grew up in different professions, with different tools, different budget lines, and different definitions of an emergency. A rack designed as one machine makes them tenders of one converged system. Facilities engineers now read GPU telemetry, and IT architects sit in utility interconnection meetings. The hardware merged first; the people are catching up. The chip and the facility used to be separate conversations. From here on, they are the same one, and the rest of this report walks the layers those merged teams share. The first is the one gating the entire industry: power.

Part III. The Integrated Machine: Systems

6. Power as a Design Partner

Power used to enter the conversation after the IT was specified. It now opens the conversation, and often ends it. “Today, speed is driven by time to power,” Carlini said, and the sentence explains more of the current market than any chip roadmap. Utility interconnection queues, substation lead times, and grid capacity now gate AI deployment harder than silicon supply. Satya Nadella made the same point from the buyer’s chair.

“You may actually have a bunch of chips sitting in inventory that I can’t plug in. In fact, that is my problem today,” he said. The scarcest input to an AI factory today is the energized shell around the accelerators.

Operators are answering the queue with every tool that shortens it. Some buy their way into existing capacity, and some build behind the meter. The energy-first developers invert the old site-selection logic, putting the data center where the power is and running the fiber to it. Crusoe built its business on that inversion. Whichever way the shell gets energized, the arithmetic that follows is unforgiving: An unpowered rack has no cost per token, since it has no tokens, and a partially powered facility pays full capital cost for partial revenue.

The premium on an energized megawatt now shows up on income statements. Nebius founder and CEO Arkady Volozh told analysts in August 2026 that the company’s pipeline makes it one of just a few players able to build more than a gigawatt of new capacity a year, and the unit economics reported from that call, tens of millions of dollars in annual revenue per megawatt, make each stranded watt a measurable loss. Carlini frames the resulting agenda in exactly those terms.

“The next challenge is not only securing more power; it is reducing the time it takes to bring that power into service and making every available kilowatt work harder to generate tokens and intelligence,” he said.

Scarcity of that order forces discipline on every watt that does arrive, and the discipline starts inside the rack. Steve Thorne, chief commercial officer at CelLink, makes power delivery a peer of the components it feeds.

“Designing at rack scale forces power delivery to be considered as a first-class constraint alongside compute and cooling, rather than an afterthought bolted on at the end,” he said. “At each step down in voltage, current rises sharply and I²R losses compound accordingly.”

CelLink’s flat, flexible harnesses attack those losses and reclaim tray space, and Thorne’s roadmap points at the same convergence Carlini described, power delivery “co-designed and co-assembled with liquid cooling cold plates as a single unified subsystem.”

Flex’s Adam Morton ties the shift to a specific electrical transition. “Within 800 VDC environments, electrical, mechanical, and thermal management decisions are increasingly interdependent, even as power, cooling, and IT equipment gets disaggregated to accommodate more computing capacity in the rack,” he said. “A co-designed approach to the rack, power infrastructure, and cooling system maximizes performance.”

AI also changed the shape of demand, not just its size. Brandon Smith, vice president of global sales and product management at ZincFive, describes a load profile no facility engineer trained for.

“An AI workload can swing from idle to full draw and back in milliseconds, then repeat it thousands of times an hour,” he said. Batteries designed to sit idle until a blackout solve a problem different from the one Smith describes.

“Rather than sitting idle until the grid fails, the system absorbs the spikes and releases energy as the workload calls for it, shaping load in real time before it travels through the facility and out to the grid. Power moves from passive backup to active stabilization,” Smith said. ZincFive’s case for nickel-zinc chemistry rests on that repetitive, high-rate duty cycle.

“A site that smooths its own demand internally reads as a better neighbor, and that can shape how much capacity gets allocated and how quickly a project breaks ground,” he said. In a market gated by time to power, good citizenship at the meter converts into schedule, and schedule converts into tokens.

Every watt the power team wins arrives in the rack with an obligation attached: It all comes back out as heat.

7. Cooling as a Design Partner

Every watt that enters a rack leaves it as heat, and operators now plan for heat with the same care they give power.  

“Cooling is now a foundational design decision for AI infrastructure,” said Rich Whitmore, president and CEO of Motivair by Schneider Electric.  

“As rack power increases, the challenge shifts from simply removing heat to delivering reliable, efficient, and scalable thermal management across an entire facility.”

The industry crossed the density line where air alone stops working, and the liquid systems replacing it cannot be bolted on after the fact. Scale changed the job, too. Whitmore sets the bar for production readiness in operational terms.

“Production environments require more than excellent thermal performance. They require repeatability, uptime, ease of service, and seamless integration with the rest of the infrastructure,” he said.

Paul Quigley, chief strategic relations officer at Airsys, connects cooling to the constraint running through this whole report: the power budget. His company has spent three decades in precision air cooling and now builds hybrid systems that add liquid where density demands it, which gives him an unusual vantage on how the two share the load.

“Power has become a primary constraint, so every infrastructure decision comes back to how much provisioned capacity can ultimately be allocated to compute,” he said. “The first question should not always be, ‘How do I get more power?’ It should also be, ‘Am I making the best use of the power already provisioned?’”

Airsys proposes measuring exactly that with a planning metric it calls Power Compute Effectiveness, which evaluates how provisioned electrical capacity gets structurally allocated at design time, a complement to PUE’s operational lens. Quigley expects liquid to carry a growing share of the thermal load and air to remain necessary for portions of the IT load, which makes the design question one of proportion.

Ryan Brown, director of data center product management at Phononic, pushes the thermal case down to the component.

“As AI infrastructure becomes more tightly integrated, performance is increasingly determined by the weakest thermal link in the system,” he said.  

The weakest link has moved past the GPU under its cold plate. Optical transceivers, memory, and NICs now throttle systems whose headline silicon is perfectly cooled, and Phononic’s solid-state thermoelectric coolers target those hotspots directly. Precision at the component pays off at the facility.

“By actively and dynamically controlling temperature at the component level, operators can maintain tighter thermal margins on the devices that matter most, even as rack densities continue to climb,” Brown said.

Tighter margins let operators run warmer coolant and cut the cooling headroom they overprovision out of fear.

“Thermal intelligence turns cooling from a supporting utility into an operational tool that helps improve reliability, efficiency, and asset management,” Brown added. A cooling system that senses and reports becomes part of the machine’s telemetry, the subject of the next section. One more wall between operations and IT comes down.

Simon Jesenko, CEO and CFO of Iceotope, closes the spectrum at the whole board. The company’s precision liquid cooling seals the entire server and delivers dielectric fluid exactly where heat appears.

“Cooling the whole server with dielectric fluid allows the operator to cool all heat-generating components within the server: GPUs, CPUs, memory, networking, and PSUs,” he said. His efficiency case rests on first principles. Every heat exchange between fluids gives up energy, so, in his words, “the most efficient system is one which has fewer (or no) heat exchangers.” He also treats the heat itself as inventory rather than exhaust.

“Captured heat can be reused in other deployments. For example, a precision liquid-cooled data center in a hotel basement could use the excess heat to heat the hotel pool,” Jesenko said.  

Two pressures from outside the machine now shape every choice inside it. Buyers apply a sustainability lens with real procurement weight. They ask about water consumption, energy reuse, and the carbon attached to each token, and liquid systems that run warmer coolant, reject heat efficiently, or hand it to someone who can use it perform better on all three counts than air. The installed base pushes back from the other side. Most of the world’s data center square footage was built for air, so operators choose between retrofitting live facilities toward liquid and reserving the densest AI for greenfield builds.

A rack full of power and coolant is still a body without a nervous system. Someone has to make the machine knowable.

8. The Control Plane: Firmware, Telemetry, Software

Integration remains an aspiration until something operates it, and the layer that does the operating spent decades beneath anyone’s notice. Firmware booted the server, reported its health, and stayed out of sight. AI ended that obscurity, said Colin Brix, vice president of marketing at AMI, a Lattice Company. Operators now demand granular telemetry on GPU performance, power draw, thermal behavior, and interconnect health, feeding orchestration systems that act on it automatically.

“Firmware is no longer just reporting health status. It’s becoming the trusted source of telemetry that drives automated decisions around workload placement, power balancing, cooling optimization, and predictive maintenance,” Brix said. “In short, firmware is evolving from a management layer into a data and control layer for the AI factory.”

The stakes of getting that layer wrong scale with the cluster. In a traditional environment, a misconfigured node inconveniences a workload. In an AI factory, Brix said, “a node that is misconfigured, running an inconsistent firmware level, or failing attestation checks can prevent thousands of GPUs from operating at full efficiency.” The boot-to-workload chain, invisible when it works, becomes the foundation of the fleet’s economics. Secure provisioning, firmware integrity validation, and hardware attestation moved from compliance checkboxes to operating requirements.

Automation is only as good as the telemetry beneath it.

“At AI factory scale, telemetry must be accurate, consistent, and synchronized,” Brix noted, since operators now correlate events across thousands of systems at once, and decisions about workload placement, power allocation, and thermal management are only as good as the data feeding them. Heterogeneity works against all three properties. A fleet whose compute, power, cooling, and networking each expose different interfaces and telemetry formats makes automation fragile and lets operational overhead grow with every added vendor.

At fleet scale, the case for unification stops being aesthetic. AMI’s MegaRAC OneTree pulls management of compute, power, cooling, and networking into one open codebase across different silicon, so a patch propagates once instead of dozens of times and every subsystem speaks the same telemetry language. The endgame is a fleet that runs itself.

“The reality is that nobody manually operates a 50,000-node AI factory,” Brix said. “The telemetry and automation architecture must be designed so the fleet effectively manages itself, with humans focusing on policy and optimization rather than individual device administration.”

Predictable cost per token, Brix said, “requires three things: trusted data, automated optimization, and strong security.” The security leg reaches all the way down to power-on.

“The control plane must be built on a common hardware root of trust that validates systems from initial power-on through workload execution,” he said, so an operator knows every node in the fleet runs authorized firmware, trusted software, and verified hardware before it earns a workload.  

The market has started pricing this layer accordingly. Lattice Semiconductor closed its acquisition of AMI on July 27, 2026, in a deal trade press valued at $1.65 billion. A sum that size for a firmware company measures what control of the control plane is now worth. Eddie Ramirez of Arm sees the same promotion happening in silicon, where “the CPU becomes the control plane for the entire rack, coordinating data movement, scheduling work, feeding accelerators efficiently, and ensuring entire system resources are fully utilized.”

A fleet an operator can see is a fleet a customer, a regulator, or a board can ask hard questions about, and those questions are the final layer of the machine.

“Boards should not view AI infrastructure as a technical procurement decision,” Gunter said. “It is a capital allocation, resilience, and competitive-positioning decision that will shape how quickly the company can turn AI ambition into enterprise value.”

9. Privacy and Policy: The Outermost Layer of Trust

The last requirement is the one no rack diagram shows. A token’s price means nothing if its buyer cannot trust where it was made, what data fed it, and who can be held accountable for it. Lynn Comp, whose vantage at Intel spans the enterprises crossing into production, watches trust move from the legal department to the architecture review.

“I’m seeing harder questions being asked about private and hybrid AI as the true costs of frontier models hosted in hyperscaler data centers become more obvious,” she said. Cost and control turn out to be the same question, and buyers have begun answering it with deployment decisions rather than policy memos.

The economics run deeper than compliance, since the data that privacy rules protect is the same data that makes AI worth buying.

“The AI is generic until it applies your business information and context, but it is difficult to see ROI from AI that spends the majority of tokens on the data input processes rather than getting insights from the underlying data,” Comp said. An enterprise’s proprietary data is its entire differentiation in AI. Where that data can safely go becomes an infrastructure requirement rather than a legal afterthought.

Diane Bryant maps the same boundary from the CIO’s chair she once occupied at Intel. In her assessment, an enterprise CIO today should move everything possible to the cloud, since no internal operation competes with hyperscale efficiency, and keep on premises only the workloads with compliance or privacy concerns. The exceptions define the territory. Sovereignty rules, data residency requirements, and sector regulations now shape where tokens get produced as surely as grid capacity does, and they are the reason private and hybrid AI keeps its seat at a table the economics would otherwise clear.

Purpose-built machinery for those exceptions is arriving. Confidential computing has moved from research program to procurement checkbox. Buyers ask whether inference can run without exposing plaintext the model should never see. The hardware root of trust Brix described in the control plane section turns out to be the foundation for every trust claim above it: The same attestation that tells an operator a node runs authorized firmware tells a regulator, a customer, or a court where a token came from and what touched it along the way. Integration pays a dividend here too. A machine designed as one system can answer for itself as one system.

Agentic AI raises the stakes on all of it, and Comp’s recommendations are design constraints rather than aspirations.

“Infrastructure needs to have very tight design constraints for a given AI-based agent, avoiding the ability of an agent to exercise functionality beyond its intended purpose,” she said. “Any decision that would be difficult to defend in a current regulatory framework needs to have a human in the loop.”

Dana Bos, TechArena co-founding advisor and founder and principal of Bos Solutions, sees that gap as the one most AI stack diagrams don’t show.

"Every AI stack diagram stops at the application layer, but the layer that actually determines ROI sits above it: the humans deciding whether to believe the output. Skip that layer and you've built a very expensive system nobody uses correctly," she said.

Trust, in Comp’s telling, is an engineering deliverable with an audit trail, and it pays like one. A token a customer can trust, produced where the rules require, from data that never left its owner’s control, commands a premium over an identical token that cannot show its provenance. Trust is the only layer of the machine the buyer experiences directly, and the whole integrated machine underneath exists to earn it.

This report has traced a single constraint from the chip through memory, fabric, and light, into the physical realities of the rack, and up through power, heat, and telemetry to the question of trust. One question remains, the one this report opened with.

Close: The Requirement

Who can make the price and quality of a token predictable over time?

The operators who run their infrastructure as one integrated machine, and the vendors who design for that machine, hold the pricing power. The ones still stitching a machine together from catalog parts inherit every seam as a variance they cannot forecast. Theirs are mints that strike a different coin every day. The market pays for the mint that strikes the same one.

The evidence assembled here points one direction. Satya Nadella compresses the hyperscaler’s entire strategy into tokens per dollar per watt. Steven Carlini’s power, cooling, controls, and software perform best “designed as one integrated technology system instead of assembled piece by piece.” Aanchal Sharma’s tokens get decided in the fabric, Randy Kreiser’s in the cache tier, Brandon Smith’s in the milliseconds between power spikes, Ryan Brown’s at the hottest component on the board.
‍

“Thinking of agent design like building an appliance helps keep the architecture clean, which then has the byproduct of being more deterministic and predictable,” Comp noted.

An appliance is the humblest thing engineering produces, and the most trusted. It does what it says, every time, at a cost its owner can state. The AI industry, for all its scale and speed, is working its way toward the appliance’s virtue.

The constraint will keep moving. The voices in this report point to where it travels next: trust and energy, the two inputs no engineering team can manufacture alone. The next year offers clean markers for anyone tracking the turn. Watch whether the scale-up fabric standards consolidate or keep splintering, and whether co-packaged optics crosses from prototype to volume on the timelines vendors like Lightwave Logic have staked. Watch how fast the firmware layer consolidates now that Lattice has priced it, and whether time to power shortens or keeps stretching. Each is a proxy for the same underlying question: how quickly the industry finishes becoming one machine.

What will not change is the shape of the winning response, which every section here has traced. Watch the constraint, follow it across the old silo walls, and design the next layer with the last one instead of after it.

Everything this report has argued comes down to a single payoff: the determinacy dividend. It accrues to the operators who treat integration as the requirement underneath all the others, who co-design power with silicon and cooling with both, who can see every node and trust what they see. Those operators will quote a customer a price per token two years out and hit it, and they will collect the capital, the customers, and the trust that follow. In this era of AI infrastructure, integration is the requirement. Everything else is a line item.

‍

Download the Paper

Executive Editor: Allyson Klein

Writer: Rachel Horton

Copy editor: Amy Reifenrath

Reporting by TechArena Editorial Team: Cinthya Anand, Rachel Horton, Deanna Oothoudt, Amy Reifenrath

‍

PORTLAND, OR – Sept.10, 2026 – TechArena today welcomed data center and AI leader Lynn Comp to TechArena Advisory as an AI governance advisor. Having driven go-to-market strategy and business operations across some of the most consequential product cycles in the history of compute, Comp currently leads Intel's enterprise go-to-market for the company's CPU business and is one of the few leaders to have run both product strategy and the revenue behind it, at Intel and AMD.

Comp has led data center businesses through their hardest transitions, launched first-of-their-kind products, and built go-to-market engines. Her range runs from silicon to services and from cloud to edge, the full sweep of the modern compute stack. Today, as an IAPP certified AI Governance Professional and enterprise GTM leader, she sits at the center of enterprise AI adoption, working with some of the largest organizations in the world as they turn ambition into deployment.

"Lynn brings an invaluable combination of deep product strategy, industry activation, and AI operational governance to TechArena Advisory," said Allyson Klein, founder and CEO of TechArena. "Most conversations about responsible AI stay abstract. Lynn makes them actionable. Beyond her unquestioned acumen and earned experience, she brings incredible ethics to the nuances of governance for organizations. I could not be prouder to welcome her."

Comp's areas of practice include responsible enterprise AI governance. She also leads TechArena Advisory's AI governance workshops, which help leadership teams move from responsible-AI principles to operating practice.

About Lynn Comp

Lynn Comp builds the commercial infrastructure that turns data center technology into market leadership. At Intel, she spent more than a decade in product marketing and data center strategy before becoming Vice President and General Manager of the Visual Cloud data center business, where she drove go-to-market for cloud graphics, CDN platforms, and video delivery infrastructure, and launched the company's first discrete GPU for data centers built on the Xe architecture. She then moved to AMD, running the cloud P&L for the EPYC processor and leading server marketing through a period of industry-reshaping competitive growth. She returned to Intel in 2023 and now leads enterprise and go-to-market for the company's CPU business, working with enterprises in the trenches navigating the AI-driven transformation. Comp holds the IAPP AI Governance Professional certification and serves on the boards of NeuReality and Napatech.

She holds a BS in Electrical Engineering from Virginia Tech and an MBA from the University of Phoenix. At TechArena Advisory, she helps companies align product roadmaps with commercial strategy, govern AI responsibly, and build the go-to-market infrastructure to win in AI and data center markets.

About TechArena

TechArena brings together business advisory, world-class marketing with tech domain expertise, and trusted insights for companies shaping the AI era. The platform covers AI, data centers, semiconductors, cloud-native infrastructure, networking, edge computing, and sustainability.

Yunshui Chen often recommends the book “The Lessons of History,” by Will and Ariel Durant, because it reminds him that technology changes much faster than human nature. As founder and CEO of Airsys, he has spent 31 years testing that idea in the business of keeping computers cool. 

Chen formed Airsys in 1995, when China was building telecom networks faster than conventional cooling could protect the digital equipment going into them. Conventional comfort cooling could not deliver the reliability those demanding environments required. Chen could see that thermal management was about to become a fundamental constraint on the industry’s growth, and he built a company to address it.

Three decades later, Airsys’ mission remains the same, Chen says: “Remove the physical barriers that stand in the way of digital progress.”

In the AI era, the barriers have changed. Today’s workloads require concentrated computing power at densities far beyond those of most telecom facilities, and cooling, electricity and water have become limiting resources.

Chen’s way of looking at the problem has not changed.

“I can think simultaneously as an engineer, an entrepreneur and a student of history,” he says. “It helps me connect technical details with a much longer view of where an industry is going.”

That view is visible in how Airsys is built. The company develops air cooling, single-phase liquid cooling, two-phase technologies and heat-rejection systems, and that breadth, Chen says, means the company “can design around the customer’s complete computing environment rather than force every load into one technology.” That portfolio reflects Chen’s belief that the transition to liquid cooling will take years and will be hybrid rather than binary.

This same long view shapes how he leads. Asked how he wants to leave his mark on the people who work for him, Chen says he wants them “to become more capable, more confident and more independent than when we began working together.”

“A leader’s greatest legacy is not a group of followers, but a new generation of leaders,” he says.

Hybrid, Not Binary 

No one cooling technology is ideal for every application, Chen says. Each has limits. Immersion cooling delivers excellent thermal performance, but servers have to be installed and serviced inside fluid-filled tanks, which changes the data center’s physical layout, its maintenance procedures and its equipment ecosystem. Conventional cold-plate direct-to-chip systems are more familiar to the industry, but they place conductive, water-based fluids close to high-value electronics and require carefully engineered cold plates, manifolds and connections. The resulting systems can carry significant equipment and installation costs, he says.

His picture of the next several years is a facility running all of it at once.

“A data center may have conventional servers, liquid-cooled AI racks, low-temperature support loads and extremely high-density processors operating in the same facility,” he says.

Air handles the lower-density and residual loads. Single-phase liquid can support many current CPUs and GPUs, and two-phase cooling goes where heat flux is most demanding. Each connects to the high-, medium- or low-temperature heat-rejection loop that suits it.

Operators who assemble that mix from several vendors inherit the seams between them. Chen lists the risks that collect at those boundaries: controls integration, water chemistry, hydraulic balance, commissioning, maintenance and performance accountability. Owning every layer lets Airsys take responsibility for the whole system. 

“We are not trying to sell one cooling product everywhere,” he says. “We are trying to optimize the entire infrastructure around the customer’s computing requirements.” 

The Megawatts That Never Reach the Servers 

Chen would rather talk about Power Compute Effectiveness, or PCE, than power usage effectiveness, the industry’s standard PUE metric.

PUE measures the relationship between a data center’s total energy use and the energy consumed by its IT equipment. It does not, by itself, reveal unused provisioned capacity. A facility might have a 100-megawatt utility allocation but still be able to deliver only 60 or 70 megawatts to IT equipment. The remaining capacity may have been reserved and supported by facility investment yet cannot be converted into computing output.

Airsys defines PCE as the usable power available to IT and computing equipment divided by the total power capacity provisioned for the facility. It is a capacity-allocation metric, not an established replacement for PUE.

“It reveals how much of the power that an operator has purchased, financed and built can actually produce computing output,” Chen says.

He sees the two metrics as complementary. PUE measures operating efficiency. PCE measures utilization of the facility’s most constrained and valuable resource. For an existing data center, improving PCE can release substantial capacity without waiting years for a new utility connection or constructing a new facility. That is the version of the problem an entrepreneur sees, and it is the one Chen keeps returning to: how much productive computing the industry can get from every available megawatt.

Beyond the Chiller 

Chen has said publicly that mechanical chillers will eventually be eliminated from data center infrastructure. He treats the prediction as a specification.

Chillers are necessary, he explains, when IT equipment needs coolant colder than the outdoor environment can reliably supply. Eliminating them means moving the entire thermal chain to higher operating temperatures. Servers and cooling components have to be designed and warranted for warmer coolant. Liquid cooling must capture nearly all of the high-temperature heat at its source, so that very little load still needs low-temperature air. Then dry coolers, two-phase systems, heat exchangers and controls get engineered as one integrated high-temperature system. Extreme weather is handled through properly sized heat-rejection equipment, thermal storage or limited trim cooling during exceptional hours.

“Chillers will not disappear from every data center at the same time,” Chen says. “However, I believe mechanical refrigeration can cease to be the default solution for high-density AI facilities. As liquid temperatures rise and heat capture improves, many data centers will be able to reject heat directly to the outdoor environment for most — or eventually all — of the year.”

The same higher temperatures also bear on the question communities are asking about water. Zero onsite water consumption for heat rejection is technically achievable today, Chen says, through dry coolers, air-cooled systems and closed-loop liquid cooling. He is careful with the term. A closed-loop system still needs an initial fluid charge and occasional maintenance. Electricity generation carries its own upstream water footprint. In very hot climates, avoiding evaporative cooling can require more heat-exchanger surface, higher airflow or more energy.

“The warmer the liquid leaving the servers, the easier it becomes to reject that heat directly to outdoor air,” he says. 

Airsys works from the chip outward: capture heat at a higher temperature, minimize low-temperature loads and design the dry-cooling and control systems around those conditions.

“Zero onsite water consumption for heat rejection is not simply a choice of outdoor equipment. It is a complete system architecture,” he says. “Properly designed, it can support large-scale AI development without placing an unsustainable demand on local water resources.”

An Energy Company 

Chen has called Airsys an energy company rather than a cooling company. His reasoning starts with what the customer is buying.

“Customers do not ultimately purchase cooling; they purchase productive computing within a limited power envelope,” he says. “Cooling is one of the places where power is either preserved for computing or consumed as overhead.”

Thinking like an energy company changes the engineering objective. The team considers how every watt moves through the facility, how much becomes useful computing capacity and how much is lost to conversion, cooling and distribution. It also weighs water consumption, equipment life, heat reuse and the embodied resources in any new construction.

The company’s phrase for this is “Balance the Environment.” Chen defines it as balancing the growth of the digital economy against the limits of the electrical grid, the natural environment and the communities hosting these facilities. He insists it be expressed through engineering: higher-temperature cooling, reduced fan and pump energy, water-free heat rejection, longer equipment life and more computing output from existing infrastructure. Sustainability that does not improve the customer’s economics, in his view, does not scale.

“The most durable solutions are those that reduce environmental impact while increasing the return on invested power,” he says.

Made in Woodruff 

In May, Airsys opened the first phase of a 60-acre global headquarters campus in Woodruff, South Carolina. It will house a large-scale 3D-printing operation for cooling components.

Chen says U.S.-based manufacturing brings engineering, production and customer deployment closer together. Customers building critical infrastructure need traceability, consistent quality, secure supply and responsive technical support, along with confidence that their cooling partner can scale with them. Working directly with Airsys engineering and production teams during development, testing and qualification shortens the loop between field performance and product improvement, and it strengthens lifecycle service after installation.

The printers also change what Airsys can design. Complex internal flow paths, heat-transfer geometries and customized components that would be difficult or impossible to machine conventionally can be produced additively, avoiding some of the constraints of conventional machining. That shortens the path from design to prototype, qualification and scaled production.

“The Woodruff campus is more than a factory,” Chen says. “It is a commitment to technological development, manufacturing accountability and long-term participation in the North American data center ecosystem.” 

Inside the Server 

Airsys’ server-level liquid cooling architecture, LiquidPack, delivers dielectric fluid directly to the principal heat sources inside the server through a spray mechanism rather than immersing the whole rack. It captures heat at the chip and preserves much of the familiar rack configuration and service workflow. Properly designed, Chen says, it combines high heat-transfer capability with a relatively small fluid inventory and strong electrical safety.

The tradeoff is precision. Spray distribution, sealing, fluid management and system controls all have to be engineered exactly. He sees particular value in brownfield retrofits, mixed server fleets and facilities that need more computing density without rebuilding their white-space infrastructure.

Two-phase cooling goes further. It uses the latent heat absorbed when a fluid changes from liquid to vapor, allowing the fluid to remove substantial heat at a nearly constant temperature. Compared with single-phase cooling, it supports much higher heat fluxes, reduces temperature differences across the chip and can operate with lower fluid-flow requirements. That can mean smaller components, lower pumping energy and more uniform chip temperatures as processors move toward multi-kilowatt thermal loads. The engineering challenge is greater. Boiling stability, pressure, vapor transport, condensation, material compatibility, fluid containment and long-term serviceability all have to be controlled.

Chen does not see two-phase replacing what came before. He describes the cooling roadmap as an integrated spectrum. Air will keep handling legacy equipment and lower-density components, and single-phase liquid will be the workhorse for much of the market. Two-phase will matter most at the leading edge, where conventional cold plates approach their practical limits. 

Looking Ahead 

Asked what has to be true for AI infrastructure to keep scaling at the pace the industry expects, Chen answers like someone who has watched a buildout before. Cooling technology must become standardized, manufacturable and serviceable at scale, “not merely thermally impressive in a laboratory.”

“Servers, electrical systems and cooling infrastructure must be designed as one system,” he says. “The industry will need higher coolant temperatures, standardized liquid interfaces, clear warranty responsibilities, water-conscious heat rejection and practical upgrade paths for existing data centers. It must also develop the manufacturing and service capacity required to deploy these technologies across thousands of sites.”

Airsys’ role is to connect those layers: air, single-phase liquid, two-phase and heat-rejection technologies, together with system controls, manufacturing and lifecycle support. Chen wants to help operators unlock stranded capacity in existing facilities and design the next generation of high-temperature, zero-water AI infrastructure.

“If we succeed, cooling will no longer be the constraint that determines how quickly AI can grow,” he says.

Technology changes much faster than human nature. That is the theme Chen takes from “The Lessons of History.” Every technology in this story, from the telecom equipment Airsys first protected to the two-phase components it is developing now, has a shelf life. The things that outlast them are human: the need for reliable computing, the limits of the grid and the communities that host the facilities, and the judgment of the people who decide how to work within them. Thirty-one years in, Chen is still building for the slow-moving side of the equation.

Garrett McKibben of Graid Technology joins TechArena's "Data Insights" podcast to unpack KV cache overflow.

For more than four decades, AMI has written the firmware that boots, manages, and secures the world's servers. That kind of longevity in a business long defined by proprietary code makes the company's latest strategic shift notable. On a recent episode of The Control Plane, Allyson Klein, CEO of TechArena, sat down with Zachary Bobroff, VP of product management at AMI, a Lattice Company, to talk about why open source has become central to AMI's firmware strategy, and how the company is untangling what he calls firmware's "five-headed hydra."

As VP of product management, Zachary spends much of his time gathering requirements from AMI's two core customer bases: system manufacturers, including OEMs and ODMs, and the hyperscalers that deploy that hardware at scale. Neoclouds and other large-scale consumers of systems are increasingly entering the conversation too, he said, reshaping how the industry defines its firmware needs.

Historically, OEMs owned platform architecture, Zachary explained. As hyperscalers rose to prominence, influence moved first to ODMs and then to the hyperscalers themselves. Now, with end customers looking to take control of firmware and vertically integrate it, open source has become the best way to reach that goal. In Zachary’s words, "Open source is the best path to innovation, ecosystem adoption, and having everybody on the same page."

OpenBMC and One Tree

AMI has increased its contributions to OpenBMC, the Linux Foundation project that originated at IBM as a lighter-weight alternative to full-featured BMCs. AMI has also built MegaRAC OneTree Community Edition, its own distribution of OpenBMC. "Our OneTree product is OpenBMC at its core, and you can kind of view it as a distribution of OpenBMC. This is a tagged, well-validated version of OpenBMC. We're testing it across multiple silicon vendors, multiple ODM platforms and OEM platforms,” Zachary said.

Customers who need more can still license AMI's proprietary version, which provides long-term direct customer support and SLAs, plus IP packages for advanced needs.

The name OneTree comes from a direct response to a problem Zachary discussed at a recent OpenBMC meetup AMI co-hosted with Meta: the proliferation of firmware forks across silicon vendors and manufacturers, a challenge he calls a five-headed hydra.

Given finite engineering resources, it’s not scalable to continue to work across these multiple trees. “What we've done is take information from those different forks and merge them back into a common tree today,” he said.

Security and Sovereign Data

Security threads through all of it. "We've taken a strong approach to security by putting out advisories to our customers and going through the effort of having third-party audits of our code base," he said, pointing to AMI's annual OCP SAFE audits of its community edition release through the Open Compute Project.

Sovereign data requirements are adding another layer of complexity. As new regional cloud providers emerge to meet local data rules, local players are staring to emerge within these regions, and many are encountering firmware at scale for the first time.

AMI backs its open source strategy with a long-term customer relationship that spans the entire hardware life cycle. "We work pre power on, well before silicon is available, all the way through 10 years to 15 years after launch," Zachary said, positioning AMI as an economy-of-scale partner that spares customers the cost of building their own in-house firmware teams.

For developers who want in, Zachary pointed to AMI's developer portal for MegaRAC OneTree Community Edition, its community portal, upcoming OpenBMC meetups, and the Open Compute Project as entry points. "We were very open to having discussions with customers and industry experts," he added.

Key Takeaway

AMI's open source pivot is a direct response to real fragmentation risk, as multiple forks across silicon paths and rising sovereign requirements threaten to make firmware unmanageable at data center scale. By feeding its OneTree CE distribution back into OpenBMC and layering proprietary IP packages on top where customers need them, AMI is betting that a single, secure, well-audited tree beats a forest of one-off forks. With AI infrastructure evolving from servers to racks to pods to rows to rooms, that bet on convergence looks well timed.

‍