The Information, a US media outlet, reported that Nvidia and AI chip startup d-Matrix announced a joint AI chip system. d-Matrix focuses on inference, with a core focus on low-latency, low-power inference acceleration. d-Matrix's technology is memory-centric computing, using 3DIMC to reduce data movement. Currently, the most expensive aspects of AI inference are the massive requests, KVcache (context cache), long texts, and continuous calls to Agents (AIs capable of autonomously breaking down and executing tasks). Therefore, reducing the cost per call allows access to higher-frequency AI applications.
The custom chip market is attracting collective attention from model companies. Today, Zhipu was reported to be evaluating its own chip development, DeepSeek was simultaneously reported to be developing its own inference chip, OpenAI is already working with Broadcom on Jalapeño (an LLM inference chip), and Anthropic is also reported to be discussing its own chip development with Samsung. Model companies are all calculating the same thing: training continues to rely on top-tier GPUs, while inference needs to use ASICs (Application-Specific Integrated Circuits) to reduce cost, latency, and power consumption. This collaboration by Nvidia represents the beginning of integrating inference ASICs into its systems. Companies like Etched, Cerebras, Groq, and d-Matrix have been vying for inference workloads. If Nvidia can integrate these into heterogeneous computing (a hybrid collaboration of GPUs, CPUs, and ASICs), allowing GPUs to handle general computing power and ecosystem control, while dedicated chips handle some inference acceleration, platform stickiness will increase.
Related stocks: Nvidia is the main focus, representing GPUs, CUDA, and AI system platforms; additionally, d-Matrix is currently unlisted, and Broadcom and Marvell Technology are worth watching, representing custom ASICs and high-speed interconnect mapping. Notable risks: The details of the collaboration are not yet fully disclosed, and the mass production, customer verification, and software adaptation of d-Matrix remain to be seen. If the cost advantage of inference ASICs fails to materialize, demand will return to Nvidia GPUs; if it does materialize, Nvidia needs to prove it can maintain profit distribution control in heterogeneous inference systems.
