Two Key Beneficiaries: Why We Think 3D Torus Architecture is Critical to ASIC

In our previous article on DeepSeek, we discussed how ASICs could benefit from lower inference costs and smaller distilled models. Now, let's explore why the 3D Torus architecture and its associated companies might emerge as the hidden winners in the GPU vs. ASIC battle.

In the era of artificial intelligence (AI), computing hardware such as Nvidia GPUs, TPUs, and custom AI ASICs play a fundamental role in training and inference workloads. However, as AI models continue to scale in complexity—requiring trillions of parameters and immense computational power—the efficiency of these hardware accelerators is increasingly bottlenecked by networking technology. Modern AI training and inference workloads often require distributed computing across multiple GPUs or custom accelerators. Therefore, the interconnects between compute nodes, the data transfer speeds, and the overall networking architecture are crucial in determining the performance and scalability of AI systems. This article explores why networking technology is indispensable to AI hardware and how advancements in networking shape the future of AI computing.

Introduction to Scale up and scale out (you may skip this part if you are already familiar with the concepts)

AI workloads can be expanded using two primary approaches: scale-up and scale-out.