The memory wall, once a niche topic for computer architects arguing over cache hierarchies, now shapes how AI data centers are physically built. As large language models move from training into round-the-clock inference, the industry is now pushing against the limits of physics. The problem is how to move enough bytes between memory and logic, and between accelerators, within the loss, noise, power and thermal budgets of copper, optics and power delivery networks.
