Executive perspectives from Google across NextGenInfra's event and research showcases.
2 video interviews · 2024–2025
Google's AI Challenge: Scaling Networks for 100K+ TPU Clusters
- Google projects it will need 448G signaling by 2028 for future TPU generations.
- The 7th-gen TPU Ironwood and successors drive the bandwidth roadmap.
- LLM training clusters exceeding 100,000 TPUs are pushing toward PAM4, PAM6 and PAM8 modulation.
Full summary
Tad Hofmeister, Optical Hardware Engineer for Machine Learning Systems at Google, outlines the bandwidth requirements and technical hurdles for the seventh-generation TPU Ironwood and future TPU generations, highlighting the need for 448G signaling by 2028. He examines the challenges of achieving higher data rates through PAM4, PAM6 and PAM8 modulations to support large language model training that requires over 100,000 TPUs in a single cluster.
Ethernet's Future in AI Networks
- Google's Moray McLaren notes Google uses an Ethernet-based proprietary network in its TPU systems.
- He sees Ethernet potentially dominating scale-out and host networks, but alternatives for latency-sensitive cases.
Full summary
Moray McLaren, Principle Engineer at Google, explores Ethernet's capabilities in addressing interconnect challenges for machine learning networks, highlighting Google's use of an Ethernet-based proprietary network in their TPU systems. He suggests Ethernet's potential dominance in scale-out and host networks for ML applications, while acknowledging the need for alternative solutions in more latency-sensitive architectures.