2023 Showcase

2023 AI and Cloud DC Networking Showcase

AvidThink and Converge Network Digest present our AI and Cloud Data Center Networking Resource Site that examines changes in data center networking to support AI and other cloud workloads! We discuss how the changing workloads are driving technologies and approaches in the space, including the use of SmartNICs/DPUs/IPUs, AI fabrics, and cloud networking. Check out the videos from thought leaders below!

Download the report for free — courtesy of our sponsors! Join the other enterprise IT, CSP, DC operators, and other thought leaders who've already download the report.
Analysis by AvidThink

What the 2023 AI and Cloud DC Networking report finds

AI's explosive growth is straining data center interconnects, and the bottleneck has shifted to the physical and optical layers feeding GPU clusters. Across these interviews, higher-speed optics, chiplets, CXL memory pooling, and open networking software emerge as the levers vendors are pulling to keep AI backend networks scaling.

  1. Optics is the scaling bottleneck

    Brad Booth, OIF's Nathan Tracy, and Marvell's Radha Nagarajan point to bandwidth density as the constraint, with OIF developing 200G electrical and 112G linear optics and Marvell shipping a 1.6T DSP for AI clusters.

  2. 800G to 1.6T backend networks

    Dell'Oro's Sameh Boujelbene projects two-thirds of AI backend ports at 1.6T by 2027 with all at least 800G, while Credo and Multilane push 100G single-lane and 800G interconnect systems.

  3. Chiplets replace monolithic chips

    OCP's Cliff Grossner, Synopsys, Thrace Systems, and ODSA lead Bapi Vinnakota advance an open chiplet economy with electronic data sheets, integration workflows, and open interfaces for heterogeneous AI systems.

  4. CXL pools memory for AI

    Siamak Tavallaei says CXL can interconnect 10,000 units and petabytes of shared memory, while 650 Group's Alan Weckel positions CXL as a competitor to NVLink for AI and ML servers.

  5. Open software and offload

    Broadcom democratizes networking via open-source SONiC with silicon telemetry, Napatech offloads infrastructure to IPUs freeing host CPUs, and Microsoft frames managing the network as an API.

  6. Co-packaged optics and power efficiency

    Broadcom's Near Margalit announces CPO switches clearing key technical hurdles, while Ampere's 192-core One CPUs target efficient inference at 200 to 400 watts.

Download the full report →

Highlights from industry thought leaders

Video interviews

650 Group

Unleashing CXL Tech's Potential in AI & Data Center Evolution

  • 650 Group's Alan Weckel positions CXL to compete with NVLink for enhancing AI and machine learning servers.
  • Growing bandwidth demand requires new and multiple switches per rack, with spending shifting from cloud to AI.
Abstract
Alan Weckel, Founder and Technology Analyst at 650 Group, discussed the potential of CXL technology to compete with NVLink, emphasizing its importance in enhancing servers for AI and machine learning. Weckel also noted the rapidly growing demand for bandwidth, necessitating new high switches and multiple switches per rack, and highlighted the shift in spending from cloud to AI, likening it to the significant architectural shift brought about by cloud technology 20 years ago.
Ampere

Ampere's Vision for Power-Efficient, Large-Scale AI Computing

  • Ampere's Sean Varley stresses optimizing AI models for power efficiency alongside large-scale computing needs.
  • Ampere's 192-core AmpereOne parts deliver high compute for inference at less than 200 to 400 watts.
Abstract
Sean Varley, VP of Product Marketing at Ampere, underscored the importance of optimizing AI models for power efficiency and the need for large-scale computing in the context of AI evolution. He also unveiled Ampere's strategy to create CPUs capable of efficiently managing inference at reduced cost and power, citing the recent launch of their 192-core Ampere One Parts that can provide high computational power at less than 200 to 400 watts.
Arrcus

Data Center Architecture Spanning Edge to Cloud with AI & 5G

  • Arrcus introduces FlexMCN network fabric overlay and SRv6 mobile user plane technology built for SoftBank Mobile.
  • ArcIQ delivers end-to-end network visibility, with 5G as transport for AI/ML while optimizing GPU utilization and TCO.
Abstract
Sanjay Kumar, VP of Products and Marketing at Arrcus, introduced the company's solutions for enhancing AI and 5G capabilities for network operators, including data center solutions — a network fabric overlay FlexMCN and SRv6 mobile user plane (MUP) technology developed for SoftBank Mobile. Kumar also highlights ArcIQ, a solution offering end-to-end network visibility, and discussed the role of 5G as a transport mechanism for AI and ML workloads, emphasizing the company's commitment to optimizing GPU utilization, minimizing latency, and reducing total cost of ownership.
Broadcom

Democratizing Networking with SONiC

  • Broadcom democratizes networking with open-source SONiC, bringing hyperscaler features to all users.
  • Enhanced enterprise-class SONiC and silicon telemetry help customers quickly root-cause network and application issues.
Abstract
Kamran Naqvi, Principal Network Architect at Broadcom, discussed the company's efforts to democratize networking by making features previously exclusive to hyperscalers available to all users, including the Sonic software, an open-source network operating system. Broadcom has enhanced Sonic to be more enterprise-class, leading to increased adoption among enterprise customers, and has enabled silicon telemetry features in its silicon, which can help customers quickly identify the root cause of network or application performance issues.
Broadcom

CPO Switches are Here!

  • Broadcom's Near Margalit says the company has tackled the main technical challenges of deploying CPO switches.
  • Focus now shifts to proving reliability and cost-effectiveness of silicon photonics and shortening time to market.
Abstract
Broadcom's GM and VP of Optical Systems Division, Near Margalit, announced that the company has successfully tackled the main technical challenges of deploying CPO switches and is now focusing on demonstrating the reliability and cost-effectiveness of the core silicon photonics technology. Margalit also addressed concerns about the reliability of the laser component and the time to market, stating that Broadcom is working to shorten the time between the availability of CPO technology for integration with the switch and its market launch.
Credo

The Year AI Skyrockets: Overcoming Bandwidth Bottlenecks

  • Credo Semi's Bill Brennan forecasts 2023 as AI's pivotal year, with interconnect bandwidth as the key bottleneck.
  • Credo targets the AI cluster backend network, driving a global shift to 100 Gig single lane connections.
Abstract
Bill Brennan, President and CEO of Credo Semi, forecasts 2023 as the pivotal year for AI, with the interconnect bandwidth bottleneck in the AI sector driving demand for higher bandwidth and denser networking. Credo Semi is concentrating on resolving this connectivity issue for the AI cluster backend network, which has led to a swift global shift to 100 Gig single lane connections and is expected to hasten the development of next-generation technologies.
Dell'Oro Group

Unveiling the Future of AI: Large-Scale Applications & High-Speed Networks

  • Dell'Oro's Sameh Boujelbene says large AI applications require a new AI backend network and market opportunity.
  • By 2027, two-thirds of backend network ports will be 1.6T and all at least 800 gig.
Abstract
Sameh Boujelbene, Vice President of Dell'Oro Group, highlights the emergence of large AI applications necessitating a new AI backend network, presenting a significant market opportunity. By 2027, two-thirds of these backend network ports will be 1.6T, and all will be at least 800 gig, indicating the industry's growing interest in the AI Network market, which will be further explored in an upcoming AI Network report.
Independent Advisor

The Arms Race for AI Clusters

  • Independent advisor Brad Booth describes AI data centers expanding rapidly as hyperscale and cloud players adopt AI.
  • High GPU and optical module demand is spurring exploration of new materials for optics and memory to meet bandwidth needs.
Abstract
Brad Booth, an independent Ethernet and Optical Technology Advisor, discussed the rapid expansion of AI data centers, driven by AI's increasing capabilities and its adoption by hyperscale and cloud market players. He highlighted the high demand for GPUs and optical modules, the impact on memory devices, and the surge in innovation, including the exploration of new materials for optics and memory elements to meet bandwidth demands.
Juniper

AI/ML Leads to Massive Traffic Across the Network

  • Juniper's Mansour Karam expects the data center market to reach $32 billion in four years, driven by AI/ML workloads.
  • Software must optimize job completion time and auto-tune AI/ML parameters, drawing on Juniper and acquired Apstra expertise.
Abstract
Mansour Karam, VP of Data Center Products at Juniper, discusses the growing data center market, expected to reach $32 billion in four years, and the significant role of AI/ML workloads in this growth. Karam emphasizes the need for software that can optimize job completion time and automatically tune parameters in AI/ML workloads, a solution that Juniper, with its expertise in AI and data feeds, and Apstra, a company Karam founded and Juniper acquired, can provide.
Marvell

Scaling AI Clusters: Optical Connectivity's Key Role in Data Transport

  • Marvell's Radha Nagarajan stresses optical connectivity for managing data transport as AI clusters grow.
  • Marvell's 1.6 DSP for the 1.60 generation is ready to meet demand in AI-centric data centers.
Abstract
Radha Nagarajan, SVP and CTO of Optical and Cloud at Marvell, emphasized the importance of optical connectivity in managing data transport as AI clusters increase in size. With the introduction of a 1.6 DSP for the 1.60 generation, Marvell is ready to meet the growing demand for optical connectivity solutions in AI-centric data centers.
Microsoft

GenAI will Super Charge Network Operators

  • Microsoft's Shawn Hakl sees software-based operations enhancing customer experience and network management efficiency.
  • Managing the network as an API can modernize and monetize networks in innovative ways.
Abstract
Shawn Hakl, VP of 5G Strategy at Microsoft, emphasized the potential of integrating operations community with software-based solutions in networking, which can enhance customer experiences, improve network management efficiency, and enable the scaling of previously complex services. Hakl believes this approach, which involves managing the network as an API, can modernize and monetize the network in innovative ways.
Multilane

Faster Interconnects for AI Data Centers

  • Multilane's Hani Daou cites rising compute needs driving investment in 100 terabit AS6 and 200 gigabit per Lambda optics.
  • Multilane sees high demand for its 800 gig systems and copper interconnect test solutions serving chip vendors and cloud providers.
Abstract
Hani Daou, Business Development Manager at Multilane, emphasized the need for data center upgrades due to increasing compute power requirements, with investments being directed towards technologies like 100 terabit AS6 and 200 gigabit per Lambda Optics. Multilane, currently experiencing high demand for its 800 gig systems and copper interconnect test solutions, is well-positioned to support semiconductor vendors, cloud service providers, and interconnect vendors with a strategic roadmap for R&D investment and bandwidth scaling.
Napatech

Offloading infrastructure to IPUs frees host CPUs, accelerates apps, and enhances security

  • Napatech's Charlie Ashton says IPUs accelerate networking, storage, and security while freeing host CPUs for applications.
  • Napatech's hardware-plus-software IPU solutions offload servers and add workload isolation for greater security.
Abstract
Charlie Ashton, a Senior Director of Business Development at Napatech, discusses how cloud and enterprise data centers need IPUs (infrastructure processing units) to accelerate networking, storage, and security workloads while freeing up host CPUs for applications and improving security through workload isolation. Napatech provides complete IPU solutions comprising hardware and software that deliver high bandwidth and accelerated performance for infrastructure workloads via dedicated processing, offloading host servers and introducing workload separation for greater security.
OIF

Optics Leads the Way for Next Gen Data Centers

  • OIF's Nathan Tracy says machine learning demands higher bandwidth density, straining power and thermal management.
  • OIF is developing 200 Gbps electrical interfaces, a 112 Gbps linear optics spec, and Energy-Efficient Interfaces.
  • OIF also works on CMIS management specs and 400ZR, 800ZR, and 1600ZR for next-gen data centers.
Abstract
Nathan Tracy, President of OIF and technologist at TE Connectivity, underscores the growing significance of machine learning and artificial intelligence in data centers, necessitating higher bandwidth and bandwidth density that will impact power consumption and thermal management. To tackle these issues, OIF is developing a 200 Gbps (Gigabit per second) electrical interfaces, a 112 Gbps linear optics specification, and Energy-Efficient Interfaces (EEI), along with working on CMIS, the common management interface specification, and specifications for 400ZR, 800ZR, and 1600ZR, all crucial for next-generation data centers.
Open Compute Project

Harnessing CXL for AI/ML Interconnectivity

  • OCP's Siamak Tavallaei stresses CXL for interconnecting large machine learning systems with a fundamental interconnect standard.
  • CXL can link 10,000 units, petabytes of storage, and hundreds of terabytes of shared memory for device and host communication.
Abstract
Siamak Tavallaei, CXL Advisor to the Board at Open Compute Project, emphasizes the importance of CXL in enabling interconnection of large systems for machine learning and artificial intelligence, and the need for a fundamental standard for interconnectivity. He asserts that CXL can facilitate the interconnection of 10,000 units, petabytes of storage, and hundreds of terabytes of memory in a shared environment, promoting device-to-device and host-to-host communication.
Open Compute Project

Future of Data Centers: Chiplets, Bandwidth & AI-Intensive Workloads

  • OCP ODSA lead Bapi Vinnakota highlights chiplets' efficiency in building heterogeneous systems for growing bandwidth needs.
  • Chiplet-based systems will be crucial for AI-intensive data centers, with open interfaces as the optimal integration path.
Abstract
Bapi Vinnakota, the ODSA Project Lead at Open Compute Project, emphasized the growing need for more bandwidth and diverse types of compute in data centers, highlighting the efficiency of chiplets in building heterogenous systems. He concluded that chiplet-based systems will be crucial in future data centers, especially those with AI-intensive workloads, and advocated for open interfaces as the optimal way to integrate systems and chiplet devices from various vendors.
Open Compute Project (OCP)

Advancing the Open Chiplet Economy

  • OCP's Cliff Grossner describes standardizations letting companies build a chiplet and publish an electronic data sheet for others' design tools.
  • OCP revealed plans to establish a marketplace for chiplets.
Abstract
Cliff Grossner, VP Market Intelligence and Innovation at Open Compute Project (OCP), discusses the advancements in the open chiplet economy vision, including standardizations that allow companies to build a chiplet and create an electronic data sheet for others to use in their design tools. Grossner also revealed plans to establish a marketplace for chiplets.
Synopsys

What's Next for Chiplets

  • Synopsys highlights cost-effective chiplets over monolithic chips as the industry nears the limits of electronics.
  • That physical ceiling makes a shift to chiplets increasingly likely.
Abstract
Yervant Zorian, Fellow & Chief Architect at Synopsys, highlights the potential of cost-effective chiplets over monolithic chips, suggesting the industry is nearing the limits of electronics, thus making a shift to chiplets more likely.
Thrace Systems

Integrating Chiplets into Complete Systems

  • Thrace Systems, leading the CDX work stream under ODSA, announced an upcoming white paper on chiplet integration workflows.
  • The paper covers all aspects of chiplet integration and will publish on the Open Compute Project website.
Abstract
David Ratchkov, founder of Thrace Systems and lead of the CDX work stream group under ODSA in the Open Compute Project, has announced the upcoming release of a detailed white paper on chiplet integration workflows. The paper will cover all aspects of chiplet integration and will be available on the Open Compute Project's website once released.
Vapor

Zero Gap AI - Vapor.io, SuperMicro, NVIDIA

  • Vapor introduces Zero Gap AI, an end-to-end service simplifying wireless and AI stack deployment.
  • The offering partners with Super Micro and leverages NVIDIA's Grace Hopper superchip and MGX platform.
Abstract
Vapor's CEO Cole Crawford and CMO Matt Trifiro have introduced a new service, Zero Gap AI, designed to simplify the deployment of wireless and AI stacks by offering an end-to-end solution in partnership with Super Micro, leveraging the Grace Hopper super chip and the Nvidia MGX platform.
Whether or not GenAI lives up to its hype and becomes the operating system for all business and consumer software, it will likely be a primary consumer of data center resources over the next five to ten years.