Track Hyper | NVIDIA withstands the performance impact of new AI rival Broadcom

Author: Zhou Yuan / Wall Street News

Recently, Broadcom's performance report for fiscal year 2024, released on December 12, showcased the impressive strength of its AI technology and products, shocking the world.

Due to the market acceptance of Broadcom's ASIC (Application-Specific Integrated Circuit), shares of general-purpose computing chip company NVIDIA fell nearly 7%, hitting a new low since October 18 during intraday trading.

What exactly does Broadcom do?

Broadcom's business primarily covers two major areas: semiconductor technology and infrastructure software, demonstrating a robust diversification characteristic.

In fiscal year 2024, Broadcom's net income reached $51.6 billion, with a gross margin of 76.5%, highlighting remarkable profitability and operational efficiency.

Among them, the semiconductor business generated revenue of $30.1 billion, while the infrastructure software business generated revenue of $21.5 billion, a year-on-year increase of 196%. In the AI field, Broadcom's performance is particularly outstanding, with AI business revenue reaching $12.2 billion, a year-on-year increase of 220%, significantly increasing its share in the semiconductor business, fully demonstrating its competitive strength in the AI era.

Broadcom's overall technical strength includes network chip technology, AI custom chip technology, and interconnect technology. Like NVIDIA, Broadcom also has a unique software and ecosystem layout.

In the era of high-performance computing and AI, network chips are an important pillar of Broadcom's business.

As the world's largest network chip manufacturer, Broadcom's Ethernet product line provides various solutions for AI clusters. For example, the Tomahawk 5 Ethernet switch chip in the Endpoint Scheduled system architecture has achieved an 80-fold bandwidth increase since 2010—from 640 Gbps to 51.2 Tbps in 2022—while reducing energy consumption by over 90%, providing efficient support for data scheduling in small-scale AI clusters.

In the Switch Scheduled architecture, the Ramon and Jericho3 - AI chips enable multi-path interconnection, with Jericho3 - AI capable of connecting up to 32,000 GPUs, each AI accelerator card providing 800 Gbps of data bandwidth, effectively enhancing network performance.

Additionally, Broadcom's Thor series network cards are continuously innovating: Thor 2 is the industry's first 5nm process 400 Gigabit Ethernet NIC device, supporting 16 PCI Express 5.0 lanes, capable of directly driving 5-meter copper cables, and supporting RoCE v2 RDMA technology. Broadcom is also developing faster network card chips to continue leading the development of network chip technology.

In the field of AI custom chip technology, Broadcom is the world's largest AI custom chip service provider, holding about 70% of the $17.5 billion addressable market.

Since entering the customized AI accelerator market in 2014, Broadcom has collaborated closely with customers to develop multiple generations of XPU accelerators.

Broadcom plans to launch the next-generation XPU based on 3nm technology in the second half of 2025, accelerating the development of high-performance computing.

In terms of IP investment, Broadcom is highly forward-looking, having invested approximately $3 billion in the differentiation and innovation of XPU IP, with a broad IP portfolio that includes SerDes IP, AI-optimized NICs IP, cache IP, CPO IP software API, etc., providing critical support for XPU At the same time, Broadcom has also made significant breakthroughs in packaging technology. Its 3.5D eXtreme Dimension System-in-Package (XDSiP) platform technology integrates a large number of chips and HBM stacks, showing significant advantages in interconnection density and power efficiency, effectively enhancing the efficiency of large-scale AI computing and reducing power consumption.

Interconnection technology and related products are also areas of technological advantage for Broadcom.

For example, in the PCIe + Retimer technology, Broadcom has strong capabilities in the PCIe technology field.

What is PCIe?

PCIe (Peripheral Component Interconnect Express) is a high-speed serial computer expansion bus standard used to connect a computer's motherboard to various peripheral devices.

Since the launch of the first generation of PCIe in 2003, Broadcom has shipped over 1 billion PCIe ports. In 2018, it introduced Retimer technology to solve long-distance signal transmission issues, which has since become a standard in the development of PCIe. In March 2024, Broadcom launched the world's first 5nm PCIe Gen 5.0/CXL2.0 and PCIe Gen 6.0/CXL3.1 retimer, providing important support for high-speed interconnection.

In the field of optical interconnection, Broadcom has long been a leader, especially in the achievements of optical interconnection technologies such as VCSEL and EML high-linearity continuous wave lasers. These technologies play a key role in high-speed interconnection for AI and ML systems, with speeds iterating up to 200Gbps.

Currently, Broadcom ships over 50 million high-speed optical communication lasers annually.

Broadcom has launched the industry's first 51.2Tbps CPO Ethernet switch, Bailly, which significantly reduces the operating power consumption of optical interconnections and improves silicon area efficiency compared to pluggable transceivers. Additionally, its 5nm Sian 2 and other products also possess robust electrical and optical interface capabilities.

Furthermore, the 5nm Peregrine and 3nm Condor SerDes technologies developed by Broadcom not only support copper cable connections but also natively support CPO, making them widely applicable across multiple product platforms. This technology has bandwidth capabilities exceeding 100GB, while the best domestic counterparts only offer 2GB of bandwidth.

Broadcom has also achieved remarkable results in infrastructure software, with its revenue accounting for as much as 41.6% of total revenue.

Through a series of strategic acquisitions, such as the acquisition of software giant VMware in 2023, Broadcom has built a relatively complete software business system, further strengthening its competitiveness in the semiconductor technology and infrastructure software markets, providing customers with more comprehensive solutions and forming an efficient collaborative network and software ecosystem.

However, to be fair, the completeness and ease of use of Broadcom's ASIC ecosystem are not as good as NVIDIA's CUDA. The software environment is relatively singular, mainly designed for specific purposes and algorithms, with a high programming difficulty. The development tools and software libraries are also not rich enough, requiring developers to spend more time debugging and developing, which limits its promotion in broader application scenarios Relatively speaking, NVIDIA's GPU products have a fast iteration speed, keeping up with technological development trends and market demands, with new products or performance improvements launched every year.

In contrast, Broadcom's ASIC chips, once designed and manufactured, are difficult to modify, with longer design and manufacturing cycles, making them less suitable for the rapidly changing AI market's frequent technological iterations, which may lead to a lag in technological advancement compared to NVIDIA.

The inability to make adjustments in technology and manufacturing as needed is the main reason why custom chips are more expensive than general-purpose chips.

Additionally, custom chips have another issue: a limited customer base.

Broadcom primarily focuses on providing custom chip services for large technology companies, resulting in a relatively narrow customer group; NVIDIA's GPU products are highly versatile, meeting the needs of businesses of various sizes and application scenarios, with a broad customer base ranging from small startups to large enterprises.

If a major customer that Broadcom relies on makes any technological shifts or business adjustments, Broadcom's performance will significantly suffer as a result.

From this perspective, NVIDIA has a greater advantage in market coverage and customer diversity, allowing it to better respond to market fluctuations and changes in customer demands.

The lack of a full-stack solution is also a shortcoming for Broadcom.

Broadcom mainly focuses on chip design and packaging, lacking a full-stack solution like NVIDIA: customers using Broadcom chips need to solve issues related to network devices, software, and other supporting facilities on their own, increasing the cost and complexity of use for customers.

Perhaps the market has fully recognized Broadcom's shortcomings when facing NVIDIA, which is why after the release of the fiscal year 2024 performance report on December 12, NVIDIA regained its "lost ground" in stock price, slightly rising by 1.70% overall by the close of U.S. stocks on December 23; meanwhile, Broadcom's stock price fell 6.84% from its peak on December 14 to $232.5 on December 23, but its total market value remains above $1 trillion, at $1.09 trillion.

In terms of future prospects, Broadcom is not only a leader in the semiconductor industry but also a key force driving the development of AI technology.

From network connectivity to AI computing power, from semiconductor hardware to software infrastructure, Broadcom's technological puzzle is complete and powerful, providing a solid foundation for the development of the AI industry.

Of course, Broadcom also faces numerous challenges, such as increasingly fierce market competition, requiring continuous investment in research and development to maintain its technological leadership; at the same time, the rapid development of AI technology necessitates that Broadcom continues to innovate to adapt to the ever-changing market demands