Data center leaf and spine switch configuration

In recent years, with the spread of mobile devices such as smartphones and digital home appliances, data traffic in communication networks is increasing more and more. Along with this, data centers continue to grow in size, and the number of optical transceivers and transmission capacity used in these facilities continues to increase.

 

Figure 1 below shows a schematic representation of part of the network configuration of a data center that is currently mainstream. Currently, the main structure is called leaf & spine type.

Switch connection example for leaf and spine configuration

Fig. 1 Switch connection example of leaf & spine configuration

A rack containing multiple servers has a ToR switch (Top of Rack, or leaf switch) that aggregates the data from these servers.

This ToR switch is mainly connected to the routing switch (spine (trunk) switch) located above it in a mesh pattern as shown in Fig. 1.

If the number of server connection ports is insufficient, leaf switches can be added, and if the switching capacity is insufficient, spine switches can be added, making the configuration highly scalable.

Optical transceiver backward compatibility

Currently, many optical transceivers with a transmission capacity of 100G are used for connections between leaf and spine switches, and as next-generation products, optical transceivers with a transmission capacity of 400G have been standardized and are beginning to appear. .

When adding switches, it is necessary to upgrade the transmission capacity that can be accommodated and backward compatibility with existing switches due to the mesh connection configuration, and the optical transceivers connected to these must be considered. I can say

 

Table 1 below shows the transmission capacities and form factors of optical transceivers, as well as form factor backward compatibility.

transmission capacity

Optical transmission method

form factor

Form factor backward compatibility

10G

1x10G

SFP+

-

25G

1x25G

SFP28

Compatible with SFP+

40G

4x10G

QSFP+

-

100G

4 x 25G (SR4/LR4 etc.)

2 x 50G (BiDi)

1 x 100G (DR/FR/LR)

QSFP28

Compatible with QSFP+

400G

4x100G

QSFP56-DD

Compatible with QSFP+, QSFP28

Table 1 Optical Transceiver Transmission Capacities and Form Factors, and Form Factor Backward Compatibility

For example, the 100G QSFP28 optical transceiver form factor is compatible with 40G QSFP+, and a switch capable of 100G is also designed to support 40G.

In addition, the form factor of the 400G QSFP56-DD optical transceiver has more lanes on the electrical side than the 100G QSFP28, but the specifications are designed with backward compatibility in mind. It is generally possible to deal with

In this way, while upgrading transmission capacity as an optical transceiver, we also consider interconnectivity between switches.

Breakout Solution

For 100G

Now, for example, let's consider a network in which the transmission capacity between switches is 100G, but only some leaf switches have 25G ports.

The 100GBASE-SR4 QSFP28 optical transceiver has a 4x25G parallel transmission on the optical side interface.

Since there is a transmission capacity of 25G per channel, if both the switch and the optical transceiver have the ability to handle 4 channels as independent 25G networks, 4 25GBASE-SR SFP28 optical transceivers can be connected on the opposite side.

This is called 100G Breakout.

100GBASE-SR4 QSFP28 Optical Transceiver Breakout Solution Example

Figure 2 Breakout solution example of 100GBASE-SR4 QSFP28 optical transceiver

For 400G

Similarly, what if the spine switch port was 400G and the leaf switch port was 100G?

400GBASE-DR4 QSFP56-DD optical transceivers are also envisioned to be used for leaf-spine connections, and are considered to be breakout connected to four 100G QSFP28 optical transceivers.

However, it should be noted that the 100G QSFP28 transceiver used in this case should be 100GBASE-DR, not 100GBASE-SR4 as mentioned above.

 

The differences between 100GBASE-SR4 and 100GBASE-DR are shown in Table 2 below.

standard

form factor

Electrical side interface

Optical side interface

100GBASE-SR4

QSFP28

4x25G NRZ

4x25G NRZ

100GBASE-DR

QSFP28

4x25G NRZ

1x100G PAM4

Table 2 Differences between 100GBASE-SR4 and 100GBASE-DR optical transceivers

The 400GBASE-DR4 QSFP56-DD optical transceiver has an optical side interface of 4x100G.

Instead of the widely used NRZ (Non Return to Zero) method, PAM4 (Pulse Amplitude Modulation 4: 4-level pulse amplitude modulation) is used as the modulation method, increasing the transmission capacity per channel. It enables 400G transmission with 4 channels.

 

When used in breakout applications, the opposing 100G optical transceiver must also support this PAM4 system, so it must be a 100GBASE-DR QSFP28 optical transceiver with an optical side interface capable of 100G transmission in one channel. It means that it will not be.

Broadcom Optical Transceiver Solution

Broadcom is mass producing or developing optical transceivers for 100G Breakout and 400G Breakout applications.

 

100GBASE-SR4 QSFP28: AFBR-89CDHZ

25GBASE-SR SFP28: AFBR-735SMZ

400GBASE-DR4 QSFP56-DD: AFCT-91DRDHZ

100GBASE-DR QSFP28: AFCT-89SDHZ

 

この他にも多数製品ラインナップを取り揃えております。

Inquiry

Please visit the Broadcom Transceiver Products website and contact us at:

Broadcom manufacturer information Top

If you want to return to Broadcom manufacturer information top page, please click below.